Practical ASP.NET

Improve Test Asserts with Shouldly

Augment your testing framework with Shouldly for improved test code readability and better test failure messages.

• By Jason Roberts
• 08/19/2015

Most testing frameworks like xUnit.net and NUnit come bundled with a test assertion API, either as part of the main NuGet package or in an associated package. Whole test suites can be effectively built using these built-in asserts, though by augmenting the chosen testing framework with Shouldly, a couple of additional benefits can be brought to the table:

• Improved test code readability.
• Better test failure messages.

Shouldly is an open source library that has its home on GitHub and is easily installable via NuGet.

It doesn't replace the existing testing framework so, for example, it would be installed into your existing NUnit or xUnit.net test project. Once the NuGet package is installed and the relevant Shouldly using statement is added, asserts can now written in the Shouldly style.

Improved Test Code Readability
One of the benefits of Shouldly is that it can help to improve the readability of test code. It does this in two ways:

• Disambiguates expected and actual values
• Produces fluently readable code

The first of these is related to the fact that some bundled test assertion APIs take two parameters: the expected value of the result of the test, and the actual value that was produced. Take the following code:

[Fact]
public void CalculatorShouldAdd()
{
  var calculator = new Calculator();

  var result = calculator.Add(1, 2);

  Assert.Equal(3, result);
  Assert.Equal(result, 3);
}

Which of the two Assert.Equal statements has the parameters the correct way round?

The same test using Shouldly would look like this:

[Fact]
public void CalculatorShouldAdd()
{
  var calculator = new Calculator();

  var result = calculator.Add(1, 2);

  result.ShouldBe(3);
}

Notice in the preceding code that there is no ambiguity around actual/expected values. Also notice that the Shouldly version reads in more fluent, natural language way.

Better Test Failure Messages
The second benefit of Shouldly occurs when tests fail. Shouldly provides richer, more contextual failure messages that also use the name of the variable being testing.

Assuming the classes in Listing 1 have been defined (that contain bugs).

public class Schedule
{
  public string Title { get; set; }
  private readonly List<string> _scheduledDays;

  public IEnumerable<string> ScheduledDays
  {
    get
    {
      return _scheduledDays;
    }
  }

  public Schedule(string title)
  {
    scheduledDays = new List<string>();

    // BUG: forgot to add: Title = title;            
  }

  public void AddDay(string day)
  {
    // BUG: forgot to add day: _scheduledDays.Add(day);
  }
}

public class Calculator
{
  public int Add(int a, int b)
  {
    // BUG: subtracting rather than adding
    return a - b;
  }
}

The xUnit.net assert tests for the classes in Listing 1 would look like the code in Listing 2.

public class TestsUsingxUnitAsserts
{
  [Fact]
  public void ScheduleTitleShouldBeSet()
  {
    var schedule = new Schedule("Gym Workouts");

    Assert.Equal("Gym Workouts", schedule.Title);
  }

  [Fact]
  public void ScheduleDaysShouldBeSet()
  {
    var schedule = new Schedule("Gym Workouts");

    schedule.AddDay("Monday");

    Assert.Contains("Monday", schedule.ScheduledDays);
  }

  [Fact]
  public void CalculatorShouldAdd()
  {
    var calculator = new Calculator();

    Assert.Equal(3, calculator.Add(1, 2));
  }
}

When run, these tests will produce the following failure messages:

Expected: Gym Workouts
Actual:   (null)

Not found: Monday
In value:  List<String> []

Expected: 3
Actual:   -1

The same set of tests using Shouldly asserts would look like the code in Listing 3.

public class TestsUsingShouldlyAsserts
{
  [Fact]
  public void ScheduleTitleShouldBeSet()
  {
    var schedule = new Schedule("Gym Workouts");

    schedule.Title.ShouldBe("Gym Workouts");
  }

  [Fact]
  public void ScheduleDaysShouldBeSet()
  {
    var schedule = new Schedule("Gym Workouts");

    schedule.AddDay("Monday");

    schedule.ScheduledDays.ShouldContain("Monday");
  }

  [Fact]
  public void CalculatorShouldAdd()
  {
    var calculator = new Calculator();

    calculator.Add(1, 2).ShouldBe(3);
  }
}

When these Shouldly-based tests are run, the following improved failure messages will be output:

schedule.Title
        should be
"Gym Workouts"
        but was
null

schedule.ScheduledDays
        should contain
"Monday"
        but does not

calculator.Add(1, 2)
        should be
3
        but was
-1

Notice in the preceding failure messages you get improved context. Shouldly is telling you the name of the variable that's being asserted on (schedule & calculator); the name of the property that is being asserted on (Title & ScheduledDays); and in the case of the calculator test, Shouldly is even telling you the values of the parameters that were passed to the Add method.

Shouldly does not have to replace all of the asserts in an existing test project -- they can be introduced gradually into existing test suites and can live alongside other tests that use the built-in assert API from the underlying testing framework.