Practical .NET

Introducing the .NET Core Unit Testing Framework (or: Why xUnit?)

• By Peter Vogel
• 11/26/2018

Let me be clear about the new .NET Core testing framework, xUnit: It's an alternative, not a replacement for MSTest, the .NET Core version of the framework you're familiar with from earlier versions of the .NET Framework. Not only is xUnit not a replacement, unlike MSTest, migrating existing .NET Framework test code to xUnit probably isn't an option. Enough syntax is different that migrating existing code to xUnit is probably more work than you'd want to do (a lot of syntax is different).

But, having said that, there are lots of good reasons why you'd want to use xUnit when you create your first new testing project in .NET Core.

Changes and Improvements
Some of the changes that come with xUnit are obvious. As a relatively trivial example, you no longer need the TestClass attribute to flag classes that contain test code: the xUnit framework will find test methods wherever you put them.

Many changes are less benign if you're considering porting existing test code to xUnit. In xUnit, for example, you will usually flag test methods with the Fact attribute rather than TestMethod. That Fact attribute also now absorbs the Ignore attribute, which is now a property called Skip on Fact. As part of enforcing best practices, using the Skip property requires that you provide a reason for skipping a test, which was optional with the Ignore attribute.

The equivalent toTestCategory is an attribute called Trait that requires two parameters instead of one. As I've discussed elsewhere, the legacy ExpectedException attribute isn't a great way to test for exceptions. As a result, xUnit doesn't have ExpectedException but, instead, has Assert.Throws, which is a better solution.

Overall, you'll find some common tests that were obscure or hard to write before are simpler and more obvious in xUnit. To begin with, in general, the Assert class's method names are more to the point (for example, AreEqual is replaced by Equal). That Equal method also now includes a version that will compare two collections to see if they're equivalent. There's also a ThrowsAny method for checking for any Excpetion being thrown, which provides a more obvious test than the equivalent code in MSTest (and xUnit also has DoesNotThrow method to check that a test case doesn't throw some exception). xUnit's Assert class includes an IsAssignableFrom method for those cases where yout want to check if you have the right class or a class that it inherits from.

A Different Architecture
Under the hood, the xUnit architecture is also far more extensible than the other Visual Studio test frameworks. You can create new attributes to control tests or extend the Assert class's Equal, NotEqual, Contains, DoesNotContain, InRange and NotInRange methods to provide custom functionality.

That extensibility has already made possible an alternative to the Fact attribute that flags test methods: the Theory attribute. The Theory attribute supports data-driven testing for methods that only work with a specific set of values: You write a single test method and the Theory attribute lets you run the method multiple times, once for every data value for which the test is applicable.

The major architectural change isn't as obvious, however. With Visual Studio Test, the infrastructure would instantiate your test class and then run all of your test methods. With xUnit, your test class is instantiated, a single test method is run and the class is discarded. The class is then recreated before running the next method. The primary benefit is that your tests are far more isolated than they were before, which is a good thing. Isolating tests ensures that you can run any combination of tests and run them in any order you want without one test affecting the results of another test.

However, it also means that some functionality that Visual Studio Test supports had to be eliminated or rewritten to work in an environment where test classes are constantly being recreated. Specifically, ClassInitialize, ClassCleanup, TestInitialize and TestCleanup are gone in xUnit. Their omission also reflects xUnit authors' belief that using these attributes was, if not bad practices, certainly "not very good" ones.

Obviously, if you agree that those attributes (and ExpectedException) are problematic, you don't have to give up Visual Studio Test/MSTest to get the "benefit" of not using them. However, if you're running an IT department (as I did, once upon a time) you'll feel safer if your team is using a tool that prevents bad practices rather than having to go around to your developers to remind them to "Do this" and "Don't do that."

Still, there may be times when you'd like to share a single context among all your tests (the rationale for ClassInitialize/Cleanup) or create a new test context for each test (the rationale for TestInitialize/Cleanup). I'm not here to say you're wrong.

In fact, I feel your pain: While I was never a fan of ClassInitialize and ClassCleanup, I've used TestInitialize and TestCleanup occasionally. The good news is there are replacements, as I'll discuss in a later column.