Desktop Developer
Write Unit Tests With VSTS
VSTS introduces data-driven unit testing; learn how this feature works and how you can take advantage of this with data-driven unit testing in your applications.
- By Jeff Levinson
- 02/01/2006
Technology Toolbox: C#,Visual Studio Team System
Unit testing—the process of testing the smallest possible piece of code that can run—has been around for a long time. Unfortunately, Visual Studio itself has never provided this functionality to developers, and those who wanted to implement this feature had to rely on third-party solutions or build their own unit-testing code and procedures. To be fair, it isn't just the Visual Studio IDE that lacked integrated unit testing. None of the major development environments—and this includes Java IDEs—have integrated unit testing directly into the development IDEs. The Visual Studio Team System (VSTS) edition of Visual Studio 2005 corrects this, integrating unit testing into a cohesive and well thought-out process for creating and running tests.
Taking advantage of this functionality isn't difficult, but it does require that you think through the process carefully; otherwise, you'll create holes in the tests you conduct. I'll show you how to implement unit tests with VSTS, first by creating a small sample app to test with, then walking you through the assorted steps.
First, let's go over the theory that underpins the reasons for implementing unit testing in your applications. Unit-testing theory holds that if all of your methods run correctly, you'll find it much easier to string those methods out into a complete application. When a software system fails, it fails in a method. You don't use unit testing to perform functional testing, system testing, or integration testing. Performing those kinds of tests should be a key part of your overall testing strategy, but unit testing aims to prevent issues from ever reaching those testing stages, as much as possible. You use unit testing to detect flaws at the earliest possible moment once coding has begun. Unit testing uses a variety of tests to determine if a method performs properly.
Unit testing can rely on several different kinds of tests (see Table 1). Note that you must also take into account methods that don't return a value (such as subroutines in VB). For example, you need to be able to determine whether a given test worked, regardless of whether the test produced any values. These kinds of tests require some creativity, and you might find yourself checking values that have nothing to do with the method to get answers to these kinds of solutions. For example, you might make a separate call to your database to retrieve saved values, which can help you determine whether a given test produced the expected result.
For the purposes of the unit-testing walkthrough, use a simple class for creating a Person object (see Listing 1).
Begin Testing
You have a simple class to test with. Now create a new Class Library project called UnitTestingExample and add a class called Person. The purpose of this class is to hold information about a person applying for a position. The first and last name is straightforward, but the DateOfBirth has three business rules you must implement. First, the person cannot be born in the future (this usually indicates a miss-key); second, the person must be more than 18 years old; and third, the person must have been born after the year 1900 (again, this usually indicates an incorrect entry). A person who meets all three criteria is eligible to apply for the position.
Creating a unit test with VSTS (Developer or Tester Edition) is—in the tradition of most things Microsoft—extremely simple. Create the test by right-clicking anywhere in the Person class and selecting "Create Unit Tests? ." Selecting this option displays the Create Unit Tests dialog (Figure 1). Next, select the DateOfBirth, FirstName, and LastName methods to test, then select "Create a new Visual C# (or VB) test project." Click on OK after you select all these elements. When the IDE displays the New Project dialog, enter the new project name as "UnitTestingExampleTests." Note that you should always place different tests in separate test projects.
Performing these steps generates a new project with a class called PersonTests. The project has several important Attributes you should understand before diving into the actual tests (see Table 2). TestClass and TestMethod are the only required attributes you must apply; all of the other methods are optional. Team System creates the method stubs and comments them out when you generate a new test class.
Before examining the generated test methods, take a careful look at the testContextInstance object. This object, an instance of the TestContext type, gives you access to all of the information about the currently running test. Discussing all of the options available through the TestContext object is beyond the scope of this article, but TestContext does allow you access to various methods that allow you to do things such as tracing, data-driven testing (discussed later in this article), locating servers, and other functions.
Examine Your Tests
Examining the specific tests is usually straightforward. For example, consider this test for the FirstName property:
// <summary>
//A test for FirstName
// </summary>
[TestMethod()]
public void FirstNameTest()
{
Person target = new Person();
string val = null;
// TODO: Assign to an appropriate value for the property
target.FirstName = val;
Assert.AreEqual(val, target.FirstName,
"UnitTestingExample.Person.FirstName was not
set correctly.");
Assert.Inconclusive(
"Verify the correctness of this test method.");
}
You decorate this method with the TestMethod attribute, which notes that this is a test. The target object is an instance of the class in which the method you're testing resides. The val variable holds the value that you use to test the method. You assign the value to the FirstName property (note that this is straightforward for a simple property test). Next, call the Assert.AreEqual method to determine whether the value you set FirstName to is the same after it is read. The Assert.Inconclusive line at the bottom of each method is generated to denote that you haven't validated the test method yet. For these reasons, you should treat any results as suspect. You can delete this line after you examine and set up the test properly. Note that if you're testing a method that is private or protected to a class, Visual Studio creates a class accessor that gives you access to the internal methods and members of a class. This is necessary to ensure that all methods within a class can be tested.
You understand how the test method works; next, you need to run a unit test. This requires several steps. Begin by setting the value of val to "Joe" and removing the Assert.Inconclusive line. Next, select (or check) the FirstNameTest test, then click on Run Selected Tests (or Run Checked Tests if you're using the Test Manager).
This runs the test—without issue, one hopes—and displays the results in the Test Results pane at the bottom of the IDE (see Figure 2). During the test run, this pane lists each test, as well as the test's status (Running, Failed, Passed, Inconclusive, or Aborted if you halt the test). When the test is complete, select the "Test run: completed" link to see the results of the test. If anything goes wrong, this link indicates a problem with this message: "Test run: errors."
So far, I've covered how to examine and run a single test. It's important to understand how that works, but in the real world, you have to run many different tests. Each method includes several tests that you might run for it; this adds up to a lot of tests and a lot of testing. For example, I worked on an application recently that had only seven methods, but I ran 38 tests (and probably missed some of them). Writing one test method for each type of test is both time consuming and difficult to maintain. You write enough code as it is; writing another couple of hundred lines to test a five-line method seems like a huge waste of time. This is where data-driven testing can simplify your life greatly.
Use Data-Driven Unit Testing
Data-driven testing lets you use a data store to hold all of the test values for each test. The data store enables Team System to execute the test once for each row in the data set. So, consider the case of the DateOfBirth property. You know that there are at least four tests that you need to run. (There are actually many more, but let's skip those for the sake of brevity). You can use any source for the test, but I'll assume you're using the new SQL Express (or you can use SQL Server, which is far better suited to the testing needs of the team in a "real" team environment).
Right-click on the UnitTestingExampleTests project and select Add | New Item, then select the SQL Database icon to add a new SQL Express data source. Name the database "TestsDatabase." Next, you need to ensure that you can view it in the Server Explorer pane. This requires several steps. Begin by selecting Add Connection from the Server Explorer option. Next, select the Microsoft SQL Server Database File option for the data source, then navigate to the folder that contains the just-added database. Select this database, then click on OK.
You want to simplify the maintenance of your test value tables, so try to create only one table per test when you create tables to store your test data in. A given test might require more tables, but it behooves you to keep extra tables to a minimum. You need to create a series of tests for the DateOfBirth property. These tests validate that each of the exceptions works correctly and that valid values are passed through without triggering an exception.
VSTS provides two methods to check for exception conditions. First, the ExpectedException Attribute listens for an exception during the execution of a test. If an exception occurs, the test passes. You can specify either the type of exception or both the type of the exception and the message the exception generates. This is useful in the current example because the test app throws three application exceptions, but you want to check for a specific exception related to a specific test.
Unfortunately, this approach doesn't work for the tests in this test suite. You can't check for an ExpectedException when using a data-driven test because some of those tests are supposed to pass, and you can't set the message dynamically during run time because the ExpectedException attribute takes data only during design time. To run the tests, you must trap the exception within the method and check for a matching message.
Before you create the table to store the data, you should write the test so you know what types of values you'll need. This enables you to create the table correctly the first time. In this case, update the DateOfBirthTest method so that it matches this value (see Listing 2).
Create the Test Table
Another item of interest: the testContextInstance.DataRow statements. I haven't covered how to build the table for the data source yet, so the column names are notional. But note how this solution uses four columns: the value you want to test, the expected value (for this test, it will be identical to the value in dt_value), the error message to validate against, and an exception message if the test fails. A couple columns remain to be added, but the four just mentioned serve as the key values for the test.
You've created the method to test both valid and invalid values. Next, you want to create the test table. Do this by right-clicking on the Tables node for the TestsDatabase in the Server Explorer, and then selecting Add New Table. The table you create has five columns (see Table 3).
Enter these columns and save the table as "date_birth_tests." The table name describes the type of information it contains. The primary key of this table is a simple identity column to keep track of the rows. The description column allows your tests to be self documenting. In many cases, the description column and the exception column can be the same. You can use a separate description column to provide additional notes on the test (see Table 4).
Note that these tests don't encompass everything you could possibly test. Testing dates can be fairly tricky in certain situations (this isn't one of them). Thus, this code doesn't check data types that are not dates because .NET throws an exception in the test method if you try to pass an invalid format. The example also doesn't check for the time portion of the date, which can cause major problems and be hard to diagnose in some cases. Nor does the example check to see whether a person who just turned 18 works. These are all good test cases in many circumstances.
Once you establish these business rules, you need to hook up the database to the test method. In either the Test View or Test Manager windows, select DateOfBirthTest and select Properties. Click on the ellipsis (?) of the Data Connection String property, set the database type to SQL Server Database File, and browse to the TestsDatabase and click on OK. For the Data Table Name property, select the dropdown and choose date_birth_tests. This adds an attribute to the DateBirthTest method:
[DataSource("System.Data.SqlClient",
"Data Source=.\\SQLEXPRESS;AttachDbFilename=
\"C:\\Documents and Settings\\jxl0575\ My Documents\\Visual Studio 2005\ Projects\\UnitTestingExample\ UnitTestingExampleTests\\TestsDatabase.mdf\";
Integrated Security=True;Connect Timeout=30;
User Instance=True", "date_birth_tests",
DataAccessMethod.Sequential), TestMethod()]
Everything is ready, so select all of the tests in the Test Manager or Test View window and run the tests. The results page should show that three tests ran, but six of the seven tests passed. You might wonder why you received this result, but remember that the Inconclusive test didn't really fail—you simply haven't set it up to run a valid test yet. Double-click on the DateOfBirthTest to drill down on the test results (see Figure 3).
The results indicate that the test ran once for each row in the table. If any test fails, the test itself is marked as failed. All these tests passed, so no error message is shown in this case. The Results window illustrates the reason you use a customized exception method. If the test fails, it enables you to know which test row failed. Yes, you could use the identity column to match up the row number, but that's a hassle. Stating an explicit error message tied to a specific test lets you quickly and easily figure out which tests didn't work in a data-driven test.
That's it for the solution proper. However, I'd like to mention one additional tool that you might find useful: the SQLCMD command-line utility. This utility replaces the Osql tool and is much better for scripted tasks. You should learn this tool well; it will save you a lot of time when prepping databases for tests. I typically use a SQLCMD script in the ClassInitialize method to delete all of the tables in a database, re-create them, and then reload them with known test values. Doing this allows you to perform the same tests over and over again and get the same results.