Practical .NET

Changing the Way You Test with Live Unit Testing

Peter tries out Visual Studio Enterprise 2017's Live Unit Testing and has what might be a life-changing experience.

• By Peter Vogel
• 02/07/2018

I think it's safe to say that I couldn't make a living as a developer without automated testing. Automated testing lets me prove (to myself, at least) that my code is, indeed, making incremental progress towards meeting my client's and my goals. Without automated testing, my client and I would have to count on my own judgment as to the quality of my code … and my judgment tends to be, shall we say, "rose colored."

On top of the benefit of demonstrating actual progress, automated testing also enables me to write better code. The first version of my code is never my best version: I get smarter as I write my code and, as a result, better understand what the right answer is at the end than I do at the beginning. Upgrading my code (refactoring) is, therefore, an essential part of my process -- after I get my code to work, I always refactor that code into a better-engineered answer. But, while refactoring is great, I think anyone who refactors code that works without using automated testing to prove they haven't inserted new bugs is a lunatic.

Live Unit Testing (only available in the Enterprise edition of Visual Studio 2017), offers a kind of continuous testing to go with continuous integration and continuous deployment. Earlier versions of Visual Studio offered a kind of "continuous testing" by executing your tests as you saved your code. I experimented with that feature but never found that it worked well for me. Live Unit Testing not only works better for me, it may even change the way I write code.

Starting Live Unit Testing
To use live testing, you first add a Unit Test project to your solution, as you would normally when using automated tests. No special configuration is required for Live Unit Testing so your existing test projects will work with it as is. I ran my tests using a .NET Core project so I added a Unit Test Project (.NET Core) project to my solution, but Live Unit Testing works just as well the .NET Framework. To start live testing, all you need to do is switch to the edit window with the code you want to test, go to Visual Studio's Test menu and select Live Unit Testing | Start.

Assuming you haven't written any tests yet, after a short wait, a set of blue dashes will appear on the left side of your editing window (you may need to do a build to have Visual Studio find your tests). Hovering your mouse over those dashes will display a Tooltip saying "Covered by 0 test" (obviously, testing for grammar was not part of the QA package for Live Unit Testing). Once you have one or more tests that exercise this set of code, you'll get either a red X (one or more of the tests associated with this code failed) or a green checkmark (all associated tests passed) beside the lines of code executed during testing. If Visual Studio hasn't finished running your tests you'll get the mark that reflects the last result with a little clock icon attached to it indicating that updated results are pending.

If you write your tests in advance, then most of the time when you see the red X, you won't care: You'll still be working on the code and, as a result, you won't be surprised that your test is failing. If, on the other hand, you write your tests after writing your code, you'll find the same blue lines, red Xes, and green checkmarks appearing beside your test code. As you complete either your code or your tests (and those tests pass), you'll get confirmation your code is working as you expected.

Changing My Testing Life
I've primarily been a Test-Driven Development (TDD) kind of guy: I write my tests first and then write the code to pass those tests. Live Unit Testing could change that.

The difference that Live Unit Testing makes is the feedback I get as I write my test code. As I add lines of code to my tests, the green and red checkmarks start appearing indicating whether the related production code has passed the test. I write my code in the standard Arrange, Act, Assert pattern so I initially get green checkmarks beside my Arrange and Act code indicating that I can instantiate and successfully call my production code. When I start writing my Asserts, I start getting red Xes (and occasionally, green checkmarks) indicating that my production code isn't producing the results I expected.

The beauty of this process is that, as I start calling my code under test, I get immediate feedback when I hit some problem code. I liked that. As I finished typing in each Assert, I found myself waiting for the red X or green checkmark by the line (and wondering if it was time to upgrade to a faster computer). I found that if I wrote my tests first, on the other hand, the experience wasn't quite as satisfying. Fundamentally, I wouldn't start getting any feedback that I would regard as useful until I returned a value from my code.

There is, however, at least one benefit in writing your tests first: As you write your production code, any code that has no test associated with it gets a blue dash beside it. I don't know that I'd spot that "untested" code as quickly (or at all) if I'm looking for feedback in my test code, which raises concerns about getting "enough" code coverage. On the other hand, I have to admit that if my code is passing all its tests, I'm less concerned about code coverage than some of my clients are. When I do worry about code coverage, I go to Test | Analyze Code Coverage | All Tests to see how I'm doing … and anything over 80 percent is fine with me. You may have a more stringent standard.

Fixing Your Problems
If you do want to find out why your test has failed, hovering your mouse over the red X in either your test or production code will tell you how many of your tests failed. However, I bet you don't care about the number (I also assume that you also don't care if you have a green checkmark beside your code indicating that you've passed all the related tests): What you want to know is why and how the test failed. To get that information, first click on the red X to display a popup box giving the names of all of the related tests and two links: One to run all your associated tests and one to debug all your related tests.

What you'll be interested in are those tests listed in the dialog box that failed (they're marked with a red X). Hovering your mouse over the name of the test in the dialog box will give you summary information about your test's output. If you're one of those people who, like me, always includes a message in the parameters passed to your Assert methods, that message is included in the summary information.

With that summary information in hand, you may be able to fix your error without any additional work. As you start modifying your code, the red X beside the code will get the clock icon attached to it. When you finish making your changes (and Visual Studio catches up to you), the clock icon will disappear to be replaced, hopefully, with a green checkmark.

However, if the clock icon disappears but the red X isn't replaced, you'll want to do one of two things: Look at your test or debug your code. To look at your test, you only need to double-click on the test in the dialog box to be taken to it.

To debug your code, all you need to do is add a breakpoint and pick the Debug All option in the dialog box. As you're stepping through your code, your error might be in either your test code or your production code. Regardless of where you find your problem, you'll stop debugging and fix your problem. You can either wait for your red Xes change to green checkmarks or, if you're impatient to see the results of your changes, click on the red X, pick Run All, and wait for Visual Studio to catch up.

More Choices
You have some other options. If you want to stop testing you can go to the Test menu and select Live Unit Testing | Pause or Stop. Pick Pause if you intend to restart Live Unit Testing later in your editing session -- you'll start getting your feedback faster with Pause than if you pick Stop and then restart Live Unit Testing later. Picking Stop also throws out any existing test data. There's a third option, Reset Clean, which (like Stop) also throws out your test data but also restarts Live Unit Testing. If you have the Test Explorer pane open, you'll see that all your testing data is still being displayed there so you won't give up your list of passed and failed test and their run times by using Live Unit Testing.

If you find that the number of tests you have is slowing down your editing session, you can exclude a test or a whole test class: Just right-click in a test method or on a test class in Solution Explorer and, from the Live Tests menu choice on the popup menu, choose Exclude . If you click in a test method you can also choose to exclude the whole test class except for the selected test. If you want to try speeding up your feedback, you can go to Tools | Options | Live Unit Testing to control how much of your computer's resources live unit testing takes.

Switching to the Enterprise Edition
So now there are two testing components that aren't available in Visual Studio Professional Edition: The Fakes Framework and Live Unit Testing. I've liked both very much. Do I like both of them enough to upgrade to the Enterprise Edition?

The short answer is "Probably Not": The difference between the annual subscription for the Professional and Enterprise editions is almost $2,500 a year (and that's for the cloud-based versions; the standard versions' price difference is almost $5,000 a year). The only reason I was able to try out Live Unit Testing is that I got a one-year, free subscription to the Enterprise edition through the MVP Reconnect program. But it's probably worthwhile to point out that my wife describes me as "A man so tight that he squeaks when he walks" so your value system may lead you to another decision.

Having said that, though, I probably like Fakes and Live Unit Testing enough that I would make a concerted effort to talk one of my clients into buying the Enterprise edition for me....