News

Test Code Coverage Tool Adds Complexity and Trends Analysis

• By Kathleen Richards
• 02/06/2009

The tooling consists of an NCover Explorer UI for developers, a build system console for the build developer and a reporting module aimed at project managers. NCover 3.0 improves performance in areas such as coverage loading. It also integrates branch coverage into the Explorer, improves method visit coverage and adds cyclomatic complexity, a metric that determines software complexity by calculating the number of independent paths in sections of the source code.

Often employed in Agile and test-driven development environments, code coverage analysis allows developers to monitor what percentage of their source code is actually touched by unit tests by measuring how many times a line of code is executed, among other metrics.

Visual Studio users are generally familiar with Microsoft's FxCop, which profiles code against the Design Guidelines for the .NET Framework. The static code analyzer PREfast is part of Visual Studio Team System.

Microsoft also provides Dynamic Link Libraries, which enable developers to build instrumentation and data collection into their build process in the Visual Studio 2008 Team System suite and the 2008 Developer and Test editions. Visual Studio Team System 2010 will add code analysis rule sets, gated check-in and advanced dataflow rules, several aimed at stopping SQL injections.

Test code coverage analysis can often result in confusion rather than conclusions, said Daniel Waldschmidt, NCover's technology evangelist. "Traditionally, code coverage was a number -- that's what it is in Team Foundation Server right now. What does it mean? Why? Numbers have context and the context shows how things really play out."

NCover extends what Visual Studio offers, he said, by providing a broader scope, employing different methodology and offering in-depth report generation. For example, developers can drag and drop Visual Studio coverage files into NCover Explorer and automatically create charts and graphs. NCover 3.0 offers 16 new reports and a "record trends" feature that can be used to automatically create HTML reports.

Visual Studio creates a hybrid build for code analysis. "We don't inject any code -- we do it all live for you," Waldschmidt said. The NCover analysis is done at the runtime layer, enabling the tooling to support any .NET language.

While code coverage analysis can show that unit tests have been executed on significant amounts of source code, it is not indicative of the quality of unit tests. Even achieving close to 100 percent code coverage on new code does not mean there won't be any bugs or security issues. These types of tools, however, do offer dev team managers a measure of how much of the code is actually tested and metrics on how that percentage is changing during the development and production cycles.

Getting developers to write unit tests -- let alone quality unit tests -- is another matter. Microsoft is now offering more tooling to steer developers in that direction with the Microsoft Test framework (MSTest), now available in Visual Studio 2008 Professional in addition to Visual Studio Team System and the emerging ASP.NET Model-View-Controller framework.  

NCover can be used with BullseyeCoverage for native code as well as build and testing tools such as open source NAnt and NUnit in addition to MSTest. The company works closely with Wintellect, a debugging and training consultancy and Microsoft Gold Certified Partner. NCover is in the process of becoming a Visual Studio Industry Partner, according to Waldschmidt.

The commercial product is available in Classic, Complete and Complete Bundle (team) editions. NCover 3.0 Complete, which supports services, Web apps and includes the new trends and reporting features, is $658 for a Professional license. A free community edition is also available.