In-Depth
Take Advantage of Continuous Integration
Compiling your applications frequently can help you save time, effort, and sleepless nights down the road. Learn how to implement this system using the open source tool, Draco.NET.
Every developer probably has a release-night horror story.
The deadline is fast approaching, and bugs start showing up that weren't there when you tested the subsystems by themselves. Making matters worse, it takes someone 20 minutes to build the project to test your fixes, and the developer who wrote the problematic code is on vacation this week.
Fortunately, it doesn't have to be this way. The practice of continuous integration involves building (and testing) your code early and often. Rather than waiting for an approaching milestone to begin integrating the work of many developers, you complete builds at least daily—ideally right after you make a change. Doing this enables you to isolate and solve problems earlier in the process, finding and eliminating bugs in subsystems when they are introduced, rather than the night before a major release when you finally put all the code together.
Continuous integration, part of the methodology of Extreme Programming (XP), has been around for awhile. Microsoft and many other development teams build their code at least daily, and the daily build is common enough that many tout it as a best practice. Continuous integration just extends and formalizes concepts that have been in use for years.
I'll walk you through how to take advantage of continuous integration in the real world. Unfortunately, you need to step outside the Visual Studio .NET IDE to practice this approach. This article's sample (available here) relies on Draco.NET, an open source tool available for free (with source code) from Chive Software. To use Draco.NET, simply download and run the installer, set up the config file with project-specific information (see the sidebar, Configure Draco.NET for Use), and start the Windows service (see Figure 1).
Before diving into how use this tool, let's take a closer look at what taking advantage of continuous integration entails. Continuous integration involves drastically increasing build frequency, but there are a few prerequisites. First, you need a source code repository. All serious software development benefits from having an easy way of getting the latest version of all the code in the project. Second, your build process should be fully automated. If you do only one build per week, getting all the code manually, compiling each project, and then copying the output to the appropriate location doesn't seem like a big deal. When this process happens several times a day, it requires a dedicated resource just for builds. Finally, automated testing contributes significantly to build quality. Although not strictly necessary, all you gain from continuous integration without an automated suite of tests verifying each build is the quick elimination of any compilation errors. Automated tests allow you to ensure that the code compiles and that it functions correctly.
As projects progress, subsystems evolve to handle both technical challenges and, sometimes, changes in requirements. Often, these changes have unexpected or unintended consequences, both within the subsystem itself, as well as with how each subsystem interacts with other subsystems. In our horror story scenario, these consequences don't become apparent until the deadline is looming, developers are already stressed, and key pieces of the team may be missing.
With continuous integration, subsystems interoperate from the outset, and you can detect problems as you check code in, rather than on release night. Continuous integration also means the developer who used to oversee builds can get back to the business of coding, as checking in code is all that's needed to kick off a fully automated build-and-test cycle.
In practical terms, you can identify and scope bugs more quickly and easily, sparing you the effort and time of tracking them down. When you check in code, you get immediate feedback about how it affected the system as a whole. You know exactly what changed if a build fails; you know exactly where to start debugging if testing fails. A nice side effect of this is that you as a developer become more aware of the general health of the project. Everyone gets an e-mail with the result of the latest build, and each developer can get a rough idea of how things are progressing.
Adjust to Continuous Integration
Adopting continuous integration means making changes both to your coding procedures, as well as how you think about application development. As a developer, you must get used to building and testing the project frequently. You must also get used to checking in code more frequently. Once you stub out a piece of code enough for it to compile successfully, you check it in. If every developer waits until release night to check in all the code, then you haven't gained anything. The whole idea is to make many small changes, instead of a few big ones.
From an organizational standpoint, continuous integration can be adopted on a per-project basis, reducing any potential risk. The first project will have some slight extra overhead related to configuring the build server, but this is a painless process, if done correctly.
At this point, you should have a good idea of what continuous integration looks like from a process perspective. It's time to put the concepts discussed so far into action.
You'll need to meet a few prerequisites to use the open source tool, Draco.NET. First, all of your project code must be under source control. This article assumes you're using Visual SourceSafe, but you can use whatever you prefer. Second, you must be able to build your project automatically. More complex build processes require the use of a tool like NAnt or MSBuild, but this article covers building your solution using Visual Studio.NET automatically. Finally, automated testing is beyond the scope of this article, but both NAnt and MSBuild support invoking NUnit after a successful compile.
Draco.NET is available for download from SourceForge at http://sourceforge.net/projects/draconet. This article uses version 1.5. Draco.NET is distributed in several formats, including a zip file with included source code, and a Microsoft Installer (MSI) file. The latest version, 1.6.0, adds support for Visual Studio 2005 projects and MSBuild, but doesn't work as well with Visual Studio 2003, so this article's sample relies on version 1.5 instead. Download the main MSI file, Draco-Server-1.5.msi. There is also a separate installer for the optional client tools (covered below).
Once you download the file, double-click on it to start the installation wizard. The wizard consists of accepting the license agreement, choosing the installation path for the Windows service, and deciding whether to install the service for the current user only, or for all users of the machine.
When the wizard finishes, the chosen installation path contains a zip file with the source code, a copy of the license agreement, the XSL file used to generate the build results e-mails, and a bin sub-directory containing the Windows service itself and its config file. The service is configured to start automatically on boot, but it isn't started when the wizard completes. You must restart the tool manually once you complete the configuration process.
Now that we know how to configure Draco.NET, let's see it in action.
Build a Sample Project
The sample solution contains two projects: a class library that provides a static method for calculating the dot product of two vectors (represented by integer arrays), and a console application that calls the class library and prints the results. This code implements the CalculateDotP method of the class library, which iterates over the arrays and accumulates the product of each respective position:
public static int CalculateDotP(
int[] v1, int[] v2)
{
int dotP = 0; //init
for(int i=0; i<Math.Min(v1.Length,
v2.Length); i++)
{
dotP += v1[i] * v2[i];
}
return dotP;
}
You can find this solution under Visual SourceSafe at "$/DracoTest/DracoTest" in the sample project. The code's specific functionality isn't important; you're interested only in the build process.
You're ready to set up automatic builds once the project exists on disk and under source control (see Listing 1). You set the poll and quiet periods to low values, so you don't have to wait on Draco.NET once you make your changes.
When setting up your automatic builds for your production projects, choose values that strike a good balance between responsiveness and server load. If your project is large and takes a long time to compile, make sure the poll period is long enough to allow the build to complete. Note that the SMTP server runs locally. This prompts the Windows service to both e-mail results and save the file to a path on disk ("C:\Builds\Output"). Make sure that the specified path exists.
The compilation and source-control sections of the sample are straightforward. You specify the name of your solution file, and the fact that you want the Debug build configuration in the devenv element. In the vss element, enter the path to your project, as well as the username and password you use for SourceSafe access.
All that remains is to start the service. Once started, the service begins writing trace output to the log file (draco.log) in the root of the installation path. If you open that file, you see messages related to checking SourceSafe for any changes to your project. Descriptive errors also appear here if you specify the source-control parameters incorrectly, allowing you to correct the problem.
You know whether SourceSafe is being polled correctly. Next, check out DotP.cs, add a comment somewhere, and then check it back in again. This counts as a change and causes Draco.NET to build the sample solution. It takes roughly a minute (after the polling and quiet periods have elapsed), but Draco.NET gets the latest version of the code and attempts to build it. The configuration for notification includes a file element, so the attempt is logged to "C:\Builds\Output." The file name starts with the module name ("DotProduct"). It also contains the date and time of the attempt.
This build fails because the Windows service is unable to find devenv.exe to compile the code (see Listing 2). To avoid this, you must specify explicitly which version of Visual Studio.NET you're using. Correct this problem by stopping the Windows service, adding the attribute "edition" with a value of "2003" to the devenv element in draco.exe.config, and restarting the service. The build should succeed, now.
Build Output
Draco.NET sends an e-mail to all the notification recipients listed in the config file after the completing each build attempt (see Figure 2).
Now it's time to see what a failed build attempt due to incorrect syntax looks like. Check out the DotP.cs file again and introduce a syntax error by typing the words "bad syntax" on the first line of the file. Next, check the file back in. You can see a dump of the compilation process undertaken by Visual Studio.NET when you receive the build output e-mail. This indicates the file and line number that caused the problem, as well as the type of problem. Everyone in the team has just been notified that you checked in code that doesn't compile, so check out DotP.cs again, fix your error, and check it back in. The build succeeds now, and Draco.NET notifies you of this in an e-mail.
Note that you can change the format of the build results e-mail if it doesn't fit your specific needs. Simply change the modifications.xsl file located in the installation root. You use this XSL file to transform the build-results XML file (in this case located at "C:\Builds\Output") into HTML for the body of the e-mail. Feel free to change this file to include as much or as little information in the build result e-mail as necessary. For example, you might want to send a much simpler e-mail in the case of successful builds that includes only the time of the attempt, and not all of Visual Studio.NET's output and the source code changes.
It is easy to get simple projects up and running with Draco.NET. Some projects, however, require a more involved build process or require a suite of automated tests you must run on each build. As mentioned previously, Draco.NET supports building your project with NAnt instead of Visual Studio.NET. NAnt provides some extra features beyond just compilation, but it is more complex to configure, and you must keep the NAnt build script in sync with your project and solution files. If you want to deploy your code automatically to a testing server (through batch files) or to run unit tests, then NAnt is a good choice. Visual Studio 2005 compiles projects using MSBuild, which also includes support for complex build scenarios. The newest version of Draco.NET supports both building Visual Studio 2005 solutions directly and invoking an auxiliary MSBuild script.
Visual SourceSafe is a common source control choice for .NET projects, but other source controls exist and are supported by Draco.NET. In addition to Visual SourceSafe, Draco.NET supports CVS, PVCS, Subversion, PerForce, and now Vault. Each of these tools has a corresponding element in draco.exe.config with its own sub-elements. If your team uses one of these products, the Draco.NET Web site and the sample config file included in the installation have more information and examples.
Download the Client Tools
You might have noticed that this article has discussed only builds started as the result of source code changes, rather than at scheduled times. If your team needs to build the code at a specific time, you must download and install the Draco.NET Client Tools. Double-clicking on the client tools MSI file (Draco-Client-1.5.msi) starts a wizard similar to the Server installation. Once the wizard completes, the Client Tools installation path includes a basic Microsoft HTML Help file and a bin sub-directory with dracocli.exe in it.
This executable uses remoting to communicate with the Draco.NET Windows service. You can use this tool to start and stop builds manually, check the status of builds, and list all available build names. For example, you can force a midnight build by scheduling a task in Windows that executes dracocli.exe with the appropriate parameters.
This tool uses remoting, so you must configure it through dracocli.exe.config. The URL attribute of the well-known element in this file must correspond to the server, port, and objectUri configuration entered in draco.exe.config on the server. By default, Draco.NET exposes the objectUri "Draco" using the TCP port 8086, so the correct value for the URL attribute is "tcp://servername:8086/Draco," where "servername" corresponds to the name of the server that the Draco.NET service is running on.
Easy, transparent, and repeatable builds are among the hallmarks of a successful development practice. When combined with automated unit testing, you can focus more on code and less on process, while at the same time improving the quality of released code. Frequent integration and testing makes the entire debugging process less painful by narrowing the scope of changes and spreading the effort over the life of the project, instead of clustering it all near the end. This way, you can spend more time sleeping instead of tracking down bugs on release night.