Code Focused

Backup On Build: Source Code Assurance (Part 2)

On VB author Joe Kunk provides a look into the code behind his automated source code backup solution.

• By Joe Kunk
• 06/16/2010

In Part One of this article, I presented the scenario that gave rise to the need for a utility that would back up critical source code at each build. I also presented a list of eight requirements to be satisfied in order for the utility to achieve the desired level of source code protection. In this second and final part of the article, I present the various functional features of the utility. The utility is named BackupOnBuild or BOB for short. The full source code of the utility can be downloaded from the CodePlex site.

Custom Settings and Capabilities for Backup on Build
It was clear from the onset that we needed a utility that would back up the source at each build and that it needed to be fast enough to be transparent to the normal build process. Providing an explicit list of the most important file extensions to be backed up for each project would keep the process tight and fast. Now that the need for custom settings on each backup was clear, the design process centered around the Backup on Build (BOB) settings class that would define the utility's capabilities as shown in Listing 1.

Public Class BOBSettings
    'Properties with default values
    Public Property SourceFolder = "C:\BackupOnBuild"
    Public Property BackupFolder = "H:\Backup\BackupOnBuild"
    Public Property TextLog = True
    'Possible Values:   Always, Never, Monthly, BiWeekly, Weekly, Daily, Hourly, 
    '                   Sunday, Monday, Tuesday, Wednesday, Thursday, Friday, Saturday
    Public Property FullBackupSchedule = "Weekly"
    Public Property SkippedExtensionReport = True
    Public Property CompletedNotification = False
    Public Property CompressBackup = True
    'Suggested Extensions: bmp, cur, doc, docx, ico, txt
    Public Property Extensions = "asax;ascx;ashx;browser;cd;config;css;cs;disco;" +
        "js;jsl;master;mdf;rdlc;resx;rpt;settings;sitemap;skin;sln;suo;vb;vbproj;" +
        "vbs;wsf;xlst;xml;xsd"
End Class

Listing 1. The Backup on Build Settings

Listing 1 shows the BackupFolder set to a network drive so that these backups are still available in case of a hard drive crash on the development computer. Both the SourceFolder and BackupFolder setting support the following tokens to provide access to special folders: {AllUsersApplicationData}, {CurrentUserApplicationData}, {Desktop}, {MyDocuments}, {MyMusic}, {MyPictures}, {ProgramFiles}, {Programs}, and {Temp}. I do not suggest that all of these locations are a great place to store backups, but they are provided to promote flexibility in folder path specification.

Backup on Build will place the backup under the BackupFolder setting location by creating a folder that follows the format of "BOB_mm-dd-yyyy_hh-mm-ss" such as "BOB_06-12-2010_21-50-12". Thus each backup can be identified by the date and time at which it occurred. Had the CompressBackup setting been enabled for the example above, the entire backup would appear in that folder under the single file named "BOB_06-12-2010_21-50-12.zip".

The list of extensions backed up by default is shown in the Extensions property separated by semicolons and with no spaces within the list.

The full backup schedule set to weekly and backs up all files from the source folder path. The schedule intervals are relative to the first use of the BOB utility with the particular project. ALWAYS will perform a full back up at each build, NEVER restricts all backups to only the listed extensions. The first backup of each folder will always be a full backup unless the NEVER schedule is selected. Various other intervals are available including specific days of the week in case that is helpful in your environment.

When true, the SkippedExtensionReport setting will create a text file in the BackupFolder listing each extension found in the source path but not included in the Extensions property. A quick visual review of this text file will indicate any critical file extensions not being backed up.

The CompressBackup utility utilizes the open source SharpZipLib library to compress the entire backup into a single .zip file with essentially a single line of source code. The backup folder and all its contents are then deleted with a call to the DeleteBackupFolderToRecycleBin method that recursively walks up each backup folder to delete the files and subfolders after they have been included in the compressed file. See Listing 2.

            'Zip the backup and delete the backup folders that were zipped, 
            'if indicated by the settings
            If CompressBackup Then
                'Compress the entire backup into a zip file
                Dim zip As New ICSharpCode.SharpZipLib.Zip.FastZip
                zip.CreateZip(BackupPath + ".zip", BackupPath, True, "")

                DeleteBackupFolderToRecyleBin(BackupPath)
            End If

Listing 2. Compressing the backup folder with SharpZipLib

To assist in debugging the utility, the TextLog setting instructs Backup on Build to output a single text file that provides a time stamped text line with full details for each action that modifies anything within the BackupFolder path. The CompletedNotification setting pops up a MessageBox indicating that the backup has completed successfully.

The settings are defined as a separate class file rather than project settings so that a different settings file can be specified for each project. Default values are provided for the settings class properties and may be modified to fit your preferences or can be modified in your favorite text editor once written to disk.

The BOBSettings class includes two shared methods to load or save an instance of BOBSettings class to disk as an xml file. See Listing 3.

    Public Shared Sub Save(ByVal FullPathFileName As String, ByVal Settings As BOBSettings)
        Try
            Dim sr As New IO.StreamWriter(FullPathFileName)
            Dim xs As New Xml.Serialization.XmlSerializer(GetType(BOBSettings))
            xs.Serialize(sr, Settings)
            sr.Close()
            sr.Dispose()
        Catch ex As Exception
            Throw New Exception("BackupOnBuild:BOBSettings:Save:Error: " + ex.Message, ex)
        End Try
    End Sub

    Public Shared Function Load(ByVal FullPathFileName As String) As BOBSettings
        Try
            Dim xmlr As New IO.StreamReader(FullPathFileName)
            Dim xs As New Xml.Serialization.XmlSerializer(GetType(BOBSettings))
            Dim oSettings As BOBSettings = xs.Deserialize(xmlr)
            Return oSettings
        Catch ex As Exception
            Throw New Exception("BackupOnBuild:BOBSettings:Load:Error: " + ex.Message, ex)
        End Try
    End Function

Listing 3. Save or Load the settings class to disk as an xml file

Recursion: It's Me Calling
Recursion is the ability for a routine to call itself. This is required to walk the folders that may be under the original source folder, and to delete the backup files once they have been included in the compressed backup file. The method Backup() sets up the folder paths and calls FolderBackup() once, which processes all files in the FulPathSourceFolder parameter and then calls itself with a modified FullPathSourceFolder parameter value, once for each sub-folder. DeleteBackupFolderToRecyleBin() calls itself with a modified path parameter for each sub-folder as well.

    'Recursive method to backup a specific folder and its sub-folders
    'This method is the heart of the Backup on Build utility
    Public Shared Sub FolderBackup(ByVal FullPathSourceFolder As String,
                                   ByVal FullPathBackupFolder As String,
                                   ByVal FullBackup As Boolean,
                                   ByVal Extensions As String,
                                   ByVal TextLog As Boolean,
                                   ByVal SkippedExtensionsReport As Boolean,
                                   ByRef SkippedExtensions As List(Of String))

        Dim Files = Directory.GetFiles(FullPathSourceFolder)
        Dim Folders = Directory.GetDirectories(FullPathSourceFolder)

        If Not Directory.Exists(FullPathBackupFolder) Then
            Directory.CreateDirectory(FullPathBackupFolder)
        End If

        For Each FileItem In Files
            Dim FileItemInfo = New FileInfo(FileItem)
            Dim FileExt = ""
            Dim CopyFile = False

            If (FullBackup) Then
                CopyFile = True
            Else
                FileExt = New FileInfo(FileItem).Extension.ToLower.Replace(".", "")
                If (Extensions.Contains(";" + FileExt + ";")) Then CopyFile = True
            End If
            If CopyFile Then
                Dim DestFileName = FullPathBackupFolder + "\" + FileItemInfo.Name
                File.Copy(FileItem, DestFileName)
            Else
                'Should we add the extension to the Skipped Extensions list?
                If (SkippedExtensionsReport AndAlso SkippedExtensions.IndexOf(FileExt) = -1) Then
                    SkippedExtensions.Add(FileExt)
                End If
            End If
        Next

        For Each FolderItem In Folders
            Dim FolderItemInfo = New DirectoryInfo(FolderItem)
            Call FolderBackup(
                FullPathSourceFolder + "\" + FolderItemInfo.Name,
                FullPathBackupFolder + "\" + FolderItemInfo.Name,
                FullBackup,
                Extensions,
                TextLog,
                SkippedExtensionsReport,
                SkippedExtensions)
        Next
    End Sub

Listing 4. The primary method FolderBackup calls itself for each sub-folder

Because Backup on Build may run for each project in a solution, it was important to prevent any visible screen interaction unless there was an error or the CompletedNotification setting was enabled. A console application displays a visible black command prompt box so Backup on Build is instead a windows forms application with its only form set to Opacity of 0% and ShowinTaskbar set to False. The Form_Load event launches the backup and exits the utility when completed.

Unit Testing Backup on Build
In order to facilitate automated unit testing, as much source code as possible has been placed into the Business.vb class and the methods were written with input parameters for all needed information.

If you have Visual Studio 2010 Premium or Ultimate, skeleton unit tests can be generated by right clicking on the Solution and adding a Test Project. Once added, right click on the Test Project and choose Add New Test. Select the Unit Test Wizard option and it will populate the selected project with a unit test file for each file in the selected project. Remove the automatically generated tests and unit test files that do not deliver much value, but the remaining files provide a good starting point for coding useful unit tests.

To run the tests for a specific scope, simply select a point in the unit test source file that represents the scope for the tests you want to run, i.e., place the cursor in a single method or at the class level, and press Ctrl-R, T or select Tests / Run / Tests in Current Context. To run all tests, press Ctrl-R, A or select Tests / Run / All Tests in Solution.

If you do not have one of the above versions of Visual Studio, the tests included in the source code can easily be converted to NUnit. Using TestDriven.Net, unit tests can then be run and debugged directly from Visual Studio.

Not all tests are completed in the download source code, but enough are completed to clearly illustrate how it can be done. The remaining tests are left as an exercise for the reader.

Implementing Backup on Build
To implement Backup on Build, copy the three files BackupOnBuild.exe, BackupOnBuild.pdb, and ICSharpCode.SharpZipLib.dll to either a folder listed in your computer's Path environment variable or into a convenient utility folder, in my case, C:\Apps. Modify a copy of the settings.xml file included in the source code download to have your desired settings and place it within your project. Then go to the Compile tab of the project's Properties page. Click the Build Events... button at the bottom and complete the Post-Build event with the path to the utility and the desired settings xml file as shown in Figure 1.

[Click on image for larger view.]

Figure 1. Adding Backup On Build as a Post-build event on successful build.

Enhancing Backup on Build
I wrote Backup on Build with a number of useful features already supported, but time limitations have prevented me from including all the features I imagined. Each backup is done independently by project and there is nothing that ties together the backups of a particular Solution build, other than the very similar DateTime values used in the backup folder names. It would be nice to group together the backups for a single solution build.

It might also be nice to examine the prior backup and backup only those files that actually changed since the last backup. I am sure you can envision other enhancements and I am very curious what those may be; please leave a comment on this article's web page with your suggestions and comments.

The Backup On Build utility can be used to provide an extra measure of protection to your valuable source code. Add it to each project in your application and it will work silently in the background to ensure you always have access to prior source code should it be needed for any reason.

Visit the Backup On Build site on CodePlex and download the source code.