New Age C++

Unit Testing With C++

Unit testing is a fine-grained technique for finding bugs. Here's how to apply it to C++ projects.

• By Diego Dagum
• 03/08/2013

When C++ was created, software testing wasn't as widely practiced as it is today. In most cases, the testing phase started when the software hit the production environment.

This discipline is imperative today. The competition to sell apps is fierce from big players like Google and Apple, to single-developer shops that fight for exposure in marketplaces. That's why it's important to have as few bugs as possible -- they erode your app's reputation, putting it at a disadvantage against so many competitors. The need to keep releasing new features (without breaking anything) makes it more crucial than ever to make tests repeatable and automated.

Test-Driven Development (TDD) is an outgrowth of the classic testing model, evolving as its perception changed from being a drag on efficiency to more of a cost-reduction strategy. Instead of defining and executing tests after writing code, TDD anticipates test definitions prior to development. User stories inspire these tests. Thus, the defined tests drive the development. Now developers focus on coding the simplest thing that passes all tests, instead of architecting flexible but potentially costly APIs. The more complete the requirements (e.g., border conditions and so on), the higher the testing coverage. This is from a domain perspective, as code coverage is actually defined as the percentage of code lines validated by at least one test.

Choosing a Testing Framework
I'll focus on unit testing, which tests components in isolation. There are many C++ unit test frameworks to choose from, including:

• Boost.Test
• CxxTest
• GoogleTest
• CppUnit
• Visual C++
• CUTE
• Unit++
• UnitTest++

There's no best framework for all situations; your app's requirements should determine which one you choose. An old blog post proposed a compelling set of criteria for picking a framework, with decision points like required effort to add new tests, exception and crash handling facilities, support for different outputs and so on. You can take all or some of these and complement them with your own unique needs. For cross-platform development, it's convenient to use a cross-platform testing framework (otherwise you'll have to rewrite your tests.) In the post-PC era, you should pick a framework that supports at least iOS and Android (which, together, have more than 90 percent market share).

Unit Test Common Aspects
For this article, I'll use the lightweight GoogleTest framework to show basic and advanced examples. Other frameworks implement the same concepts in their own way.

A test suite groups test cases around a similar concept. Each test case attempts to confirm domain rule assertions.

// my_stack<T> is a hypothetical stack abstraction which is not
// the one included in the STL.
#include "my_stack.h"
#include "gtest/gtest.h"

// A test suite with four test cases
TEST(my_stack_test_suite, test_for_size) { ... }
TEST(my_stack_test_suite, test_for_elements) { ... }
TEST(my_stack_test_suite, test_for_empty) { ... }
TEST(my_stack_test_suite, test_for_exceptions) { ... }

Once the target code is ready to be tested, it's practical to run all tests, even if some fail. Sometimes, before a failure, completing the rest of the test cases in which it occurred becomes irrelevant.

Most frameworks enable test developers to specify whether a failure should cancel the rest of the test cases or just keep going. It usually makes sense to keep going when assertions work on independent data. Instead, it's better to cancel the test case when the remaining tests are based on data in an inconsistent state, as shown in Listing 1 .

TEST(my_stack_test_suite, test_for_size) {
    my_stack<int> s, t, u;
    t.push(10);
    u.push(20); u.pop();

    // If any of these fails, keeps going
    EXPECT_EQ(0, s.size()) << "Size of stack with no pushes should be 0";
    EXPECT_EQ(1, t.size()) << "Unexpected size";
    EXPECT_EQ(0, u.size()) << "Size of stack with as many pushes as pops should be 0";
}

TEST(my_stack_test_suite, test_for_empty) {
   my_stack s;
   // Anything that fails here cancels the rest of this test
   ASSERT_TRUE(s.is_empty()) << "Stack should be empty when created";
   s.push(10);
   ASSERT_FALSE(s.is_empty()) << "Stack shouldn't be empty if last action was push";
   s.pop();
   ASSERT_TRUE(s.is_empty()) << "Stack should be empty when got as many pushes as pops";
}

A test run shows results like passed vs. failed tests, either onscreen or as a stream (in XML, for example). This latter format can be leveraged by an automated process to reject a code change merge while firing email escalations to the integration leaders.

There are situations where a group of test cases need a similar environment setup before running. A test fixture, like the one in Listing 2, enables a setup definition to be executed before each of its test cases. The fixture can include a teardown function to clean up the environment once a test case finishes, regardless of its results.

// my_db is a hypothetical db proxy
#include "my_db.h"
#include "gtest/gtest.h"

// This class 
class my_db_test_fixture : public ::testing::Test {
    my_db db;
    void restore_initial_data() { ... }

protected:
    // if any exception happens at set up, the test case isn't run
    virtual void SetUp() {
        db.open_connection("..."); // connection string
        restore_initial_data();
    }

    virtual void TearDown() {
        db.close_connection();
    }
}

TEST_F(my_db_test_fixture, test_insert) { ... }
TEST_F(my_db_test_fixture, test_select) { ... }
TEST_F(my_db_test_fixture, test_update) { ... }
TEST_F(my_db_test_fixture, test_delete) { ... }

Robust frameworks enable testing for exceptions, and so should you. Such tests force the conditions that lead to some exception throwing, to test if that has indeed happened. The framework must capture the exception in order to mark the test as passed; the exception isn't allowed to leak, as this would crash the run.

TEST(my_stack_test_suite, test_for_exceptions) {
    my_stack<int> s;
    ASSERT_THROW(s.pop(), pop_in_empty_queue_exception)
        << "Exception expected when popping an empty stack";
    s.push(10);
    ASSERT_NO_THROW(s.pop())
        << "Exception expected when popping an empty stack";
    ASSERT_THROW(s.pop(), pop_in_empty_queue_exception)
        << "Exception expected when popping an empty stack";
}

Some frameworks even enable tests for abnormal program termination (e.g., a purposeful call to std::terminate()). In GoogleTest they're called Death Tests.

Earlier frameworks expected developers to switch tests on and off by manually registering them. Now, registration implicitly happens, while certain tests can be explicitly skipped. For instance, skipping a test makes sense when it fails, but the fix won't come immediately.

// In GoogleTest a test is skipped if its name starts with DISABLED_
TEST(my_stack_test_suite, DISABLED_test_for_exceptions) { ... }

You can also define test templates: these are generic tests that can be later specialized to concrete types. A frequently-faced aspect is the need to test private members. There are many strategies, but a popular one leverages the pImpl idiom, which becomes the true test target.

Finally, unit testing is frequently complemented with mocking. Mock objects are a lightweight implementation of the test target's complex dependencies. Mocking allows the developer to concentrate on coding the test without too much plumbing. Popular mocking frameworks include GoogleMock and Mockator. (I'll cover mocking in a future installment.)

Unit Testing Post-PC Applications
Last month, I discussed how C++ needed complementary technologies for the most important platforms in the emerging post-PC landscape. This includes Objective-C and Cocoa in iOS, Dalvik Java in Android, and so on. This mobile environment requires unit testing too, as some applications may start executing non-C++ logic, then bridge to a C++ component which later sends its results back to the non-C++ layer. A complicated value chain like that increases the possibility of error, making unit testing necessary to detect where issues originate (as opposed to where they show up.)

Another issue is the use of emulators. Don't consider the testing phase done if your tests are validated only in a virtual environment. Although you may be confident that they'll pass muster on the actual device, you should confirm this before going live.

Not all test frameworks work on ARM for all platforms. GoogleTest is included in the Android SDK; this post shows how to use GoogleTest in iOS.

Adopt Unit Testing to Pass All the Tests
Unit testing is one of the practices that make Agile processes viable. These processes encourage collective code ownership to avoid writing components used by just a few developers. This improves the ability to nimbly react to high-priority change needs, but this agility comes at the risk of accidentally introducing new bugs or breaking changes. A rich set of repeatable, automated unit tests will help you find these problems before Murphy does it for you.