C# Corner

Write Automated UI Tests for iOS and Android Apps

Writing automated UI tests for your iOS and Android apps might not be as difficult as you think. Learn how to set up tests for your apps with Xamarin.UITest.

• By Greg Shackles
• 02/17/2015

Previously in this column, I've written about unit testing iOS apps, but what can you do when unit tests just won't cut it? If you're a Web developer you're probably already familiar with libraries like Selenium, which allow you to drive automated testing of your Web app's front-end.

Xamarin offers a library named Xamarin.UITest that enable the similar possibilities for your iOS and Android apps. In addition to being able to run those tests locally on an emulator or device, you can even push the same tests out to Xamarin's Test Cloud, allowing you to validate your tests across more than a thousand different devices.

Setting Up Your Tests
Getting started is as easy as it gets. Start out by creating a normal .NET 4.5 class library, just like you would for any other test project, and then add the NuGet packages for NUnit and Xamarin.UITest. That's everything you need for your test project! Android apps already have enough accessibility hooks in them out of the box to allow Xamarin.UITest to interact with them, but there's one small thing you need to do in your iOS app to open up the required hooks. In your Xamarin.iOS app, install the Xamarin.TestCloud.Agent NuGet package. Once that is installed, add the following code to the app's AppDelegate class:

#if DEBUG
Xamarin.Calabash.Start();
#endif

When this runs, a small HTTP server will be started inside the app using the open source library Calabash, which allows Xamarin.UITest to interact with the app. This is not something that should be left in builds of the app meant for release, which is why in this example I've surrounded the call with a #if DEBUG directive to make sure it's only included in debug builds.

One of the great things about Xamarin.UITest is that if you set your tests up properly, you can share the exact same tests across both iOS and Android. The one part that needs to be slightly different is where you tell Xamarin.UITest which app to use for this test run. Each platform-specific setup object implements a common IApp interface that contains all the methods you'll want to call in your tests. If tests are written against this interface instead of the platform classes they'll be portable across both platforms.

One pattern I've found useful is to put test builds in known folder that my test fixture can scan and determine which app it is meant to use. That tends to end up looking something like this:

[TestFixture]
public class TipCalculationTests
{
   private IApp _app;

   [SetUp]
   public void SetUp()
   {
      switch (TestEnvironment.Platform)
      {
         case TestPlatform.Local:
            var appFile = 
               new DirectoryInfo("some/path/goes/here")
                  .GetFileSystemInfos()
                  .OrderByDescending(file => file.LastWriteTimeUtc)
                  .First(file => file.Name.EndsWith(".app") || file.Name.EndsWith(".apk"));

            _app = appFile.Name.EndsWith(".app")
               ? ConfigureApp.iOS.AppBundle(appFile.FullName).StartApp() as IApp
               : ConfigureApp.Android.ApkFile(appFile.FullName).StartApp();
            break;
         case TestPlatform.TestCloudiOS:
            _app = ConfigureApp.iOS.StartApp();
            break;
         case TestPlatform.TestCloudAndroid:
            _app = ConfigureApp.Android.StartApp();
            break;
      }
   }
}

For local runs it scans a known folder, chooses the latest file with an extension of .app or .apk, and initializes Xamarin.UITest with it. For TestCloud builds you don't need to specify a file path since it will be provided automatically, so all that's necessary is calling StartApp() and it's good to go. There are more options that can be customized as well, such as specifying a specific device or emulator to run the tests on, but this is the basic set of methods to call to wire up a new test fixture.

Writing Your First Test
With the fixture now in place, let's start writing some tests. For this example I'm going to use an app that is a basic tip calculator. The user can enter their totals, adjust for the percentage they'd like to tip, and the app will show the correct amount to leave as a tip.

When you're first getting started with Xamarin.UITest, the best thing to do is write a quick test that looks like this:

[Test]
public void Repl()
{
   _app.Repl();
}

When the test reaches this method, execution will pause and a command window will pop up. You can use this prompt to interact directly with your app while it's running using the Xamarin. UITest API. This is a great way to explore the UI and determine the right methods to call in your tests. Entering the "tree" command will print out a visual tree of the current screen, which can be invaluable at times. The REPL also has basic autocompletion built in, which makes it much easier to find your way around without having to stop and start the tests over and over.

Let's take a look at a few of the most common methods you'll use to interact with your apps in tests. Many methods in Xamarin.UITest accept a Func<AppQuery, AppQuery> as their argument, which allows for using a nice lambda-based syntax for writing queries. For example, to find an element on the screen marked with an identifier of LoginButton:

_app.Query(e => e.Marked("LoginButton"))
Similarly, to tap on that button:
_app.Tap(e => e.Marked("LoginButton"))

The concept of a "marked" element is important here, especially as you're updating your apps to make them more testable. On Android you can leverage the identifiers you're likely already using in your layouts and Marked() will be able to find them. In some cases on Android you can use Marked() to find an element by its text. In iOS, views have an accessibility identifier you can set that achieves the same thing. Usually you'll want to keep these identifiers consistent across both platforms to help keep tests as portable as possible.

Entering text into a text field is just as simple through Xamarin.UITest:

_app.EnterText(e => e.Marked("EmailAddress"), "greg@gregshackles")

One more useful method is Screenshot(). This becomes particularly important when running tests in TestCloud, as it tells TestCloud to capture an image of the UI right at that moment -- very useful when stepping through test runs. One lesser-known aspect of Screenshot() is that it returns a FileInfo object for the image it just captured. This means you can do anything you like with that screenshot! I've leveraged this for some apps to automate taking screenshots of apps across all the different devices and screen sizes I needed, so Xamarin.UITest isn't only useful for tests! In order to take a screenshot, I just need to provide it with a title:

var imageFile = _app.Screenshot("When I enter my email address")

These are just a few of the methods provided by Xamarin.UITest, but there are many more for performing interactions like swiping, going back, double tapping, etc.

Putting it All Together
With the basics out of the way, let's write a real test for that tip calculator app:

[Test]
public void CalculateTip()
{
   var subTotal = 10M;
   var postTaxTotal = 12M;

   _app.EnterText(e => e.Marked("SubTotal"), subTotal.ToString());
   _app.Screenshot("When I enter a subtotal");

   _app.EnterText(e => e.Marked("PostTaxTotal"), postTaxTotal.ToString());
   _app.Screenshot("And I enter the post-tax total");

   var tipPercent = decimal.Parse(_app.Query(e => e.Marked("TipPercent")).Single().Text) / 100;
   var tipAmount = decimal.Parse(_app.Query(e => e.Marked("TipAmount")).Single().Text.Substring(1));
   var total = decimal.Parse(_app.Query(e => e.Marked("Total")).Single().Text.Substring(1));

   var expectedTipAmount = subTotal * tipPercent;
   Assert.AreEqual(expectedTipAmount, tipAmount);

   var expectedTotal = postTaxTotal + expectedTipAmount;
   Assert.AreEqual(expectedTotal, total);

   _app.Screenshot("Then the tip and total are calculated correctly");
}

This test takes advantage of many of the helper methods introduced in the last section. When the test starts, it enters the values for the totals. It then finds the default tip percentage from the UI, and uses that to determine what the expected tip amount should be if things are working correctly. At each step I use the Screenshot() method to describe that step in a BDD-like fashion. Finally, I assert that the correct tip was calculated using normal NUnit assertions. That's it!

Figure 1 shows what this test looks like when run through Xamarin's TestCloud, where it becomes clear why the titles sent to Screenshot() are so important. Especially as tests become more complicated, these steps provide a great way to quickly see what scenarios are failing in your tests as you run them against different devices.

Running your tests against TestCloud is quite simple, and can even be worked into your continuous integration processes so that you're constantly verifying your apps against every device you care about. Inside the Xamarin.UITest NuGet package installed earlier is a tool named test-cloud.exe that can be used to upload your build and kick off a TestCloud run:

test-cloud.exe submit MyApp.ipa <API key> --devices <device key> --series "master" --locale "en_US" --app-name "TipCalc" --assembly-dir "path/to/test/assemblies" --nunit-xml test-results.xml

When you set up a test in TestCloud's web interface it will actually supply this command too, so don't worry about having to remember the exact syntax. The important part here is that it can be automated this in any way you like, and even output the test results as standard NUnit tests so they can be imported into your CI builds.

All Tapped Out
Now that you know how to leverage the powerful Xamarin.UITest library to write automated UI tests for your iOS and Android apps, we've only scratched the surface of what can be done with these types of tools. The important takeaway is that there is actually very little friction to getting started with them. As you work on your next app I highly encourage you to consider writing some automated UI tests to help make sure you deliver a top-notch and stable experience to your users. The time you ultimately save may be your own!