Practical .NET

To Build a Great UI, Test the Experience, Not the Code

Your UI could work exactly the way you intended it to work and still fail during usability testing.

In previous columns, I've talked about the power of personas and user stories in designing a UI. After reading those articles, you may have come away with the impression that UI design isn't a process that's as well defined as, for instance, relational database design. And you'd be right: It's still a process that looks more like a craft than a science.

As a result, there's a very real possibility that you can go through the whole UI design process and still deliver a UI that has issues. Those issues can only be caught through testing. But UI testing is fundamentally different from any other kind of testing -- and different in a way that programmers often miss. Here's why it's different and how to set up useful tests.

End-User Testing
You need to be careful, for instance, that you're testing for the right things. The kind of testing that Visual Studio 2012 provides for UIs isn't going to help here. That testing is intended to prove that the code behind the UI is working correctly. With the kind of testing that Visual Studio supports, "correct" means that the UI is working the way you intended it to work. Unfortunately, the only judges of a UI's correctness are the actual users drawn from the population represented by your personas.

Because usability testing involves end users, it can be confused with "end-user testing." You should use end-user testing to test your code because users are cheap and available, and because users have some understanding of the business, so they're easier to train to do the tests (and their feedback makes more sense than what you'd get from strangers off the street).

In end-user testing, you give end users a script to follow and tell them what the right answer looks like. End users tell you when the right answer doesn't come up. In end-user testing, you're not interested in the user's opinion of the quality of the code, just in the correctness of the results.

Usability Testing
In usability testing, on the other hand, you're keenly interested in the end user's opinion of the quality of the UI: how the UI does its job. Your UI could work exactly the way you intended it to work and still fail during usability testing.

In usability testing, like end-user testing, you ask the user to perform some task that the UI is intended to support. But unlike end-user testing (where you wait for the end user's report), in usability testing you observe the user trying to perform that task. This highlights another difference between usability testing and end-user testing. In end-user testing, the user gets a script; in usability testing no script is provided (providing a script would just prove that the user can read and that the code is executing correctly).

During end-user testing, if the code executes and the user is still unable to carry out a task, you would regard that as a failure on the user's or the script's part. You'd then step in to provide some additional direction to help the user complete the test. During usability testing, if the code executes and the user is unable to complete a task, you'd regard that as evidence of a failure in the UI design. Instead of stepping in, you would measure how long it takes the user to find the right way to complete the task and record what the user does in attempting to complete the task. In usability testing, you only tell the user what to do when you're terminating the test and preparing to test some other part of the UI.

After completing end-user testing, you may revise the scripts you give to your end users to support the next test run. At the end of usability testing, you interview the user to find out what the user expected and to get the details of how the UI succeeded and failed.

It's easy to get end-user testing and usability testing confused and, when that happens, the only thing that gets improved is the code. That's not going to solve your UI problems.

comments powered by Disqus


  • What's New in Visual Studio 2019 v16.5 Preview 2

    The second preview of Visual Studio 2019 v16.5 has arrived with improvements across the flagship IDE, including the core experience and different development areas such as C++, Python, web, mobile and so on.

  • C# Shows Strong in Tech Skills Reports

    Microsoft's C# programming language continues to show strong in tech industry skills reports, with the most recent examples coming from a skills testing company and a training company.

  • Color Shards

    Sharing Data and Splitting Components in Blazor

    ASP.NET Core Version 3.1 has at least two major changes that you'll want to take advantage of. Well, Peter thinks you will. Depending on your background, your response to one of them may be a resounding “meh.”

  • Architecture Small Graphic

    Microsoft Ships Preview SDK, Guidance for New Dual-Screen Mobile Era

    Microsoft announced a new SDK and developer guidance for dealing with the new dual-screen mobile era, ushered in by the advent of ultra-portable devices such as the Surface Duo.

  • How to Create a Machine Learning Decision Tree Classifier Using C#

    After earlier explaining how to compute disorder and split data in his exploration of machine learning decision tree classifiers, resident data scientist Dr. James McCaffrey of Microsoft Research now shows how to use the splitting and disorder code to create a working decision tree classifier.

.NET Insight

Sign up for our newsletter.

Terms and Privacy Policy consent

I agree to this site's Privacy Policy.

Upcoming Events