Practical .NET
To Build a Great UI, Test the Experience, Not the Code
Your UI could work exactly the way you intended it to work and still fail during usability testing.
In previous columns, I've talked about the power of personas and user stories in designing a UI. After reading those articles, you may have come away with the impression that UI design isn't a process that's as well defined as, for instance, relational database design. And you'd be right: It's still a process that looks more like a craft than a science.
As a result, there's a very real possibility that you can go through the whole UI design process and still deliver a UI that has issues. Those issues can only be caught through testing. But UI testing is fundamentally different from any other kind of testing -- and different in a way that programmers often miss. Here's why it's different and how to set up useful tests.
End-User Testing
You need to be careful, for instance, that you're testing for the right things. The kind of testing that Visual Studio 2012 provides for UIs isn't going to help here. That testing is intended to prove that the code behind the UI is working correctly. With the kind of testing that Visual Studio supports, "correct" means that the UI is working the way you intended it to work. Unfortunately, the only judges of a UI's correctness are the actual users drawn from the population represented by your personas.
Because usability testing involves end users, it can be confused with "end-user testing." You should use end-user testing to test your code because users are cheap and available, and because users have some understanding of the business, so they're easier to train to do the tests (and their feedback makes more sense than what you'd get from strangers off the street).
In end-user testing, you give end users a script to follow and tell them what the right answer looks like. End users tell you when the right answer doesn't come up. In end-user testing, you're not interested in the user's opinion of the quality of the code, just in the correctness of the results.
Usability Testing
In usability testing, on the other hand, you're keenly interested in the end user's opinion of the quality of the UI: how the UI does its job. Your UI could work exactly the way you intended it to work and still fail during usability testing.
In usability testing, like end-user testing, you ask the user to perform some task that the UI is intended to support. But unlike end-user testing (where you wait for the end user's report), in usability testing you observe the user trying to perform that task. This highlights another difference between usability testing and end-user testing. In end-user testing, the user gets a script; in usability testing no script is provided (providing a script would just prove that the user can read and that the code is executing correctly).
During end-user testing, if the code executes and the user is still unable to carry out a task, you would regard that as a failure on the user's or the script's part. You'd then step in to provide some additional direction to help the user complete the test. During usability testing, if the code executes and the user is unable to complete a task, you'd regard that as evidence of a failure in the UI design. Instead of stepping in, you would measure how long it takes the user to find the right way to complete the task and record what the user does in attempting to complete the task. In usability testing, you only tell the user what to do when you're terminating the test and preparing to test some other part of the UI.
After completing end-user testing, you may revise the scripts you give to your end users to support the next test run. At the end of usability testing, you interview the user to find out what the user expected and to get the details of how the UI succeeded and failed.
It's easy to get end-user testing and usability testing confused and, when that happens, the only thing that gets improved is the code. That's not going to solve your UI problems.