Quick Summary: There are many reasons to love remote user testing; in part 1 of this 2-part article I explain what I consider to be the 5 most important benefits. In part 2 I’ll explain why sometimes it’s wise to avoid remote testing entirely.
I recently completed a fairly exhaustive user testing project for a large financial firm. For a variety of reasons the client elected to have all the sessions conducted remotely. The project got me to thinking about the circumstances in which remote user testing works well, and those where it doesn’t.
In this two-part article I’ll tackle the pros and cons of remote user testing, starting first with the top benefits of remote user testing. Then I’ll turn the tables and explore some of the important pitfalls, and the reasons remote testing may or may not be a good choice for you.
What’s Remote User Testing?
Remote user testing is the process of conducting research with the respondent located somewhere other than the test moderator and/or anyone else participating (like the client). A typical example is a user test session where the moderator is in one city and the respondent is somewhere else, perhaps in a different time zone or all the way on the other side of the world. In its most basic form, a remote testing session might simply be a telephone call between the moderator and the respondent (basically an interview), but more often it involves the respondent interacting with an application or website while the moderator watches and speaks with the respondent.
Remote testing has enjoyed increasing popularity in recent years, in part because the tools to share screens and applications have matured, and also because the user experience field itself has grown.
Why Remote Testing?
There are a number of reasons to consider testing users without ever seeing them face-to-face. These include:
1. Access to a broader geographic mix of people. If your plans call for testing users in very different locations, remote testing is a huge benefit. Suppose for example you had a product that would be used by customers in the US and UK, and you had reason to believe that users in the two countries would have different needs or use the product in different ways. Or suppose your users in New York have demonstrably different needs from those in Los Angeles. Remote testing makes it possible to gather feedback from geographically diverse sets of users without the need to travel to all of them. Under some circumstances this can be a huge benefit and a substantial savings in time and cost.
2. Elimination of other geographic boundaries. With remote testing, the user experience professional(s) can be in one location, the client in one or more additional locations, and the respondent somewhere else entirely. Everyone can listen in and/or participate in the sessions..
3. Rapid set-up. In most cases, remote testing sessions can be created easily on-the-fly or with little advance notice using nothing more than web-based tools. WebEx for example can quickly begin a new session and generate a conference number (toll free if desired). This makes it possible to schedule test sessions around respondents’ schedules, something that can be very helpful especially if the particular user group is short on time or difficult to schedule. Ever try to schedule a group of doctors or business executives for user test sessions? Remote testing offers a solution to an otherwise thorny problem.
4. Reduced cost. As I alluded to above, remote testing can be less expensive than traditional in-person testing, though this depends on a few factors. If you’re testing remotely there may be no need for a traditional testing facility with its ubiquitous one-way mirrors and jars of peanut M&Ms at the ready. It may also be possible to pay respondents a lower incentive since they’re not required to take the time to drive to a research facility. WebEx and GoToMyPC [others?] offer suitable service plans beginning at $50 per month for unlimited use. By comparison, $50 wouldn’t even cover the cost of the bagels the test facility serves when a client is observing test sessions in person.
5. No special tools required. Remote testing can be accomplished with nothing more than a good Internet connection, a willing respondent, and a screencast/conference service like WebEx or GoToMyPC. Both of these services can record sessions for you, so there’s no need to even use a screen capture application like Camtasia or Silverback. This lower bar to entry means that even garage companies can afford to conduct user testing with representative customers from anywhere in the world. A side benefit to using screencast solutions like WebEx is that many companies are already familiar with such tools, and can therefore attend and/or participate without needing to learn to use software.
So, shouldn’t all usability testing be done remotely?
No, not even remotely (pardon the pun). With the geographic boundaries eliminated, costs lowered, and processes simplified, you might wonder why all user testing isn’t conducted remotely. The answer lies in understanding the significant and sometimes deal-breaking tradeoffs. I’ll cover that in part two of this article, where I’ll switch sides and explain why remote user testing isn’t necessarily everything it’s cracked up to be.