Moderated user testing

What’s the difference between moderated and unmoderated user testing? 

When differentiating moderated and unmoderated user testing, it comes down to the depth of insights, guidance, cost, and time. And keep in mind, moderated tests aren’t necessarily better than unmoderated tests and vice versa. Many organizations need both throughout the development process, and factors at play include budget, the development stage, and your research objective(s). 

Depth of insights 

Aside from looking for patterns and insights from what test participants say, their reactions and non-verbal cues offer just as telling detail. Moderated user testing is a great way to observe body language, facial expressions, and subtle behaviors and responses. Some tasks may evoke joy, confusion, or some other emotion worth noting. Observing and developing a rapport with your test participant can help establish trust—and leads to candid feedback that might not be possible with other qualitative research methods.

And while some unmoderated test types also require a contributor to speak aloud and turn on their camera, it might feel less natural compared to a 1:1 or group interview.  Some unmoderated test types also only capture a participant’s voice, if at all, which can limit the number of insights you receive. And with non-verbal test types like card sorting or surveys, results are restricted to one’s answers alone. 

Guidance 

A considerable benefit of moderated tests is that moderators can lead participants back in the right direction. This added guidance can come in handy if a participant is going off-topic with their responses or if they encounter any hurdles with a limited functioning prototype or even a technical error. Moderators aren’t just there to ask questions; they provide a deeper explanation of the tasks to alleviate confusion, clarify any misunderstandings, and help establish trust. 

Since unmoderated tests lack a moderator, this places even more importance on fine-tuning test scenarios and questions and ensuring that participants understand the tasks clearly. To minimize the risk of a participant speeding through the tasks or questions, we recommend reminding participants to slow down or asking them to take a designated amount of time for a certain task. And suppose you anticipate any technical errors or non-functional elements. In that case, it’s always a good idea to tell participants what to do if they encounter any barriers to task completion—whether it’s moving on to the next task or asking them what they expected to happen.

Cost 

Moderated usability testing tends to be more expensive than unmoderated tests, especially if conducted in person, due to the need to find a location to rent and more. However, even virtually, moderated tests require more costs as participants earn a higher rate, which needs to be accounted for in your budget. Contributor payments aren’t the only charge; there’s also the cost of time. Both in-person and remote moderated testing requires more planning and scheduling, and potential cancellations or no-shows can lead to significant project delays. 

Time 

Moderated testing is known to take longer than unmoderated testing, thanks to the lengthier administrative efforts. This test type requires thorough planning and searching to find target participants and a qualified moderator. And if you’re designating a moderator from your own team, researchers will need to allocate time for both conducting the study and analyzing the results after. Meanwhile, individuals need room in their schedules to participate in the study as moderated tests can take up to an hour, while unmoderated tests can be as short as 10 to 20 minutes. Additionally, researchers may receive moderated results back slower. Depending on your time constraints, these factors can play a significant role in which one you choose. 

What are the benefits and drawbacks of moderated usability testing?

The benefits 

Moderated testing is a great way to conduct interviews, understand the customer journey, and discover pain points. This study type works best when you need a high level of interaction between you and your contributor. For example, if you want to study a prototype with limited functionality, or a complicated process or concept, moderated testing provides you with the interaction you’d need to guide a contributor through the study and address any hiccups. It enables you to probe contributors for more information if they seem stuck or confused or are generally less vocal—and minimizes the risk of a contributor speeding through the tasks or questions. This type of test allows you to develop a rapport and have a natural conversation with your customers.

Potential challenges

One of the challenges of moderated usability testing is that it can require a bit more planning upfront since you’ll need to coordinate schedules with contributors and have them commit to the set date and time. It also usually requires a designated, quiet location where you can interact with your contributor, free of interruptions. 

And because of the required additional time and resources, moderated testing can be costly, especially if a contributor were to cancel last minute. However, you can reduce some of that cost by conducting remote moderated studies instead of in person. Opting for a virtual format gives you the opportunity to tap into a global network, requires less administrative work and planning than an onsite interview, and offers more convenience to participants. 

How UserTesting helps top organizations build better experiences with moderated testing 

With UserTesting, you get access to a team of research experts who will help moderate sessions—either for you or with you. Global travel technology organization Expedia Group came to UserTesting to improve efficiency and scale throughout their teams. They found in-lab testing labor-intensive and slow, so they looked to develop a faster, more agile approach to augment the small but expanding team. 

After conducting Live Conversations, with the help of the Invite Network, Expedia Group reduced the time and costs for customer interviews and expanded their audience to other markets like Japan. The organization increased its research team from six to 28 researchers, boosting the customer insights they collected and showing the importance of a customer-centric mindset. 

In another success story, healthcare software organization CoverMyMeds turned to UserTesting to determine how to prioritize patient needs during the prescription authorization process. After conducting weekly Live Conversation interviews, the organization was able to address pain points better, reduce costs, and highlight the importance of UX. This customer success story proved how beneficial Live Conversations can be and validated moderated tests as the right choice.