A Quick Guide to Collaborative Usability Testing

Today's post was written by Jerry Cao, UX Content Strategist at UXPin.

Usability testing affects the whole team, so why not involve more people?

Collaborative testing ensures that everyone’s questions get answered, offers fresh perspectives, minimizes bias during the data analysis, and quickens everyone’s access to the results.

Like we described in the free e-book Design Collaboration in the Enterprise, an inviting design culture ensures that you uncover insights (as well as potential objections) as early as possible. That’s why, to make the most out of your user research, try to involve your product team (e.g. PMs and developers) in reviewing your tasks, moderating the sessions, and analyzing the results.

Source: “PE024: Figure 3.3.” Rosenfeld Media. Creative Commons.

In this piece, we’ll cover how to approach usability testing with a collaborative mindset, specifically user research sprints and other methods, and then how to analyze the results as a team.

The Collaborative User Research Sprint

Short, focused, and effective, the user research sprint—first created by Google Ventures—is a four to five day exercise that adapts the Build-Measure-Learn process of startups for iterative design in larger companies.

• Motivation - What motivates our users? What are their pain points?
• Competitive landscape - How do users evaluate products? What other products are in their mind?
• Product adoption - Can users easily learn how to use your product? What would stop them from using it?
• Differentiation - Do users understand your value proposition? What’s the best way for you to explain this?

The product managers and designers at Lyft (valued at $2B as of February 2015) adapted the procedure to cover the essential phases from research to prototyping all within five days. There method follows the days of the week:

1. Monday - Designer and product manager align to features that should be tested. Entire product team reviews the product specifications and existing mockups. Finally, the team defines the user demographic.
2. Tuesday - Create research plan (complete with goals), develop user tasks and follow up questions, and build the interactive prototype. Send test plan to stakeholders.
3. Wednesday - Recruit participants and schedule interviews, and make any must-have changes to the test plan based on stakeholder feedback.
4. Thursday - Finalize the prototype and testing prompt.
5. Friday - Introduce tasks to users and walk through the questions. Record the test and take plenty of notes. Finally, summarize findings and send key insights to all stakeholders.

The research sprint is thorough, efficient, and naturally includes others in the planning and analysis. Google Ventures even recommends a schedule that covers only five days instead of four:

• Day 1 - Recruit participants and draft test plan
• Day 2 - Schedule interviews and prepare interview questions
• Day 3 - Finalize schedules, interviews, and prototypes
• Day 4 - Conduct prototype tests, interviews, and analyze results as a team

This can be further reduced to three days if you implement remote testing. Most remote usability testing tools handle the heavy lifting required for usability research (e.g. recruiting, task creation, and results analysis), freeing up your time to concentrate on only design.

Additional Collaborative User Research Activities

The fast-paced rush of user research sprints isn’t for everyone. There are a variety of research methods that are naturally collaborative, for different reasons suiting different purposes. Let’s take a look at a few below:

• Field Studies - Aside from the benefits this type of study adds to user understanding, field studies are a great team-building activity in that they get a group of people out of the office and observing users in their natural environment. Because developers may notice something different than product managers, the quality of the results is proportionately improved by the teamwork implemented.
• Participatory Design - The idea behind participatory design is, in essence, collaborating with the user. These tests all require the user to co-create designs or provide critical feedback to how the product should operate. Like other forms of collaboration, the best ideas sometimes come from the most unlikely places. For instructions on the different types and how to conduct them, check out the Guide to Usability Testing.
• Heuristic Evaluation - If you’re short on resources, an internal heuristic evaluation can give you valuable usability data while involving multiple team members. Basically, you administer the product to several team-members and they evaluate the design based on a uniform set of pre-determined criteria, such as Jakob Nielsen’s famous 10 Heuristics.

In general, incorporating collaboration in any usability test will yield better results, but these above, plus the user research sprints, really lend themselves to teambuilding.

Analyzing the Results

Clearly, we recommend circulating the results of the study to everyone even remotely involved in the design process.

If you use a common cloud folder, upload the formal report, analysis of data, graphs & figures, and any media, such as the feedback videos discussed above. A summary spreadsheet also works well for non-design stakeholders.

Photo credits: UXPin via Optimal Workshop for Yelp redesign exercise

Try to interpret the results with multiple people. If product managers and developers observed the testing, ask for their interpretation of the results as well. When you invite people who didn’t create the design, you stand a better chance of unbiased interpretation. In fact, as Alla Kholmatova reports, one study of 44 usability practitioners found that a second evaluator increased problem detection by 30-43%.

Once you’re done, you can generate a video clip and use a free usability report template to document your findings for everyone to analyze.

Takeaway

When practicing collaboration during user research, the main instrument is communication. Simply keeping everyone informed of what's happening, and making it clear that their feedback is welcome, will create any number of valuable new ideas or criticisms to make the venture more successful.

Creating a one-page document and distributing it to stakeholders and team-members alike will be a great initial announcement to explain what’s happening, as well as an invitation to receive their opinions. Allowing others to participate or at least observe the tests as well will delve deeper into results that would have otherwise gone unnoticed. And, of course, communicate the results of the test to everyone – the more knowledge even the smallest member of the team has, the more informed their decisions will be going forward.

If you’d like to learn more about design collaboration without committees, check out our free 50+ page e-book Design Collaboration in the Enterprise (Book 1). The book includes practical design advice and analyzed examples from Amazon, Hubspot, and Venmo.