17 usability testing methods you should know

Even if you’ve conducted hundreds of studies before, getting started on a new usability testing project can be complicated.

Does the product design team need qualitative or quantitative feedback? What if you only have a wireframe or prototype? Is there a right and a wrong time to conduct a study? What if you need feedback quickly? Do you even have time to conduct a study? Who’s going to do it?

There are many ways to conduct usability tests with pros, cons, and scenarios in which they work best. The key to effective research lies in planning. You’re on the right track if you know what you can study, how, and when. This section is for anyone looking for guidance on how to conduct usability testing when to run it, and how to interpret the results.

The only way to find out if it works is to watch people try to use it." - Steve Steve Krug, Author of Don't Make Me Think

The right usability testing methods lead to the right products

Usability testing is instrumental in identifying design flaws that might otherwise go unnoticed. It provides vital insights into how well a design or product functions in real-world scenarios.

The main objectives of usability testing are to determine whether users can complete tasks successfully and independently, assess their performance and mental state during tasks, gauge their overall satisfaction, identify problems and severity, and find practical solutions. By addressing these aspects, usability testing ensures that the final product is user-friendly and meets the needs of its target audience.

"If you don't have time to test it, you don't have time to build it." - Whitney Hess, UX designer and consultant

Usability testing is an iterative process

Maximizing the benefits of usability testing should be done through an iterative design process. This involves planning the test, recruiting participants, facilitating or moderating the testing sessions, and analyzing the results.

The goals of usability testing are to assess user behavior, create a comprehensive test report, and clearly define design issues and best practices. It’s essential to focus on the user’s experience of the design, not just its functionality. By continuously testing and refining the product, designers can ensure it evolves to meet user needs effectively.

WEBINAR: Continuous discovery: transform your product development process

1. Qualitative vs. quantitative usability tests 

Qualitative usability testing focuses on understanding the “why” behind user behavior, providing deep insights into users’ experiences and motivations. This method involves observing users interacting with the product, allowing researchers to gather rich, descriptive data about their actions and thoughts.

On the other hand, quantitative usability testing focuses on the “how many” and “how long” aspects of user behavior. It measures users’ performance on specific tasks, such as the percentage of users who complete a task or the time it takes.

Qualitative and quantitative usability testing are valuable tools in the user research toolkit. Qualitative usability testing is beneficial for identifying usability issues and areas for improvement, while quantitative usability testing is essential for measuring a design's effectiveness.

Remote usability testing, which can be conducted through video conferencing or online testing tools, and unmoderated usability testing, where users complete tasks without a moderator, are also important methods to consider. By combining these approaches, designers can comprehensively understand user behavior and create compelling and enjoyable products.

"Pay attention to what users do, not what they say." - Jakob Nielsen, Usability expert

When to use it 

Use quantitative usability testing methods to validate hypotheses, find answers or opinions, uncover emotions, and dig deeper into user feedback, such as wants, needs, concerns, frustrations, etc.

Use qualitative usability testing methods to learn "the what" of human behavior. This involves interpreting data analytics and yes-or-no responses.

Limitations 

The limitations of quantitative testing methods are that they are often not a statistically representative form of data collection and can require multiple data sessions, which can lead to varying analyses.

The limitations of qualitative testing methods are that it's traditionally time-consuming and expensive, impossible to replicate, challenging to interpret raw data, and analysis is subjective.

2. Unmoderated remote usability testing methods

In unmoderated usability testing methods, users complete pre-written tasks with their device in their location. Participants record their screens and speak their thoughts aloud as they complete the tasks, and the researcher reviews the recorded sessions later.

The researcher doesn’t communicate with the participant in real-time. The researcher may include follow-up questions as part of the study or follow up with the participant after viewing the session.

When to usability test

Remote unmoderated usability studies can be used at any stage of the development process. They are great when you'd like to:

• Gather quick feedback: You can get results within hours since no scheduling is required.
   
• Test with larger sample sizes: Since a moderator doesn't need to be present during each session, you can include as many participants as you like.
   
• Avoid bias: With no moderator present, participants may feel more inclined to be open and honest.
   
• Allow for unique, individualized follow-up questions in real time.
   
• Ensure participants don't get stuck while performing a technically demanding task, such as working with a complex or low-fidelity prototype.
   
• Ensure quiet participants are more vocal since the moderator can prompt them to share their thoughts.

Limitations

• No guidance: If you're testing prototypes that aren't fully functional yet, users might get confused or sidetracked.
   
• Fixed tasks and questions: Participants stick to the script the researcher wrote, so there's no opportunity to dig deeper or change direction like in a moderated usability test.
   
• Less scalable: They require moderator time, which can limit the number of sessions you can reasonably conduct.
   
• Risk of bias: Even an experienced moderator's presence and demeanor can sway participants' actions and responses.

In a remote moderated usability test, a moderator meets virtually with test participants to guide them through a pre-written test. The participants typically think out loud as they complete the tasks. The moderator observes the users as they work through the functions, and the moderator may ask questions to clarify or gather more data.

3. Prototype usability testing

Prototype testing involves sharing prototypes or wireframes with participants to view on their desktop or mobile devices as part of user testing.

Testing your prototype with users lets product design teams iterate based on feedback before any code is written, reducing your risk of an unsuccessful launch. It can be done in person or remotely, with or without a moderator.

When to usability test

Let your participants know that they'll be interacting with a concept rather than a working product so they don't become confused or frustrated if links don't work and interactions aren't fully fleshed out. Remind them to express what they expect will happen next.

Usability testing pro tip

Let your participants know that they'll interact with a concept rather than a working product so they don't become confused or frustrated if links don't work and interactions aren't fully fleshed out. Remind them to express what they expect will happen next.

4. Competitor studies

A competitor study provides insight into what your competitors are doing well and where your competitive opportunities are. In a competitor study, participants interact with one or more of your competitors' products.

You can also have participants complete the same tasks on two products, yours and a competitor's, and compare the experiences. They can be done in person or remotely, with or without a moderator. 

When to usability test

Run a competitor study any time you want to observe the UX of a competitor's product, such as when they launch a new feature or make a big announcement.

These studies can also be particularly helpful when developing a new product or feature. Testing during this phase will help you identify competitive opportunities and advantages before your product is developed.

You can also test the competitor experience at regular intervals to keep a finger on the pulse of the competition.

Usability testing pro tip

While comparing your design to several competitors is tempting, remember that these tests can exhaust participants. To avoid fatigue and confusion, ask each participant to use and evaluate no more than two experiences.

Check out our competitive comparison template here

5. Benchmark studies

Benchmark studies are a series of identical usability tests run at intervals to evaluate how your product’s UX changes over time.

These studies use larger sample sizes than standard usability testing and always include a quantitative element, making quantitative testing particularly useful for measuring task performance and usability metrics. They also use the same core tasks and metrics each time to accurately compare one study to the next.

Our proprietary QXscore is best for tracking UX performance in benchmarking studies. It’s a holistic, easily understood standard for measuring user experience that quantifies users’ attitudes and behaviors into a single score and identifies opportunities to improve.

It can measure user experience progress relative to competitors or across multiple lines of business, digital properties, and products.

When to usability test

Benchmark studies are typically run regularly—monthly, quarterly, or yearly. They can also be helpful before and after rolling out a redesign or new feature.

6. Longitudinal studies

Longitudinal studies collect qualitative data from the same participants on multiple occasions over time. Studies can be run over a few weeks, months, or even years. A longitudinal study will provide valuable insight into how users behave and interact with their surroundings, an experience, or brands over time in their everyday lives.

This study is an excellent tool for understanding how users interact with your product or service over time rather than just a moment in time. It can be done in person or remotely, with or without a moderator.

When to usability test

Longitudinal studies can be done at any time. They can be especially helpful during the ideation phase before product development as a way to understand how users solve a problem without your product. You can also run them after development, which will give you insight into how, when, and why people naturally use your product or service.

Usability testing pro tip

Be sure to let your participants know that they'll be conducting a series of tests, and be clear about the amount of time required to complete the study.

Include a requirement in your instructions that notes participants will be expected to take a specific number of follow-up tests and that the test will last a specific amount of time. This can dramatically reduce the participant drop-off rate for this type of study.

7. Diary studies

A diary study is a longitudinal study where participants create “diary entries” as points of qualitative and quantitative data. These diary entries can be audio, text, images, videos, or a combination of all and are typically combined with other studies to gauge participants’ expectations at the start and conclusion of a more extensive study.

Diary studies can provide valuable insight into how users behave and interact with their surroundings or a product or service in their everyday lives. These studies are most effective when done remotely and unmoderated.

When to usability test

Diary studies can be conducted at any time. They can be done before development, to gather requirements, or after launching a product to gather data on how, when, and why people naturally use it.

8. Card sorting

In a card sorting study, participants sort topics like pages, categories, or pieces of content into related groups. This can provide insight into how people group and label content and information in their minds. This is helpful when designing or assessing an information structure. 

There are two types of card sorts:

• Open card sorts: Users place items in groups and name the groups.
   
• Closed card sort: Users place items into predefined categories.

This type of study is often most effective when done remotely and unmoderated.

When to usability test

An open card sort is typically done in the early stages of the design process or to evaluate an existing design, and the findings can be used to inform the new design or redesign. Closed card sorts can be done before a redesign to see if the existing or proposed structure makes sense to users or when adding new content or functionality to see if it fits within the existing structure.

9. Tree testing

Tree testing is a way for product design teams to evaluate the findability of topics within a website or app. Users are shown a text version of the site or app structure and are asked to indicate where they’d expect to find specific items or topics without the influence of the visual design of the navigation.

Tree testing gives you insight into how easy or difficult it is for users to find essential items within a website or app.

This type of study is also most effective when conducted as remote testing and unmoderated

When to usability test

A tree test is usually conducted before the design phase to evaluate a current structure that will be redesigned or during the early design phase to evaluate a proposed structure.

10. Preference testing

In preference testing, participants interact with and review multiple designs to determine which one they prefer and why. Participants can compare aesthetics, interactions, and content.

If you have more than one design option, this type of test can provide insight into which one stands a better chance of resonating with users. It can be done in person or remotely, with or without a moderator.

When to use it

Preference testing should be conducted in the early design and prototyping phases so that you have plenty of time to iterate without having to spend much time in rework later.

11. A/B and multivariate testing

A/B testing presents each unique user with one of two versions of a design—the A version or the B version—and conversion rates for each version are compared to see which one performs best. A variation of A/B testing is multivariate testing, in which you test three or more design versions.

A/B and multivariate testing are most valuable when only one element differs in each version. For example, using a different call to action for each design approach while keeping everything else the same allows you to draw firm conclusions about why one design performed better.

When to use it

A/B and multivariate testing can only be conducted on a live site or product. Unlike most other methods in this guide, A/B testing is strictly quantitative. Therefore, combining A/B testing with other methods is recommended to understand why users prefer one version.

12. Multichannel studies

In a multichannel study, participants complete the same tasks on more than one channel or device. There are two approaches to a multichannel study:

• Groups of participants complete the same tasks on one channel (e.g., desktop, tablet, or mobile). For example, Group A will complete the study on desktop, Group B on tablet, and Group C on mobile. This option is fast, but users only provide insight on a single channel.
   
• Each participant completes the same tasks on all channels. This option takes longer to recruit for and run. You'll need more time to allow users to set up and switch devices, but it enables participants to compare and contrast the experiences across channels.

This is different from an omnichannel study, where participants complete an activity that spans more than one channel or device, like researching a family vacation on one device and booking it on another. 

These studies can be completed remotely or in person, with moderated testing.

When to use it

Multichannel studies can be conducted at any time, but they're particularly valuable when conducted before any redesign effort.

13. Omnichannel studies

In an omnichannel study, participants complete an activity that spans more than one channel or device. For example, if a customer finds a recipe on their desktop, adds the ingredients to a shopping list on their smartphone, refers to the list on their phone while at the grocery store, and follows the recipe at home on their tablet, they've completed an omnichannel experience.

An omnichannel study is valuable when you want to understand how a single user experiences a brand, company, and organization across touch points.

These studies can be completed via unmoderated testing or in person, with moderated testing.

When to use it

Omnichannel studies can be conducted anytime, but they're particularly valuable before redesign efforts. By doing so, researchers can identify what's working well and what needs to be improved in the existing omnichannel experience.

14. Focus groups

A focus group involves a moderator asking people about their behaviors, preferences, attitudes, and experiences on a topic or item of interest. They can be conducted in person or remotely, and they require a moderator.

These groups allow participants to interact and converse with others while discussing a topic and gathering in-depth details about a group's actions, thoughts, and feelings. Brainstorming leads people to build on each other's ideas, which allows a group to dig into an idea or issue. 

When to use it

Focus groups can be conducted at any time during the life cycle of a product. However, they're typically done before development to gather requirements.

15. Surveys

A survey is a series of questions answered by the target audience. Responses are open-ended or multiple choice, typically about preferences, attitudes, and experiences on a topic or item of interest. The sample size must be large enough to capture statistically significant results to claim a user group.

When to use it

A qualitative or quantitative survey can be conducted anytime during a product's life cycle, but earlier is usually better. It allows you to identify usability issues that you would miss otherwise, leading to extremely expensive usability issues that cost time to resolve later.

16. 1:1 user interviews

1:1 interviews are conducted by asking participants questions and taking note of their feedback while facilitating further discussion. Interviews can provide in-depth details about a user's actions, thoughts, and feelings without presenting a product.

Like a moderated study, 1:1 interviews are often a good idea when probing for more information is necessary to receive in-depth reasoning from the test participant.

This can be done either remotely or in person.

When to use it

User interviews can be conducted at any time during the life cycle of a product. However, they're typically done before development to gather requirements or after the product is launched to gather feedback on how people use it.

17. Guerrilla testing usability testing methods

Guerrilla usability testing is an informal, cost-effective, and quick method for evaluating a product or interface’s usability. It can be conducted in casual, everyday environments (like cafes, parks, or public spaces) rather than in a controlled lab setting. The term “guerrilla” reflects the process's flexible, opportunistic nature.

When to use it

Guerrilla testing is fast, easy to set up, and ideal for teams with limited resources. Testers rely on real users unfamiliar with the product, providing a more natural and honest reaction. This is ideal for getting feedback early on without using a lot of budget or resources.

Limitations

However, there are some clear downsides to guerrilla testing for usability testing. Being in a less controlled environment in a public environment means distractions can easily affect your results. Feedback is less professional, structured, and detailed. Finding a sample of people who meet your target audience can also be difficult.