There's no debate about the impact of a customer-centric organization. Companies that succeed all have one thing in common—they keep their customer at the center of everything they do. World-class teams are constantly evaluating their products and experiences, across all channels and between digital and in-person, and elevating those experiences to meet customers' evolving needs.
As the scope of experiences continues to expand, so does the need for organizations to stay close to their customers. Teams need access to customer insight faster than ever before if they want to keep ahead of the competition. This evolution to a customer-driven process requires fast, reliable feedback from real people at a moment's notice—something traditional, in-lab testing made impossible. Teams can't afford to spend weeks, if not months, planning, coordinating, and executing a few sessions to gather customer feedback on a new product concept, prototype, or brand messaging campaign.
Real human insight is capturing what real people experience in the course of their normal, daily life. Without that in-the-moment insight, products and experiences risk being developed without the customer at the center, and risk failure.
This is where remote testing and feedback become a critical component of any customer-centric organization's process. Yet, despite so many companies being bought-in on the importance of a customer-first mindset, implementing a fully remote testing and feedback strategy may still be a challenge for many teams. In this guide, we'll share some key benefits to incorporating remote testing and feedback into your process as well as actionable tips, examples, and real customer stories to illustrate the impact of remote studies done right.
If your organization is committed to your customer, then remote testing and feedback is an essential solution to incorporate into everything your team endeavors, from branding to messaging to product concepts to in-person vs. digital experiences. Your customers have high expectations that will only get higher. The only way to ensure you're exceeding those expectations is to meet your customers where they are, through remote testing and feedback.
Repeat after me: you are not your customer. That’s why putting yourself in their shoes is arguably one of the most important skills customer experience (CX) professionals should learn (and practice) in order to develop the customer empathy needed to build products and experiences their users need and enjoy. While that may seem like a challenge, remote testing can make it less of a burden.
Through remote qualitative testing, designers, product managers, marketers, and researchers alike will uncover and understand how real people respond to products and experiences. From what they like and dislike, to where they get stuck and confused, to areas of improvement, the valuable insight gathered from these tests are sure to be eye-opening.
Remote testing is an efficient way to capture real people completing real tasks. Here are some of the key benefits:
One major benefit of this type of research is that your test participants aren’t limited to a specific location. This allows you to get the most accurate representation of the user experience—out in the wild.
Some key benefits include:
Some research just can’t be done in a lab. For example, if you’re trying to answer questions about how users interact with a product in a specific context—like a fitness app at the gym—then you’ll need to make sure your research happens in that location. We call that destination-based testing.
There may also be times when you need to observe something other than a digital experience on a computer or mobile device. Tasks, like navigating a theme park or installing a home thermostat, are great practical applications of remote testing.
Additionally, users often don’t complete an activity in one sitting (or through one channel), so it’s important to evaluate your omnichannel experience. That means any interaction a user has with your company while completing a single activity across multiple channels, like websites, apps, and even physical locations.
Probably one of the greatest business benefits of remote testing is that it’s faster and more cost-efficient. Because you’re not burdened with the logistics of in-lab testing, the constraints of travel and scheduling are nonexistent. Moreover, by testing with an insight platform like UserTesting, you have access to a panel of test participants (or as we like to call them, contributors) that allows you to bypass the hassle of recruiting.
Rather than trying to find and compensate a group of target users on your own, you just designate your target audience and UserTesting will recruit users from its panel that match your description. This shortens the testing period to a few hours, rather than days or weeks, allowing you to get feedback and iterate right away.
In summary, remote testing allows you to:
"It's amazing that I can sit in my office and set up a study and get all the results back within 30 minutes. That's about the amount of time it would take me to take a cab to the airport. It's an amazing time saver."
Rachel Edelman
Product Designer, Code for America
Teams are changing the way they work—with more work being done remotely. While working remotely is easier than ever, it requires us to think differently about how we engage with and get feedback from our customers and users.
And customer feedback is critical in this volatile environment. Truly understanding behaviors, motivations, expectations, and needs will give companies the insight they need to pivot business models to meet shifting demands.
As companies respond to the current environment, many are transitioning even more of their customer feedback sessions to remote methods and technologies, and some are introducing new teams to remote approaches in lieu of in-person and face-to-face feedback sessions. Some in the industry are even calling for companies to shift to fully remote research as a way to define their role in “minimizing transmission of the virus.”
To help those who are making or leading the shift to more remote approaches, we’ve created some best practices that may help as you get going:
For those that are used to doing in-person feedback sessions, it’s best to start with interviews over a web conferencing platform, such as UserTesting’s Live Conversation, as they allow you to have real-time conversations over video. Interviewers can interact with customers using verbal and non-verbal queues, ask follow-up questions, probe, and see their reactions—just as they would in-person. Some tips for before, during, and after your session:
You can still get rich feedback from your customers without speaking to them 1:1 and in real-time. Self-guided customer feedback is a great way to reach your customers when it is convenient for them, and it adds efficiency to the team’s workflow and processes, as the sessions are completed while they are focused on other priorities. For example, 80% of UserTesting’s self-guided sessions fill within a few hours. If you’re newer to self-guided, recording customer feedback, here are a few tips to get started:
When gathering customer feedback remotely, teams lose the opportunity to observe in a co-located group session. While this can be a challenge, there are several ways to consume insights via remote technologies:
As your practice of gathering customer feedback moves to more remote approaches, you'll learn best practices that are specific to your industry and team, and you’ll want to build those into your overall strategy as well.
"They [teams] need to know what it's like to be in the shoes of customers. And the only way you can be close to the customer is if you're listening to them and observing them."
Mani Pande
Former Head of UX Research, Cisco Webex
Being customer-obsessed isn’t just a competitive advantage these days, it’s a basic necessity. Having a human-centered mindset at an organizational level is important, and one way teams can validate their efforts is through qualitative studies like moderated and unmoderated testing. While unmoderated and moderated testing can sound intimidating, we’re here to help break it down for you with everything you need to know.
One question that we often hear from customers is which is better: moderated or unmoderated testing? The answer, as you might’ve guessed, is that it depends. Like any research strategy, there are differing benefits for each approach making them more or less suitable depending on what you’re studying and your ultimate research goals.
Remote testing can be conducted using two different methods: moderated or unmoderated. While there are pros and cons to both methods, choosing which is right for you will depend on your specific situation and what your goals are.
We’re about to dig into the pros and cons of each approach, but before we do, it’s important to understand how the two strategies differ.
With moderated testing, a real person will be there to help facilitate (i.e., moderate) the test. The moderator will work directly with the test participant, guiding them through the study, and will be there to help guide and answer questions if the participant encounters any challenges while completing their tasks. What’s great about moderated studies is that they can be conducted either remotely or in-person. One of the major challenges with moderated testing, however, is that it can require a bit more planning upfront since you’ll need to coordinate schedules with participants. It usually requires a designated, quiet location, where you can interact with your participant, free of interruptions.
Unmoderated testing is just like it sounds. It’s not monitored or guided, so there’s no one else present during the study except the participant. The participant completes any tasks and answers questions at their own pace, on their own time, at a time and location of their choosing. Pretty nice, right? Unmoderated testing tends to be faster as participants can complete their tests at any time without any disruption to your daily workflow. Now that we’ve established the differences between the two let’s dive into how and when to best apply each strategy based on your research needs.
It helps to think of moderated user testing as an interview or a real-time conversation that you’re having with a participant or customer. Everything that you’d do to prepare for that type of interaction is what you’ll need to consider for a moderated study.
Moderated testing works best when you need a high level of interaction between you and your participant. For example, if you want to study a prototype with limited functionality, or a complicated process or concept, moderated testing provides you with the interaction you’d need to guide a participant through the study. It's also an excellent way to conduct interviews, understand the customer journey, and discover pain points. Moderated user testing is also a great way to observe body language and pick up on subtle behaviors and responses. It enables you to probe participants for more information if they seem stuck or confused. This type of test allows you to develop a rapport and have a natural conversation with your customers which helps establish trust and can lead to candid feedback that might not be possible with other qualitative research methods.
Because of the additional time and resources, moderated testing does cost more than unmoderated testing. However, you can reduce some of that cost by conducting remote moderated studies, rather than in-person. This will give you the opportunity to connect with participants all around the world and reduce the need to block out a dedicated time and space for onsite interviews. Who doesn’t love getting some hours back in their day?
This type of testing is best for validating concepts and designs quickly with a diverse group of participants. Unmoderated testing works great if you have specific questions that you need to be answered, you need a large sample size, you need feedback quickly, or if you want to observe a participant interacting in their natural environment.
The beauty of remote unmoderated testing is that it can be done anytime, anywhere, and you typically have actionable feedback within a day if not sooner. Because a moderator isn’t needed, the cost is typically much lower than moderated tests, which can enable you to run more tests with a wider variety of participants.
Since unmoderated tests are completely unsupervised, it requires a fixed set of questions and tasks for participants to complete and doesn’t allow for follow up questions or clarification. If participants run into issues, have technical difficulties, or don’t understand the tasks or questions, you won’t have the ability to step in and guide them. At the end of the day, the choice is up to you, but now you can make an informed decision.
"The most valuable part, at least for me, is the conversation the team has after we've done the interviews with the patients. Talking to patients builds empathy for them, and helps us understand the ‘why’ behind whatever we decide we are going to build."
Frank Rosile
Product Development Manager, CoverMyMeds
To keep pace with the changing attitudes, needs, and behaviors of people adapting to the new normal, you’re required to gather insights faster and more frequently than ever. To help with this, we pulled together a list of our top ten remote research strategies to help you uncover valuable insights more quickly while conducting remote unmoderated tests on the UserTesting platform.
Taken together, these strategies will help you pinpoint aha moments so that you can identify the trends and themes you want to understand by watching videos more strategically.
Instead of relying on verbal response questions and listening to entire video sessions for all participants, consider using a combination of tasks, multiple-choice questions, and rating scale questions to structure participants’ time. A little extra time spent upfront planning and structuring the test will help you pinpoint moments you care about quickly—without having to watch a lot of videos to find that needle in a haystack. How this might look:
By providing an expectation for how long you want people to spend on a task, you’ll set the expectation for how they should spend their time and control the amount of time you need to devote to watching videos.
Use rating scale questions to get a quick sense of how people feel, create counts, and pinpoint outlier participants. When you prompt people to speak to their rating, you can then watch an automatic highlight reel to provide more context around the ratings.
These questions enable you to create counts as well and help you understand how participants think or feel about a concept. In addition, you can create an automatic highlight reel to listen to all of the feedback, back-to-back.
In another example, you might consider adding a multiple-choice question that is structured like a quiz question that enables you to gauge whether participants understood the text—again, without having to watch all of the videos.
Now that you have a highly structured test, you can use automatic highlight reels to easily watch all of your participants do the same task or answer the same question back-to-back. The advantage of using this strategy is that themes surface very quickly.
Typically, it’ll take between 5-15 minutes to watch all participants complete a certain task, where the average time each participant took to complete the entire test was 15 minutes. This also means that you don’t need to watch all participants complete the entire test to get quick answers.
Make sure you’re very clear about your research objective and only review the data points and videos from tasks that help you answer that question. Typically, this means that you’ll want to select 1-3 objectives; the more you have, the longer the test will need to be to cover all of them.
Consider framing out your report first so that all you need to do is plug in your findings. This has the added benefit of forcing you to recheck the tasks and questions to make sure they capture the findings you want to share.
A limited set of study objectives doesn’t mean that you have to limit the test to only capture the information related to your current research objectives. On the contrary, consider collecting more information than you need. For example, ask a few questions about needs, behaviors, and perspectives, so you can go back and use the information later for tracking and trending purposes.
In this way, you can create a data set over a series of studies that enables you to get a pulse on what people are thinking, feeling, and wanting over time. This can be an efficient way to gather information that you can use to inform persona research rather than conducting a larger scale study all at once.
Use the visualizations available in the platform to pinpoint specific videos and moments of interest that exemplify what is normal, show why specific participants behaved in unexpected ways, or help you to pinpoint the ‘why’ that you want to focus on.
When you download the Excel spreadsheet, you’ll bring together all of the text data that’s generated from a study. It’s faster to read text than watch videos. In addition, you can create charts and graphs based on the counts you’ve created from metrics or multiple-choice questions. Transcripts are also available in the download. These can be used to pull quick quotes for use in your report.
To develop a sense of themes or trends, you get good data testing 3-5 participants per segment. If you find that you need more information, you can always add more later. By starting with as few as three participants and watching their entire videos, you’ll be able to get a sense of what a larger group might tell you. You can use the “long and deep” (watch the entirety of a smaller number of videos for studies with a smaller number of participants) or “short and broad” (watch single tasks or groups of tasks across all participants).
The UserTesting Professional Services team has developed a number of templates that are available both within the platform and via our help page. They cover a wide variety of objectives, from discovery to validation. Additional templates have been developed that focus on COVID-19. Using these templates as a starting point to develop your own Saved Test Plans that can save you a substantial amount of time as well as ensure that you start with a robust approach.
Think through your availability and workday as you plan when you want to launch tests. We find that launching at the end of the work day means that you can start the following day over a cup of coffee watching videos that spark ideas or next steps that you can share with your team as they also start their days.
Other teams that are working on quick iterations can launch tests first thing in the morning, iterate prototypes in the middle of the day, and launch the next test immediately to get through 2-3 rounds of testing in a day.
Alternatively, schedule launch for a time that people are naturally thinking about or doing the activity that you want to capture. This might be over the weekend, during a special incentive period, or during a holiday.
The video player allows you to speed up the rate of playback up to 2.8x. We find that most people tend to be comfortable with watching videos at 1.5 to 1.8x. This enables you to discern what people are saying as well as watch and compare what they are doing.
While structure can provide you with speed to insights across a typical interaction, another potential strategy to consider is asking a small number of questions, around 3-5, that are more exploratory in nature or a single, very short task.
This approach enables you to get quick answers, but mirror actual behaviors more realistically. Balancing these approaches and integrating them into your workflow can help you to make decisions based on user feedback more frequently.
Uncover human insights that make an impact. Book a meeting with our Sales team today to learn more.