Testing at every stage of development

It’s widely known that a majority—up to 80%—of new products fail. The reasons are manifold. Perhaps the new product couldn’t oust a longtime customer favorite. Maybe the new product was aesthetically wonderful but was also so hard to use that everyone gave up. Or maybe, despite being a superior product, the marketing and go-to-market failed to compel.

Despite the best-laid plans and substantial investments of time and resources, failure to (successfully) launch happens all the time.

While there’s always some risk in new endeavors, you can hedge your bets by ensuring that you understand customer expectations at every step of your development process. Doing so allows you to identify and remedy small problems along the way before they have the chance to become critical issues down the line.

This guide is organized with tips and customer examples so that you understand how and why to test during:

• Early-stage discovery
• Prototyping
• Development and pre-launch 
• Post-launch

Early stage discovery

During the discovery phase, you’re looking to better understand your customers. This can entail becoming acquainted with different personas or understanding high-level pain points. At this stage, you may not have a product idea or an endpoint in mind. You’re at a starting stage and in search of clarity on areas of opportunity.

Another idea: Get feedback on initial concept ideas before moving forward with any designs.

Self-guided test

While it’s better to do a live interview for the discovery phase (it offers the opportunity to ask follow-up questions or pursue new areas and topics as the conversation evolves), if you’re short on time you can also gather high-level information using self-guided tests.

Here’s an example of a test plan you could create:

Task: Go to the URL for a brand that you would visit to complete {activity X}.

Task: Explain why you selected that brand.

Task: Tell us about the last time you completed {activity X}.

Task: Now show us how you did that.

Live Conversation

Conduct an open-ended interview on any topic, including:

• Who are you, what is your background, and what are your needs?
• What are the challenges you encounter {doing activity X, in your job, etc.}?
• How do you accomplish {activity X}?
• What brands do you turn to, to complete {activity X}? What do you like about each?

Try UserTesting's needs and frustrations discovery template

Feeling inspired? Try a needs and frustrations discovery template to identify opportunities to improve customer experiences or activities.

View template

Autotrader The Autotrader team wanted to better understand the car buying experience. Notes Bradley Miller, Senior User Experience Researcher at Autotrader.com, “There was an institutional perspective about the kind of influence third-party automotive classifieds sites like Autotrader have on car shoppers, and therefore, we developed products and experiences around that perspective.” After conducting interviews using Live Conversation, the team discovered that the process of buying a car online resembled standard digital behavior: it all starts with a search engine. “Buying a car isn’t a new or different circumstance. It’s just another problem people have. People have questions, so they turn to Google or another search engine to get an answer. Adds Miller, “I learned that we need to meet consumers where they are. They don’t necessarily seek out a site like Autotrader; rather, they’re more likely to stumble into us from a search result. “Realizing that they are starting with a search engine and could be dropped onto any page of our site challenges the notion that car buying follows a set sequential process. This is important because when we're designing pages, we have to ensure that each part of the experience draws them further into the website. We can’t assume they’ve been on other webpages and have greater context.” Having real-time conversations enabled us to gain deeper insights that confirmed hypotheses, dispelled others, and uncovered new truths about the consumer shopping process. —Bradley Miller, Senior User Experience Researcher at Autotrader.com View customer story

Prototyping

When you have some early sketches or designs, get feedback to validate usability before investing resources to build it out. Does the design make sense? Is it clear and intuitive to the user? And if you discover that something in your early designs is really problematic, you can change it and then test again to see if it’s a better experience before proceeding to development or production.

Can’t decide on a single design? You don’t have to! Ask test participants to evaluate multiple design ideas in a single session. Continue to iterate, getting feedback as you hone in on preferred design(s).

There are two things to keep in mind when determining which method to use:

• The level of control you want over your content/designs
• How confident you feel that a test participant would understand how to navigate and provide feedback on your designs

Self-guided test

If you want to ensure your designs are only shared with the test participant during the test, you should share your screen during a Live Conversation session or use UserTesting’s secure prototype hosting during an unmoderated test. If you think a test participant may get confused about a prototype or designs, you should present them during a Live Conversation session or be very clear in your instructions if running a self-guided test.

Provide a link to your design (such as from a third-party design tool, or a third-party hosting solution) to get feedback about early design ideas. Learn more about sharing options.

You can also upload an image or prototype to UserTesting’s secure prototype hosting. Your designs will only display during the test. Learn more about secure prototype hosting.

It’s common for test participants to mistake prototypes (especially high-fidelity ones) for live sites or apps. Make it clear in the task directions that this is a prototype with elements that the test participant can’t select. They should speak out loud about what they would do if they encountered the webpage or app.

Live Conversation

Share prototypes during the live session to maintain control over your designs while getting feedback. Learn more about sharing your screen in our Help Center.

As with interviews conducted during the discovery phase, the flexible and dynamic interaction gives you an unrestricted opportunity to gain insights. This helps you to radically reimagine and redesign your product before any development begins.

This is also a good option if you have more complex designs that might be confusing if presented during a self-guided, unmoderated test. Presenting your designs during a live session offers you the best of both worlds. It gives you control over when, where, and with whom your designs are shared. It also allows you to guide test participants if they encounter challenges.

Try UesrTesting’s prototyping template

Feeling inspired? Try a prototyping template to observe users interacting with a pre-production experience here.

View template

Pediatric health system The largest pediatric health system in a major US metropolis wanted to update its website and apps—hubs of critical information that helped to attract prospective patients and kept current patients healthy and informed. Given the great scale of the project, and inversely the size of the lean team, they sought to rebuild the website template by template. Using this approach, they could create optimized designs that could then be applied across all relevant webpages—for rapid change at scale as well as consistency throughout the digital experience. Starting with the template used on department and program pages of the website, the team gathered feedback to figure out what was working, what wasn’t, what people liked, and what needed to change. The team asked questions like, “Do we have the right information there? And of the information that is there, is it in the right order?’” Using these insights, the team created a wireframe of a potential new design. They then sought feedback on the revised wireframe as well. The digital marketing lead shares, “We cut the wireframe up and then had people change the order of the parts according to their preferences—sort of like having them build their optimal page. We then created a design based on the most popular order and again asked people to assess whether this new version provided all of the information that they needed.” Do we have the right information there? And of the information that is there, is it in the right order? —Digital Marketing team

Development and pre-launch

Get feedback as you reach each milestone to ensure that you end up where you planned. And if feedback indicates there’s a problem or something that requires attention, you have the opportunity to fix these issues before proceeding to the next milestone. Studies show that problems that are discovered and fixed after a product release have 100x the cost compared to when issues are identified and addressed while in development.

It’s also worth noting that many things are happening in tandem as a product or experience is readied for launch. This is around the time that marketing and other GTM teams are preparing their campaigns and other promotional assets. These GTM projects also benefit from customer feedback to ensure that the messaging and promotional activities line up product capabilities with customer needs and preferences.

Self-guided test

Share a link (plus any necessary login information) for test participants to access your staging environment so you can see how they attempt to accomplish tasks using the product at that point.

You can also upload an image to UserTesting to get feedback on imagery or even ads prior to launch. Learn more about image hosting.

Live Conversation

Conduct a live feedback session and share your desktop or mobile screen to show the new product in a staged environment. Bonus: Give the test participant mouse/ keyboard control to give them more “free reign” as they interact with the experience. Learn more about screen sharing during an interview.

Try UesrTesting’s multichannel journey template

Feeling inspired? Try a multichannel journey template to understand how customers engage with your brand or products across multiple touchpoints.

View template

Code for America While developing ClientComm, a web app that allows probation officers (POs) to exchange text messages with their clients from their computer, tablet, or mobile device, the Code for America team ran unmoderated tests to understand the needs of the product’s two user types. “We have users with a wide range of digital literacy, and we wanted to be sure we covered everyone’s needs,” notes Rachel Edelman, Product Designer at Code for America. “Our customers (POs) operate in a world where fax machines, phone calls, and personal meetings are the main modes of communication. Many have legitimate fears about their ability to adopt new technology. Being able to do usability testing and ensure we were upholding our mandate to provide intuitive, helpful solutions was very important.” Adds Edelman, “We just wanted to make sure that people can complete the tasks they need to do in order to do their jobs.” And by using unmoderated testing and the UserTesting contributor network, the team was able to locate the types of audiences they were seeking and quickly receive feedback to support the team’s busy schedule and resource constraints. “There’s only so much we can travel. It’s amazing that I can sit in my office and set up a UserTesting study and get all the results back within 30 minutes,” notes Edelman. “That’s about the amount of time it would take me to take a cab to the airport. It’s an amazing time saver.” "...it’s amazing that I can sit in my office and set up a UserTesting study and get all the results back within 30 minutes...That’s about the amount of time it would take me to take a cab to the airport. It’s an amazing time saver.” —Rachel Edelman, Product Designer at Code for America View customer story

Post-launch

Even after a new feature, product or campaign has launched, you’ll want to keep monitoring to address problems or to keep evolving the experience to drive greater adoption or conversion. For example, you may wonder, “Why are people dropping off at this stage in the workflow?” You can understand why this is happening by gathering qualitative insights.

Or conversely, hearing why customers are taking certain actions can arm you with new and novel use cases you can include in your educational or marketing materials to influence user behavior moving forward.

Qualitative insights are great when paired with analytics or quantitative metrics you may be tracking. Together, they can provide a fuller understanding of the reasons (the ‘why’) behind user behaviors.

Self-guided test

Identify areas where you want a fuller understanding of how customers are interacting with or perceiving products, experiences, or messaging. They can include:

• Areas with low user adoption
• Unexpected user flows
• Areas with high user drop-off / low conversion
• High-performance areas, so that you can replicate winning strategies

Live Conversation

Identify areas where you want to get more insight and have a dialogue with customers about those sections in particular.

Understand how well the new release is meeting customer needs and uncover new wants and needs.

Another idea: Have users show how they’re actually using the design that was released (and compare this to what you learned during Discovery!)

Try UesrTesting’s landing page conversions template

Feeling inspired? Try a landing page conversions template to optimize conversions with insights about what motivates your customers.

View template

Adobe In its efforts to continually improve upon Photoshop, the perennial graphics editing software favored by design experts and novices alike, the Adobe team set out to launch an image extraction feature frequently requested by users. However, usage data following the launch as well as posts on user forums revealed that customers were confused and struggling to use the new feature. The excitement of releasing the high-priority feature wore off as the team realized they had to make some major revisions. The Photoshop team turned to UserTesting, using the on-demand contributor network to connect with new Photoshop users representing a wide variety of skills and levels of expertise. Based on the feedback, they made a number of updates, including giving the feature a clearer name, better interface, and menu navigation. Doing so helped them launch an improved experience that is helpful and easy to use for the wider spectrum of users. View customer story

Let feedback from customers be your guiding light

Finally, it’s worth mentioning that while the steps of product development or any project plan are often presented as linear, in reality, inspiration and challenges can crop up anywhere and at any stage. It’s rare for a team to follow the process from start to finish.

If in the course of development you get feedback that creates the opportunity to pivot and change direction or creates an offshoot project for another team to pursue, these are moments to regroup and decide how to proceed. The point is: let feedback from your customers be your guiding light—your proverbial North Star—to ensure that all the work, time, and other resources you put into creating newer and better experiences line up with what your customers want.