American Airlines + UserTesting

According to Lucas Lemasters, American Airlines’ UX Research Principal, “UserTesting provides us with both the quantitative and qualitative information that we need to make decisions about products.”

The positive impact of UserTesting has been particularly apparent in American’s benchmarking programs. Partnering with UserTesting's Research Consultants, American established QXscore™ as a shared metric to drive product decisions and success. QXscore is a proprietary metric that quantifies how well a person can use a digital experience and how much a person enjoys the experience. 

American Airlines put its entire sequence of digital experiences for searching and booking flights in front of potential customers with UserTesting, and then they quantified the usability and enjoyment of their website by QXscoring four key customer experiences:

1. Booking a flight
2. Changing and managing a flight
3. Checking in for a flight
4. The AAdvantage program

For the benchmarking programs, they measured the performance of their digital experiences against the same experiences from competing airlines, carefully tracking every conceivable engagement a customer could have when using the American Airlines website. 

American’s UX team provides QXscores to the organization’s product owners. This gives the teams a common, quantifiable metric to track success over time. The teams share the related data sets and provide a common set of measures to senior executives, to justify their decisions. 

Kimberly Cisek, VP of Customer Experience, says, “Without human insight, you may become reliant on data that is static. What you really want to know is how the customers are using and engaging with the functions that you put out there for them.” 

The new insights help to show how qualitative and quantitative measures impact American Airlines’ digital experiences. For instance, when the UX team evaluated the path of customers buying flights, they noticed that people had not booked as many multi-city itineraries as the organization had anticipated. When they reviewed a heat map from their click tests, they noticed that the multi-city search option appeared below the fold on the page. Everyone searched the page at the top, but they struggled to find the multi-city search option.

Insights like these will help American Airlines refine the company’s digital experiences so customers can more intuitively book and manage their flights in the future. And for the digital experiences American has already measured and changed with UserTesting, the company has achieved a 15% average increase in QXscore.

Lucas Lemasters shares, "When we analyzed our experience for changing and managing a flight, we found several key areas where customers were frustrated. They couldn't find where to put passport information, where to add their loyalty account, where to request a meal, or even where to email an itinerary to themselves. When we went back and changed the concepts, we were able to achieve a 37% increase in customer completion rate across all of those tasks."