Why data scientists should embrace human insights

How data science and qualitative, human insights work together comes up often at UserTesting. When organizations or individuals are resistant to qualitative insights, it’s often because of the story their data is telling them. For example, if a large number of customers are behaving a certain way—according to data—then what’s left to deduce? We often hear things like, 

“What does it matter if 20 people say one thing if 1,000 people are doing something else?”

That’s a fair question. And to answer it, I’ll discuss a bit about how the use of data is evolving, and how everyone—including data scientists—can benefit from pairing quantitative data with qualitative insights.

Going beyond the survey

Earlier in my career, companies were constantly requesting and relying on basic survey data to inform business decisions. Over the past several years, however, I’ve started to see a shift in the data that companies were collecting in order to help executives make strategic decisions. 

Boiled down, the shift was from basic survey data and analytics to a phenomenon called big data. Through big data, companies were beginning to aggregate various data sources—including both quantitative and qualitative—to obtain a more well-rounded view of their customers.

This shift suggests that successful data scientists will need to embrace qualitative data as part of their toolbox. Even if some data scientists are already using qualitative data, it’s important that they don’t just look at it from a quantitative perspective. Qualitative insights tell the story behind the quantitative data. And ultimately, both types of insights valuable for gaining a holistic view of the customer experience.

The new data scientist 

Generally, a data scientist’s main job is to sift through large amounts of data, create predictive models, and pull insights from those models. It’s naturally tempting for them to simply code qualitative data in a way that makes it quantitative in nature. (I’ve used that strategy often in my career.) Unfortunately, that strategy rarely reveals actionable insights. 

Instead, qualitative feedback needs to be viewed through a different lens than quantitative data to effectively paint a picture of customer behavior. As consumer needs and expectations continue to evolve, data scientists will find that qualitative insights need to be a tool in a data scientist’s arsenal. This will allow them to paint a full picture as to what consumers are currently doing while also painting a more futuristic picture of their behavior.

Understanding the why behind data

With many companies embracing big data, qualitative data is often pushed to the side because of the perceived inability to build predictive models from it. 

That was until key stakeholders and executives started asking, “Why?” when looking at quantitative data. Just having a Net Promoter Score or Customer Satisfaction Score wasn’t enough. They wanted to know why the scores increased or decreased. And it wasn’t just CX metrics they were looking at—executives also wanted to know why their customers were not purchasing products or services through various channels. 

Pressure is being placed on executives, now more than ever, to make strategic decisions that are proven to be exactly what their customers want. To do this, executives need to give customers a seat at the table. They need to actually talk to customers. They need to understand what drives them to make the decisions they make. They need to understand how the company can better serve their needs. And these answers won’t come from quantitative data alone.