The importance of proper ‘framing’ in UX research

At the risk of sounding like a broken record in the UX space, ‘you ≠ user.’ But as often as this term is thrown around, it's habitually easier to say than to put into practice. So why is it sometimes easier to say stuff than do stuff?

Well, Daniel Kahneman (author of Thinking Fast and Slow and Nobel Prize winner in economic sciences) and his late friend and fellow psychologist Amos Tversky provided an explanation. They call these mental shortcuts cognitive biases and heuristics.

In this post, I’ll outline an example of one of these heuristics and how knowledge of them can help us ask our users better questions, collect better data, and make better decisions so that, ultimately, we can provide a better user experience.

What is framing and why should I care?

Commonly when we ask a user to carry out a key journey on a live website, we identify multiple routes to a successful outcome. A real interaction with a user interface is frequently different to the designer’s ‘perfect’ path. So, when we ask a participant to complete a task, the route in which they take to a successful or unsuccessful outcome can be dependent on the context or frame in which the task is set.

What I’m getting at here is that our study task instructions are especially vulnerable to biases.

So what is ‘Framing’?

Definition: “A frame is the context used to describe an idea, question, or decision. Frames heavily influence our interpretations and conclusions by emphasizing (or ignoring) certain aspects of a situation.”

Kahneman’s investigations into framing found that the exact same information can lead to opposite conclusions depending on the frame or context used to present the decision.

A remedy for this bias may seem flippantly obvious (don’t ask leading questions and don’t specify leading task instructions). For example, below are a couple of questions. One is a leading question, the other is slightly more neutral:

Is the updated design (Y) better than the old design (X)? Indicate your preference below:

VS

Please indicate your preference for either design X or design Y below:

This (as mentioned before) may seem obvious. However, the framing effect can sometimes present itself in more subtle ways.

Imagine you are testing your site’s Information Architecture (IA) using a remote unmoderated tree test method and want to remove some items that appear more than once in your navigation tree structure.

Using a question like…

Where would you look for more information about X in the tree?

may steer people towards the ‘INFO’ level in your tree, more so than:

Where in the tree would you find X?

The first task description here is framed in the context of looking for more information about a specific item in the IA. This may influence participants to look in the level of ‘INFO’ rather than where they would normally go.

Hopefully, you can see the effect that framing can have on your participants' behavior, which could lead to incorrect conclusions being drawn from research.

Why framing can be useful

Framing is, however, sometimes needed to elicit a user’s true behavior. Let’s take an adaption of Krug’s IKEA website example to illustrate this. Imagine we want to investigate the user journey to the bookcases section. We could phrase a question like:

Imagine you want to use this website to find and purchase a bookcase. Please find a suitable product, add it to your cart, and complete the checkout process.

Often, an instruction like this one would suffice, but imagine you observe an uplift in participants' use of the search bar when they simply search for ‘bookcase’ compared to your previous research that says otherwise.

Framing the task by giving the participant a specific product to look for may skew the results in favor of the search bar. Here, by providing a framing scenario, we can give the participant some context and help elicit their true behavior:

Imagine you have several boxes of books in your kitchen and want to find a way organize them. Please find a suitable product, add it to your cart and complete the checkout process.

This real-world task may leave the interpretation more open and grant you much richer insights into how people use or would use the site.

If you take one thing away from reading this post please let it be that framing can affect your UX research results in both good and bad ways. And next time you’re writing a task description or designing a question, imagine they’re works of art and you want to place them in the best frame possible.