Whether it’s a card sort, an online survey, a moderated or unmoderated remote usability test—if the people you recruit for the test aren't your target customers, the test will not be helpful.
That’s not to say that you should only recruit your actual customers for a study. But there are people you’ll want to filter in or out, depending on what you’re trying to achieve.
Let’s start with the basics.
Before running any kind of usability testing, you’ll want to find people to participate. People who will carry out the tasks you sent them.
A screener (or screener question) is an opportunity for you to have more control over who takes your test.
This will also help you filter out anybody who wouldn’t be the best fit.
For instance, if you’re testing the navigation of a website whose sole audience is civil engineers, and you want to ensure all the relevant categories are represented, there’s no point in recruiting an audience that doesn't even know what a civil engineer does.
The point is that in the above example, you would want to screen for a certain level of knowledge of civil engineering terms.
You’ll likely have two types of questions available:
Be careful when selecting the type of screener question you want to use and think about how people might want to answer it. If the participant is likely to give more than one answer, then give them that option.
Here’s an example of a screener question for an online car dealership:
Which of the following statements apply to you?
This is created so as to not lead the user towards one answer over another.
Try to ask open questions, as yes/no questions are commonly seen to be too leading. For example, if you're looking to recruit Ford Drivers, then rather than asking, "Are you a Ford driver?", it would be more effective to ask which brand of car they drive and provide a list of car brands to choose from. Check out some more examples of leading questions to make sure you know what to avoid.
Before setting these questions, you’ll want to clearly define who your ideal test participant is. And if your team or organization has already defined your business’s key personas, the hard work has already been done.
Personas are a way to help organizations understand their potential and existing audience in a more personal way. In essence, personas are detailed profiles of a particular audience member, who represents a distinct group of people who share similar behavior, attitudes, personalities and preferences of your product, but are the figurehead for a larger demographic.
You could therefore tailor your questions to whittle down potential participants to the most valuable people.
Getting the best fit for your research involves juggling between three critical variables: representativeness, time in field and cost.
If exact representation is critical for your research questions then it will take more time in the field and often cost more. Usually when running research that requires assessment of a user’s ability to make a decision on specific things, high representation is important.
If getting results quickly is most important, then relaxing the representation criteria and being willing to pay a little more will ensure the fastest possible turn arounds. Classic usability type of questions could fall into this category.