How to run an online survey

What are online surveys?

Online surveys are a series of questions used to gather quick information about consumer attitudes, preferences, sentiments, past experiences, or overall impressions of a brand or product.

They're often used for exploratory purposes or as the first level of inquiry that can be used to dictate larger research initiatives. Online surveys can also be used to gather descriptive information such as attitudes, preferences, brand perception and experience in order to better define and understand your customer or user base.

What are common use cases for online surveys?

Online surveys are used in a wide variety of ways. Examples include:

• Basic information gathering – Gathering simple data such as demographic information, general impressions or preferences.
• Voice of the customer (VOC) – Giving customers an avenue to leaving feedback about your website and/or products.
• True Intent – Intercepting live visitors to your sites or apps in order to learn more about their intent and reasons for visiting.
• Usability scales and indexing – Questions that allow users to rate usability metrics like organization and appearance of the site, or trustworthiness and value of the content on the site, for example.
• Exploratory purposes or to inform larger research initiatives – Gathering in-depth information about users’ preferences, impressions, intent to purchase and persona building are some common examples of this.

As you can see from the above examples, online surveys can be used for websites, products, and research studies. It's also common for them to be completed long before an experience is live. 

How do online surveys work?

The general process for setting up an online survey is the same as most research engagements.

First and foremost outline what you want to learn in a research plan. Write out the purpose of your survey, define the concepts you want to measure and come up with your hypotheses. This is an important step that will help you identify the type of survey questions you want to include and ensure that you measure what you actually intend to measure.

Once you have your research plan in place it’s time to decide on the length of the survey and the types of questions to include.

Survey length: One benefit of online surveys is the flexibility with length. Depending on the purpose of the survey, it can be short and sweet (making it easier to get larger sample sizes) or it can be longer and more detailed.

VOC and True Intent surveys are typically short. Visitors didn’t know they were going to complete a survey when coming to your site and are more likely to drop off if there are too many questions. The shorter the better here.

Exploratory surveys tend to be longer and more detailed. Often exploratory surveys contain advanced logic that is dependent on previous responses. In some of these studies participants may see questions that only pertain to them. In this situation, the logic can help control the length of the survey for any given participant.

Overall we see the best results for online surveys that take no longer than 10-15 minutes to compete. Anything longer and you risk participant burnout (often leading to decreased quality of responses) and higher rates of drop-off.

Question types: There are two main question types when it comes to online surveys: closed questions and open questions.

• Closed questions are questions with a predefined range or set of answers to choose from (rating scales, ranking, multiple choice, etc.). These will provide the bulk of your metrics.
• Open questions are typically open text fields or video questions where participants generate the response rather than select a response. These help provide context around the metrics.

What results do you get?

The results can largely be broken down into two different types: quantitative and qualitative depending on whether you chose closed or open questions, respectively.

Starting with the latter, qualitative results are going to be in the form of text or a video response. Either way you will get verbatim responses from your participants.

Quantitative results often depend on the types of questions included in the survey, however, counts and proportions are the most common output for most question types. Rating scales typically include a mean score in addition to the counts and proportions for each rating. More advanced researchers might want to look at the interaction between two or more variables in which case the output might result in F Statistic or F Value or plots on a line graph.

Tips for analyzing your survey results

The analysis of survey data will be dictated in part by your research plan and the intent of the survey, as well as the type of data you collected.

Insights derived from closed question types are often reported with percentages or mean ratings (or both).

For example:

• Single or multiple select questions are typically reported in percentages: “70% reported the use the filters when searching for a product on an e-commerce site.”
• Ranking questions are also typically reported in percentages: “50% ranked option 1 as their most preferred.”
• Rating scales are typically reported by the mean score, however, many researchers also like to include the percentage of those who gave a high or low rating: “Participants indicated they are less likely to buy option B, (mean rating of 3.5 on a 7-point scale, 40% providing a low rating of 1, 2, or 3 on a 7-point scale).”

Insights derived from open question types are qualitative in nature and are most often used to support the quantitative metrics gathered in the survey. Sometimes, researchers will choose to quantify the comments by identifying common themes and reporting the percentage of participants whose comments fall into a particular theme.

For example, “60% of those who gave a low rating indicated that they didn’t care for the type of offer they received.”

Now that you have a better understanding of online surveys, you're ready to try them yourself. Take some time to learn best practices: prepare the format, questions and audience you need, then give it a shot and see how useful this research tool can be.