A great survey design should start with the needed results in mind.
Each survey project should have a goal and the survey data should support that goal.
Use that knowledge to design an effective survey to meet your goals.
Basic Details for Good Survey Design
To start the survey design process, ask yourself the following questions:
- What are your goals for the survey? Identify no more than three goals for each survey project and design your survey to provide the data to meet those goals.
- What will you do with the data? Will it be analyzed alone or with other data? Will it be used only internally or publicly? Is it for your organization’s use or for a client?
- What do you want to ask to support those goals? What are the questions you need to ask in order to reach your project goals?
- Who is your audience of respondents? Are they customers or potential customers? What is their geographic location, language, age, financial status?
- How will you distribute the survey? Online via a website or email, in person or phone interviews, on paper?
- What kinds of reports do you need? Are you reporting only for yourself or for others? Do you need to provide reports in a particular format (Word, PDF, PowerPoint, Excel, SPSS)?
Once you have these key questions answered you’ll be able to start designing an effective set of survey questions.
Designing Survey Questions
There are number of considerations in creating effective survey questions. Once you know what the goals of your project are you can design your questions to support the goal. There are some basic guidelines for creating questions that will serve you well.
Keep your questions brief and language simple. After all, it is much easier for a respondent to give a good answer if they fully understand the question.
Be specific and direct in phrasing your questions and answers. Tell your respondent exactly what is needed in answering each question.
Be sure each survey question is relevant to the goals of the project. For instance, if the goal of the project is to only to determine where a respondent heard of a particular brand, you would not want to include questions on pricing or service.
Survey Question Types
Most survey softwares allow you to choose from a number of different question types. Understanding the different question types can help you choose the ones that will produce the type of data that you’re after.
Quantitative question types allow you to set up a discrete set of answers that a respondent needs to choose from. Radio button, checkbox and dropdown menu questions are good examples of this type of question along with rating scales and ranking questions.
These kinds of questions produce clean, easy-to-analyze charts that will allow you to draw conclusions about trends and patterns among your respondents.
Qualitative questions allow respondents to enter their own answer, such as textbox and essay type questions. While harder to efficiently analyze, qualitative questions are important because they can give you direct insight into a respondent’s point of view.
You will need to determine how you’ll handle the data these questions produce before you distribute your survey. You can create a word cloud with the most commonly used words, use open text analysis to identify patterns, simply read each one and make notes, or a combination of all of these.
Choosing the right type of question for each survey question will make a big impact on getting the most effective survey data for your project.
Reporting Values: Standard vs. Custom
Survey questions can be designed so that the answer itself is the value that is seen in reports, or you can design customized values that you see only on the report but that remain invisible to those taking the survey.
For instance, if your question is “How did you hear about us?” your answers may be “From a friend,” “From a online search,” or “From a newspaper ad.” It’s possible that your reports can simply use those same answers; in that case you don’t need to make any adjustments.
However in some cases you’ll want to specify reporting values that are different from the answers in your survey. If it will best suit your analysis of response data or the reports that you need to provide for your project, you might choose to use different reporting values.
Numeric reporting values could be used, such as 1 for a referral from a friend, 2 for online search and 3 for a newspaper ad. Or you could just use shorter versions of your answers for a reporting value, for example, “Referral”, “Online Search” or “Newspaper Ad,” to simplify the appearance of your reports.
Knowing what purpose your survey data will serve will help direct what type of reporting values would be most effective.
Adjusting the Language of Your Survey
Depending on your audience you may need to design your survey in more than one language in order to get good responses. If your intended audience is more at home in a different language than English your survey design needs to account for that.
For surveys in multiple languages you have a couple of options. Individual surveys in a single language can be sent to each particular audience, or you might design a multi-language survey that allows the respondent to choose the language they would like to respond in.
If you are using a multi-language survey, it is critical to evaluate the meaning of each question and its answer options. Each language has its own nuances and you’ll need to make sure the survey questions that you design convey the same meaning in each of the languages used.
Depending on how you set your survey up, you could also have reports in different languages. This might also be beneficial for those you share your survey response data with.
4 Things That Sabotage a Survey Design
No matter how carefully you plan your survey, there are some things that can creep into your design and create problems with your data. Keep an eye out for these four survey saboteurs so your data stays clean and actionable.
Lack of Focus
Only ask those questions you need to ask and resist the temptation to throw in a little extra. Once you have written out the questions you’d like to ask, go back and judge them to make sure they fulfill the goal of the survey.
If a question is not related to the survey’s goal, it shouldn’t be included.
A lack of focus in a survey can make it seem to a respondent that they are walking on a path and are not sure where they are going and when they will get there. If your questions are consistently related to your goals and the design of the overall survey is consistent, you’ll get better responses.
Bias
Getting objective answers is much more useful that getting the answer that you might wish for. When you are designing survey questions you’ll want to be sure that question wording is not leading respondents to particular answers.
For example, if you include a rating question about customer service, you might use a scale question.
If you ask “How satisfied were you with our great customer service?” where possible answers are “Very Satisfied,” “Satisfied,” and “Dissatisfied,” you’ve told the respondent that the service was great and only given them answers to indicate they were dissatisfied.
To remove the bias, maintain your neutrality by rephrasing the question to ask, “What was your level of satisfaction with your recent experience with customer service?” and offer them a wider range of answer options, including, “Very Satisfied,” “Satisfied,” “Neutral,” “Dissatisfied,” and “Very Dissatisfied.”
Evaluating your survey and questions to make sure that you are not introducing any bias or leading your respondents to particular answers will help you get much more effective survey data.
Fatigue
We are offered opportunities to give input or provide feedback on a regular basis, leading to a high level of survey fatigue for most people. When you do have a respondent’s attention, make sure that your survey design does not introduce opportunities for your respondents to experience fatigue and possibly drop out.
Respondents can experience fatigue when there are too many questions total, too many questions on a page, questions are too long or too complicated, or when there is a confusing design for the overall survey. As you are going through your survey design consider your respondents reactions at every stage.
Test the survey at different stages during your design by taking the survey yourself, and by having teammates or friends take the survey and give you feedback on their survey experience.
Miscommunication
A respondent’s misunderstanding of a survey or a survey question can definitely cause your survey data to be unusable for your needs.
Using clear and simple language in your survey questions and having a survey design that gives a direct and clear path to complete the survey will make it easy for your respondents to give good answers and complete the survey.
Review each of your survey questions for ease of understanding as well as for bias.
Shorter surveys might have one question per page so that respondents can focus. If your survey is longer, you might show more questions on a page (but no more than 10). Questions that can logically be included in sections might be together on one page or a section of pages.
Advanced survey tools can allow you to display your respondents’ progress as they move through the survey. This can be particularly useful in longer surveys, where people will be less likely to abandon their task if they can see that they are nearing the end of the questions.
Other Elements of Great Survey Design
There are a few final pieces to the survey design/survey data dance that we need to cover, including writing questions for your audience, distributing your survey, and making branding decisions within the survey itself.
Audience
Determining who the audience is for your survey is part of your initial project design. For example, if your audience is your customer base, survey response data can add to your understanding of your customers and their use and satisfaction with your product.
Through working with your customers you may have information that will help you design a survey that they will be more likely to complete so that you can get that needed data.
Your audience may not yet be your customers, for instance a general audience in a specific area of the country where you want to do a brand recognition survey.
You might access that audience through a panel management company where you pay for survey responses from their panelists. Defining the panel well and reaching that specific audience will provide the most actionable survey data.
Define your audience as well as you can as part of your project planning so that you can consider that knowledge when creating the survey design.
Distribution Method
You can distribute surveys in a number of ways, and you may want to use more than one in order to get the responses that you need to meet your goals.
Surveys can be embedded on a website, sent out via a link in an email, administered in a personal or phone interview, or taken on paper. Based on the goals of your project and the makeup of your audience you can determine which distribution methods will be most effective to reach your audience and have them provide responses.
Branding
In some of your projects you will want to include your brand, while in others you’ll want feedback that’s free of brand influence.
For instance, in a customer feedback survey, communicating your brand in the form of logo, colors and fonts will make the survey more recognizable to your customers. Building trust through the use of your brand encourages respondents to complete the survey, allowing you to gather the desired survey data.
During a brand awareness study, on the other hand, you don’t want to unduly influence respondents’ recall of your brand or your data will be skewed.
Great Survey Design: Great Survey Data
Setting goals for your survey, designing great questions, avoiding common pitfalls in survey design and paying attention to each of elements in creating a great survey all can improve the quality of your survey data.
Following the steps to design an effective survey will result in great survey data to allow you to meet your project goals.