In my last article, I discussed the first five points on my list of the most common mistakes made when writing surveys. Now, it’s on to the remaining five.
As I said last time, although it’s nice to know the most common survey mistakes made, the real value is in understanding how to avoid them. You can do that by understanding why each is a mistake.
6. Asking unnecessary survey questions
It is easy when writing surveys to fall into the trap of wanting to know everything. However, as a survey writer you owe it to your respondents to only ask questions from which the resulting data will be used to take action or make a decision.
Respondents can sense when you are asking a question that isn’t needed and will not be used. The two most common types of unnecessary questions are asking about something that has already been decided and asking about things over which you have no control.
7. Asking too many questions
This survey mistake appears straightforward, but is often misunderstood.
I’m often asked how many questions can be asked on a survey; however, there is no magic number for the right number of survey questions. The two limiting factors are:
- The commitment and attention span of the target audience
- The resources and time the survey owner has for acting on the information received.
For example, when it comes to commitment to a survey, you can’t ask as much time of a general audience without an investment in your subject than you can of a loyal customer or a dedicated employee. If respondents are committed to your subject and are kept engaged by the survey instrument, they will spend the time it takes to complete a long survey.
The second limiting factor, the resources and time the survey owner has for acting on the information received, usually overrides the first.
The only reason to do a survey is to use the information obtained. Thoughtfully using this information and making changes or improvements takes a considerable amount of time and effort. It is better to implement shorter surveys more often than to implement one survey that produces five years of work.
8. Asking two survey questions in one
This is a great way to frustrate your respondents and give you ambiguous data. For clarity, let’s look at two examples.
The first is the question: “Please rate the technician’s knowledge and professionalism.” This is clearly two questions. The technician’s knowledge may be great and his professionalism lousy.
The second example, asked of a high school counselor, illustrates a more subtle way of making this survey mistake.
“Do you interact with your students’ parents about college?” Again, this is really two questions: “Do interact with parents?” and, if so, “Do you talk about college?”
This mistake can be fixed in either of two ways. You can ask both questions separately or you can ask the one question and adjust your response choices to include both “I don’t interact with students’ parents” and “I interact with students’ parents but not about college.”
9. Making questions too general
The problem with questions that are too general is that two respondents can sometimes answer the question the same but for completely different reasons.
For example, “Do you believe wireless devices can cause health problems?” Clearly, there are many ways for people with very different views to answer this question “Yes.”
One person may feel it is a remote chance while another may think it is an absolute certainty. The quality of the information obtained from a survey depends on asking focused, unambiguous questions specific to the survey objectives.
A better approach for determining people’s beliefs about the dangers of wireless devices might be to ask, “Do you curtail your use of wireless devices specifically to avoid risk to your health?”
10. Putting too little time and effort into writing the survey, period
The fact is, it is easy to write a survey with lot of questions and send it out to a broad group of people. The difficulty is getting usable information that can help with solid decision-making.
Every question in a survey needs to be well thought out and evaluated against the survey objectives and the target audience. Too often the results from a survey raise more questions than they answer because the questions weren’t well thought out, reviewed, tested and reviewed again. Extra effort spent writing your survey will pay big dividends when using the data.
Related Articles
10 Tips For Crafting Good Survey Questions