Figuring out what questions to ask in your survey and getting into the right people’s hands can feel like the most challenging part of running a survey, but once your survey responses come in a new phase of work begins.
In order to create useful, actionable reports you’ll need to follow these four steps that turn your responses into insightful data that’s easy to present:
- Clean Data
- Run Initial Reports
- Analyze Data
- Create Final Report(s)
The Importance of Clean Data for Quality Survey Responses
When you have reliable data you can be confident that you’re building decisions on a solid foundation. For optimum reliability, you need to clean your data to identify outliers before you start analyzing it.
This process helps ensure all of your data is relevant (and comes from a real person who cared about taking the survey).
Survey Response Data Warning Signs
Here are the top five warning signs to be on the lookout for when reviewing your survey responses:
- Suspicious Answer Patterns: Sometimes referred to as “Christmas Tree” or “straightlining,” these type of responses to checkbox or radio button questions follow a very clear pattern, and they don’t reflect thoughtful, accurate answers.
- Very Fast Completion Times: You should know your average survey completion time, and you should be suspicious of responses that come in well under that time.
- Choosing All Checkbox Options: Whenever someone picks all the checkbox options consistently it’s often a sign that they’re just speeding through the survey without reading all the choices. Typically these responses can be thrown out of your data set.
- Red Herring Fails/Logically Inconsistent Answers: Particularly on longer surveys, you’ll want to include a few roadblocks that will throw up red flags to identify poor quality responses. Something like, “Please check the box next to the word, ‘red’ below” and then a list of several colors will help you find out who’s really reading your questions.
- Nonsense or Missing Open-ended Answers: A look at what people type into required text boxes will often give you immediate insight into which responses are valuable and which should be excluded.
Preparing Your Survey Response Data For Analysis
Now that your data is clean, it’s time to prep for analysis. Hopefully you identified any inconsistencies in response options during the testing and validation phase, but if not, keep an eye out for inconsistent numerical values and any breaks in validation that arise.
Whatever problems you find, be sure that you don’t introduce a new source of bias by changing the question text AFTER you’ve collected responses.
Analyzing Qualitative Survey Response Data
As you may remember, qualitative questions allow respondents to type in their own answers. They offer great insight into the “why” behind your survey questions, but they can be challenging to analyze.
Prepare options for how you’ll derive conclusions with qualitative data from open text or essay questions.
Some good choices are:
- Tracking keyword frequency, or how often particular terms are used by all your respondents.
- Word clouds, a handy visualization of the words based on how commonly they appear in answers. The more a word appeared in your responses the larger it is.
- Rate each response as positive or negative based on the emotional words being used.
- Bucket responses with an open text analysis tool. This software feature will let you categorize responses when they use a certain term or phrase.
Remember your survey learning objectives? Now’s the time to pull them back out again.
You’ll want to run an individual report for each learning objective in order to determine the “highlights” of the data you collected as it relates to future actions.
This way you can truly understand the most significant findings of your research.
Based on each report, determine what actions you’ll be recommending for each learning objective.
Run Preliminary Reports on Your Survey Response Data
Once you’ve cleaned your data and refreshed yourself on your survey goals, it’s time to do a first pass at creating your reports.
These initial reports are designed to help you determine:
- If you got your original questions answered.
- If the data is in the format you expected.
- Whether you’re seeing the expected trends.
Data Types to Consider
Depending on the purpose of your survey, you may collect demographic details about your respondents, firmographic data, or both.
Demographic data are the statistical characteristics of human populations, such as age or income, and is used especially to identify markets.
Firmographic data are the characteristics of an organization, such as size or location of a company, and are almost exclusively used in business to business research.
Often your survey will contain demographic and firmographic questions so you can create segments in your survey and reports.
These segments can give you great insight into relationships among your responses, and they should remain the same from start to finish of the survey process.
Is There a Trend in Your Survey Response Data?
When you have data that isn’t statistically sound but is still interesting, you can call it “directional data.”
This data gives you an idea of what your population is saying, thinking, or feeling, but you cannot use statistics to back it up.
Analyzing Your Survey Response Data: Ratios
If you collected too many responses from a certain segment of the population, sometimes you will need to adjust the weight of your responses in order to keep it true to outside ratios.
For example, if the population of the US is 52% women but your respondents were 54% male, you’ll need to make some adjustments if you want your results to accurately reflect the real ratios of the US population.
Report on Your Survey Response Data Findings
There are four stages of the reporting process, during which you reveal the brilliant findings of your well-designed survey to the world.
Stage 1: Write a summary
Stage 2: Write a mini-report for each individual learning objective
Stage 3: Reveal interesting and unexpected trends
Stage 4: Conclusion
Stage 1: Write a Summary
In this section you’ll recap much of the information you pulled together while deciding on your survey goals and objectives, as well as the criteria you used for selecting your respondents.
It’s also a good place to clarify any confusion about your data collection method, including why you chose the method that you did.
Be sure to include these points:
- What was the ultimate goal of this survey?
- Who was surveyed?
- Who was the population?
- Who responded?
- Include basic highlights of the survey audience and your data to introduce the findings.
Stage 2: Write Mini Reports
Each learning objective gets its own mini report so you can specifically address the goals and outcomes for each one.
The last section for every learning objective report should include the recommended actions to take based on the results of the survey (these should not be a surprise!).
Stage 3: Interesting and Unexpected Findings
While optional, this stage can lay the groundwork for future projects and reveal things about your audience that you weren’t specifically investigating.
For example, maybe you found a new segment of your population that could help you to make good business decisions going forward.
This section is:
- Good- to-know, not need-to-know
- Going the extra mile!
Stage 4: Conclusion
This is the big finish of your report! Recap the actions that you recommend the team take based on your survey’s findings, and include as much supporting evidence as you need to in order to get stakeholders to agree to those actions.
You can also create a survey to be sent to stakeholders in order to gain feedback for the project and put actions in motion. In order to take meaningful action, ask stakeholders to provide metrics that can be used to measure the success of the actions that will be taken.
Tips for Communicating Survey Response Data
Finally, we want to leave you with a few helpful tips for communicating your data.
Because after all, having the best data in the world isn’t very useful if you can’t convey its value.
- Understand your audience and their interests
- Try to be brief
- Keep your report and findings clear
- Have more than one clear course or possible way forward with the data
- Include data visualization to convey key points
- Try to anticipate questions about the reports
- Know the details
- Be honest