About 10 years ago, a series of experiments in online survey design created a new trend in online survey research and question presentation.
Online researchers began to use non-standard question types with participants in order to use the rich interactivity of online media to increase participation and survey engagement.
Experimenting with how we ask questions has roots deep in the research community, but these new question types have received a great deal of resistance by the community.
The the research community has called this trend by a few names (and called it few names): “rich media questions”, “highly interactive questions”, and “2.0 question types”, for example.
These new methods of asking questions have fundamentally changed the interaction behavior of respondents and taking of surveys, mostly for the positive but have also raised questions — most of which have not been definitively answered.
- Are the answers obtained by these question types more accurate?
- Can they still be compared with existing data?
- Is this the future of online research? Or is it just a distraction?
- Do the benefits of highly interactive questions out way the problems and concerns?
We have some great new data and insights to share with you about this controversial issue.
Examples of Highly Interactive Questions
There are a variety of interactive questions types. Here are some examples:
Image selection questions allow the survey respondent to look at a series of images and then click on the image that best answers the question at hand. The image select question generally comes in both multi-select and single select flavors.
Heat maps are another form of image selection. In a heat map question, an individual image is uploaded to the survey and survey takers are asked to click on areas of the image to answer the question. An example heat map instruction might be: “Click on areas of the image that you feel best represents the product being displayed.”
Drag and drop ranking questions are a way of asking individuals to rank a series of items. Items are displayed in a list to the left-hand side of the screen and the survey takers then drag items in their desired order into a bucket on the right-hand side. The act of dragging and dropping can be much more intuitive for survey takers.
Slider questions are a continuous-scale question, represented by a drag-gable sliding indicator that updates your answer value as you slide it from left to right or up and down. Although the data collected is identical to a normal continuous-scale question, the survey taker’s experience is distinctly different.
The Pros
Highly interactive questions types are a great way to engage survey respondents that are surely feeling the fatigue of surveys being everywhere. Go to the market? Get a survey on your receipt. Pick your car up from being serviced? Get a survey on the way out. Call Alchemer for support? We are going to ask you to fill out a survey about it.
Interactive question types help energize respondents, reduce survey abandonment and increase responses rates.
And, even though our parents taught us not do things just because everybody else was doing it, well, everyone else is doing it! You do not want your survey to be boring compared to everyone else’s. More folks are using these question types to break through the survey clutter.
Based on the Alchemer Benchmark Guide survey, surveyors have jumped on the interactive bandwagon, with 45 percent of surveyors saying that they will be increasing their use of interactive question types in the next year.1
The Cons
Highly interactive question types do not always accommodate all survey takers. For example, screen readers for accessibility are very unlikely to be able read most highly interactive question.
There is also a possible collision with the uptick of survey taking on mobile devices. Many interactive question types that work well on a desktop device don’t work on mobile devices and, given the fact that that Alchemer has witnessed an increase of over 100 percent of respondents taking surveys on mobile devices from a year ago, there’s room for potential disaster.
Even with the foreboding disaster of users trying to take surveys on with highly interactive questions on mobile devices, more and more, organizations are moving to these question types in order to encourage survey-taker engagement.
The Hesitancy of Using Highly Interactive Questions
There has been some hesitancy to using these question types stemming from a lack of understanding about how to compare existing data collected with traditional question types to data collected in these new question types
This difference has led to some concern that the cognitive psychology of answering questions that are highly-engaging may differ from this psychology of answering a standard radio button or checkbox question. There is, but we have not noticed enough of it to bias the data one way or another.
Tips for Using Highly Interactive Questions
If you’re going to use highly interactive questions in your survey, we recommend that you:
- Know your target audience: Know your potential respondents and how they will interact with interactive questions. Are they most likely to take your survey on a mobile device? What are their accessibility needs? Determine if interactive questions would be right for your population.
- Split test: Since interactive questions are basically radio button or checkbox questions with a pretty interface, you can do a split test. You can run the survey with traditional questions and run the same survey with interactive questions. You can compare the data to see if the type of questions affects your responses significantly.
- Detect mobile browsers: You can build the survey with both traditional and interactive questions and then auto-detect the type of browser your survey is being taken on. You can deliver the appropriate survey experience to each user and then combine the data for analysis.
- Test, test, and test some more: You should always test your survey before putting into the field, but interactive question require an extra layer of testing Pay special attention to all the different types of devices you expect survey takers to use.
We want to know about your experiences with highly interactive question types! What has worked best for you? What has not worked? We really want to know!
1Source: Alchemer Market Research Benchmark Guide: 2012, Comparative Analysis on Survey Metrics, Techniques and Trends, Do you plan on using more, less or the same amount of Highly Interactive questions in your online surveys in the next 12 months, then you have in the last 12 months? In the next 12 months, I plan on using… n=323 Those that have used Highly Interactive questions in the last 12 months