Alchemer CEO Christian Vanek was among those industry leaders invited to speak at this year’s Festival of NewMR. This virtual event brought leaders in the market research industry together to share experiences and visions for the future of the market research industry. His presentation, “A Comedy of Errors: Anonymous Stories of Research Mess Ups,” covers the all-too-real pitfalls that researchers face when embarking on a new project.
The presentation is not just a cautionary tale, either. Christian shares practical, doable tactics to lessen the impact of these common mistakes – and ways to avoid them altogether.
Watch the video below (it takes less than 20 minutes), or read our summary below.
Pitfall #1: Bloated, Overly Long, Mega Surveys
When we asked researchers what problems they see, this is the most common response we received. What surprised us is that the Alchemer customer support team sees this issue all of the time.
The solution is good project management. Create a step-by-step process and set realistic expectations with clients from the very start. The best survey studies have as few questions as possible, with each question dialed into exactly the information that clients need.
It’s inevitable that clients, stakeholders, and researchers will want to ask more and more. It’s only human to be curious! But, resist the urge. Establish survey objectives from the start, then go through the first few drafts of your survey questions one by one to tie them back to the survey objectives.
For the sake of accurate research, it’s imperative that you ditch the 600-page survey and stick to something short, sweet, and to the point. Respondents are much more likely to answer shorter surveys and do so with more accuracy and consistency.
Pitfall #2: Questioning That is Too Wide-Ranging and General in Scope
Clients don’t necessarily understand that surveys may not be the best way to start a market research study. Before you can do a quantitative study, it’s important to take a look at and understand what’s going on in the marketplace.
The best trick for avoiding this pitfall altogether is to establish objectives for the study and an action plan for the results before the study takes place. Without a clear objective and an idea of what the data will be used for at the end of the study, questions can wander and question types may not return the kind of data your client really needs.
Objectives and an action plan keep questions on target and make it easier to ensure that the data collected is in the right format and beneficial for future decision-making.
Again, project management is essential.
Pitfall #3: Sampling That Does Not Represent the Wider Population.
For many researchers, this is a nightmare scenario, because data collected from a non-representative portion of the population is often completely useless. A study that doesn’t successfully reach a representative population can result in making the wrong decisions for a business if it’s not corrected, and added cost and time if it is.
Christian recommends involving a panel services manager early on in the process so that person can more accurately gauge who your study needs to reach and how to set up disqualifying questions to ensure that those people make it through to your study.
In addition, ask many, many questions of your clients about who they really want to reach with this study. Researchers must use this conversation as an opportunity to determine whether or not clients are making assumptions about their audience. Often, companies really don’t know who all of their customers really are – Alchemer included!
Pitfall #4: Late Stage Changes from Clients That Lead to Retesting and Logic-Busting
This problem is experienced by both large and small consulting firms, as well as internal research teams, and it’s a headache for the people designing and programming surveys.
Again, setting expectations in advance and good project management can minimize the instances and impact of last minute changes. Explain from the very beginning, and remind clients periodically, that last minute changes cannot be easily accommodated and will negatively impact your team’s ability to get the project done on time and provide the insights they need to make decisions.
Even putting it into a project plan and even putting a hard date for changes, this pitfall will come up anyway. They happen. The best researchers can do is to try to identify necessary changes early on in the process.
From a tactical standpoint, Christian recommends building survey logic and validation into each survey at the very end, just prior to final testing. Most last minute changes are most disruptive to these features and saving them to the last step can help ensure that every branch of logic gets tested accurately.
Pitfall #5: Internal Stakeholders Don’t Respond to Clear Results
There’s nothing worse in research than putting your heart and soul into a project that you know serves a higher business purpose, only to watch as stakeholders look at the data… only to do nothing with it, except maybe as a coaster for their cup of coffee.
In terms of trying to solve this problem, creating clear expectations up front with stakeholders about when they need the data and whether they are willing to defer making a decision until that data is collected is an important first step. If your key stakeholders say no, maybe a full scale survey study isn’t the best way to get them information. Perhaps short surveys or customer interviews is a better way to help stakeholders make the right decisions.
Surveys may not be the right answer.
You can do the best research possible, but if it doesn’t register with the stakeholder, then it’s a waste of a great opportunity.
The Trick to Market Research Success is Working Hard With the Right Tools
Avoiding many of the most common pitfalls comes down to project management skills like managing stakeholder expectations and maintaining a high level of communication with clients. True success comes from navigating these pitfalls as best as possible, working hard, and using the right tools along the way.
The Festival of NewMR is a virtual event designed so that busy market researchers can get access to the industry’s latest and best practices, as well as key insights into which research tactics are making a difference for other market researchers. All speaking events were recorded. Videos and slides can be accessed on NewMR’s Play Again page.
Looking for a data collection solution built just for market research? You need the Alchemer Market Research License.