Understanding Survey Design: When Not to Do a Survey

Yikes! I talked someone out of doing a survey today. Or as the prospective client put it, “So, you are trying to talk yourself out of a job, eh?”

I’ve been asked that question before and it may seem like a strange thing to do, but ultimately the success of my business depends on my ability to provide value.

In this particular case the prospect was developing a new product and he wanted to determine if there would be a market for the product. He did not yet have a prototype and it was going to cost him approximately $15,000 to build one. I discouraged the use of a survey in this case for several reasons:

1. Without a prototype to look at, try out, and imagine, survey response would likely not be consistent enough to give an accurate perception of the market.
2. The prospect had only a general idea of the specific target market. He was looking to go to the general population with something that was clearly a niche product. Before survey questions could be designed, there needed to be more research done into a likely target market.
3. The difference between the cost of the market study ($6,000 to $10,000) and the cost of the first step, creating a working prototype, was not enough to justify the study. He would get a much better idea of the market demand from showing the working prototype to whomever he could than he would from asking questions of strangers asked to imagine the product.

I encouraged this prospect to save his money for building the prototype and exploring the market himself by talking to those he think might be interested in the product.

This brings up the question, “When is a survey not such a good idea?”

A survey is not a good idea when…

1. You don’t have the enough money and resources to act on the information you get back.
2. You have already made your decision; you just want to see how it is going to be received.
3. Your reason for doing the survey starts with, “It would be nice to know….”
4. You don’t have enough money to do your survey right without cutting corners.
5. You’ve never done a survey before and you aren’t getting help from someone who has.
6. You don’t know much about your target audience and you want to use the survey to learn about them (focus groups or personal interviews would be a better first step).
7. You want to use the survey to help you make a decision on whether to invest an amount less than $20,000. (As a business, here it may be better to just invest the money.)

Knowing when not to do a survey can be as instructive as knowing how to execute an effective survey.