Today, researchers and marketers are under a lot of pressure to survey customers more frequently and get answers faster.
At the same time, it’s getting more difficult to get responses from our customers. Survey response rates are dropping across almost every industry.
One of the main reasons for the the drop of response rates is bad email list management leading to both email and survey fatigue.
Few if any researchers are taking the time to maintain good email best practices and segment their customer lists for the purposes of research.
By taking a few simple steps, you can take control of both survey frequency and selection bias for your customer surveys and improve your response rates and quality of your customer research.
In this article we’ll talk about sending out more surveys, but to smaller segments of your customer list.
Plus, I’ll walk through a very basic segmenting technique you can implement right now to be able to use these ideas for your organization.
How Many Responses Do You Really Need?
The truth of the matter is you don’t need as many responses to your survey as you’d think.
When it comes to survey responses, the percentage of your customers that answer isn’t important — what’s important is getting a valid sample and selecting respondents as randomly as possible.
Here’s a quick look up table that shows the number of responses that you need to collect to draw a conclusion about your customer base or target customers.
If you are going to draw conclusions about your existing customers then “population size” is your total customer count. If you are using your customers to draw conclusions about an entire market then the population size is the size of that market.
| Population Size | Sample Size (95% @ 5%) |
|---|---|
| 500 | 217 |
| 1,000 | 278 |
| 5,000 | 357 |
| 10,000 | 370 |
| 50,000 | 381 |
| 100,000 | 383 |
| 250,000 | 384 |
| 300,000,000 | 384 |
Table represents a confidence level of 95% (repeatability) and a confidence interval (margin of error) of 5%.
Unless your customer base is very small, you only need a fraction of your customers to answer each survey.
It’s best to spread the love and use good email marketing techniques to only invite as many customers as needed to each survey (and only invite qualified customers). This will let you continuously survey your customers for valuable insights without burning anyone out.
Because you can send out more surveys in any given time period, you can also focus your surveys on specific subjects and reduce the total number of questions you ask in each survey.
Frequent smaller surveys are much more effective than a single massive survey once per year.
Deciding How Often to Survey Your Customers
Before you start doing surveys with this technique, you need to answer how frequently you want to invite people to take your surveys. Also how frequently should you allow your customers to answer them?
If your customer base is quite large, I would suggest that you don’t invite customers to take a survey more than once every two months. If they answer one of your surveys, I suggest you give them at least three months before you ask them to take another one.
Ultimately, this decision is entirely up to you and is going to differ form company to company. Just remember that if you survey too frequently (or send surveys to customers that get disqualified on the second page) then you are going to start lowering your response rates and hurting your customer reputation.
Also remember, that your list size will dictate maximum number of surveys you can send in any given period.
If you use the table above, you should be able to do some simple math to plan out how many surveys are possible before you start repeating customers. Use your customer list size, email response rate, and needed sample size to figure this out.
# of surveys per period = (customer list size * response rate) / target sample size
Using this calculation, if you have a customer list of 100,000 emails, your response rate is 25%, and target sample size is 371 then you should be able to run 67 surveys before you have to worry about repeating! Kinda fun, right?
That basically means we can send out 67 surveys every three months (or 22 surveys a month) “safely” without breaking our frequency rule of no more than one survey every three months.
Try plugging in your own numbers. If you don’t know your own response rate, try experimenting with a 5% to 25% rate. Even if your response rate is unclear, it shouldn’t stop you from estimating how many surveys you can field.
Let’s see how to setup a system to maintain this process.
How To Setup Your Lists For Segmentation
In order to keep track of how often we have surveyed someone and provide random sampling capability to our email tool we need to setup our lists appropriately.
In essence we need to store two pieces of information (at a minimum) with each email address in our list: a random selection id and the date of the last survey this person took.
Most email tools let you create custom fields for contact records. Find that part of your email software (You may want to look in the documentation section under “custom fields”) and setup these two fields:
“SelectionId” – this is a random number that we’re going to assign to each of our customers. We’ll be using this to randomly select customers out of our list using normal filters in the send process. This is a quick way to support sending emails to a small randomly selected group.
“LastResponse” – this will store the last date that this customer answered a survey for us. This is what we’ll use to make sure we are distributing our surveys evenly, and never over-surveying a customer. It also ensures the customer doesn’t bias your research by answering more surveys than they should. You want to hear from a good sample of your customers — not just the same customers all the time.
Populating the Data
Getting the data into your email tool is the most tedious step in this entire process, but even this is not very hard.
I recommend exporting you contacts to Excel (Or a .CSV file) and using Excel to populate SelectionID.
Here is a quick formula that you can use in Excel that will create a random number for each subscriber in your list.
=FLOOR(RAND() * 10000,1)
Add this formula to the column called SelectionID, and fill the entire column to generate ids for everyone. Then, import the data back into your email program. (There will be repeats of selection ids — but that doesn’t matter much for our purposes.)
You’ll want to populate LastResponse after each survey project.
If your survey tool supports integration and/or API callbacks (like Alchemer) then this is relatively simple. You can place an action at the end of your survey to update the subscriber record.
If not, you just need to collect the subscriber ID as a hidden field in the survey and update the email list after your survey is complete. You’ll import the data again into your email tool from a .CSV file.
Once you have the fields populated in your email tool — you are good to go!
Segmenting on Demographics and Firmographics
Customers can’t stand being invited to partake in a survey only to be kicked out. Once they are disqualified from a survey, they are much less likely to try to take another one.
If you have disqualification questions in your survey, try to use demographic and firmographic based segments to exclude customers from an email campaign that would be disqualified from your survey.
Use the same technique for creating custom fields that we are using to control survey frequency, but instead of SelectionID use a custom group code or demographic field to differentiate customer types.
Your marketing team will likely also appreciate this — because it will give them a clearer insight into different customer groups for marketing and sales purposes.
Using the Segmentation to Send Surveys
Now we are ready to send out an email invitation to take a survey.
First, if it’s been a while since you created the SelectionID field, check to make sure you don’t have any new customer without a number. I recommend doing this on a regular basis — just schedule it so you don’t need to worry about it in the midst of your survey project.
Now create your email message and invitation as you normally do. When it comes time to choose the contacts to send your survey to, you are going to use the fields we just created to filter only the appropriate respondents.
Use your email tool to build a filter that completely excludes any customers that have taken a survey in the last three months (using the LastResponse field).
If you are also using demographic segments, add the segment you want to survey to the filter now.
Your email software should now have selected all customers who match your target audience for the survey who have not taken a survey in the last three months.
Fantastic, even this is an improvement. You could stop here and send out your survey, but it is probably still going out to too many people. So let’s use SelectionID to select a small random sample from this segment!
Add an additional filter to your campaign that pulls a range of SelectonIDs you set. Start at the bottom of the list with 0 to 500, then on the next survey use 501 to 1000, and so forth.
Make sure to select a wide enough range of numbers to cover your target sample size at your estimated response rate. Most survey tools will tell you how many contacts qualify for the filter before you send, so you can tweak it from there.
That’s all there is to it. This simple setup will let you control over emailing your surveys. If you don’t get enough responses, just do a resend and grab a few more SelectionIDs.
If you follow this methodology you’ll be able to run more surveys in a year (hopefully deriving more insights and positive change) and your customers will be happier too!
Good luck and good emailing!
Key points for this article (for busy people that like to skim):
- You only need a few hundred responses for a survey no matter how large your customer base or target market.
- It’s better to send smaller surveys to a minimum number of respondents than one large survey to many.
- Use your email software’s segmenting and filtering features to avoid sending surveys to unqualified respondents.
- You can dramatically increase your research opportunities by following some easy segmentation and email frequency guidelines.