5 Survey Terms Everyone Should Know to Run Better Surveys

Like most professions, the world of data collection is full of jargon.

Market researchers who spend years running complex projects can casually throw around sophisticated survey terms, but the rest of us need a little help from time to time.

That’s why we’ve compiled this glossary of five common survey-related terms that may be a little tricky to get right. Take a glance, and soon you’ll be talking about non-biased cross tabulation using panel data with the best of them.

#1 Accessible

– adjective \ik-ˈse-sə-bəl, ak-, ek-\

1. able to be reached or approached
2. able to be used or obtained

Accessible surveys aren’t just intuitively designed. They’re created in compliance with Section 508 accessibility guidelines. These guidelines were created to ensure that Federal agencies make their electronic and information technology (EIT) accessible to people of levels of physical ability, and they’ve been adopted by software companies like Alchemer as well.

When a survey is “accessible,” it works the same for respondents using speech to text software, screen readers, or keyboard navigability as it does for those responding in more traditional ways.

To meet these requirements, you’ll need to be mindful of the question types, fonts, alt text, and interactions. Here’s a rundown of best practices.

#2 Bias

– noun \ˈbī-əs\

1. a tendency to believe that some people, ideas, etc., are better than others that usually results in treating some people unfairly

Honestly, the Merriam-Webster definition doesn’t provide all that much clarity for our purposes, so let’s create another definition.

1. a distortion, either conscious or unconscious, of survey data. It may affect the accuracy of conclusions or actionable decisions based on that data.

There are several kinds of bias related to surveys and data collection, but the two most relevant to us are response and non-response bias. The first, response bias (Hawthorne effect), is a skewing of data due to quality of the questions asked or responses collected.

For example, if you ask, “Do you think purple is the best color in the world?” you’re going to get very different answers than if you ask, “Which color is your favorite?”

Always remember that wording a question in a neutral, fact-only fashion, can vastly improve the quality of data you will collect.

Another kind of bias to be aware of is non-response bias. This type of survey bias is related to the sample of respondents rather than the way a question is structured.

For example, if your goal is to survey an accurate sample of the population to learn about overall color preferences, you would try to target a perfect cross-section of the population to get accurate data.

The bottom line: surveying an appropriate and significant sample relative to your research goals is imperative in avoiding non-response bias.

#3 Crosstab Data Analysis

– noun \ˈkrȯs\ \ˈtab\

1. a type of analysis or summary for categorical analysis that usually concludes with a contingency table.

Cross-tabulation analysis, also known as cross-tab, is often used by market researchers to show the relation between two different variables (in the simple example below, gender and pet preference).

The value of this type of data analysis is being able to quickly determine the correlation between the two variables that might not be apparent when viewing the entire data set as a whole.

Simply put, a cross tab report allows you to cross tabulate at least two questions, one on the horizontal axis and one on the vertical axis. You can use cross tabs to analyze what is driving your survey results by more closely examining trends and patterns.

#4 Panel

– noun \ˈpa-n^əl\

1. a group of people who answer questions, give advice or opinions about something, or take part in a discussion for an audience

A panel in the research world isn’t too far from the standard definition above. Survey panels are simply a collection of people who have indicated a willingness to take surveys. They’re paid for their time and effort, and the organization who facilitates survey distribution to their panel audience (called a panel company) also requires a fee to access their audience.

Why use a panel?

Let’s say you’ve been assigned to find out what the target demographic of your new website (still in beta) thinks of your designs. But you haven’t launched the site yet, so you can’t depend on current customers to provide you with feedback.

This is a great time to employ a panel.

Panel companies collect a lot of data about their panelists, so you can get very specific about the type of people who want to take your survey. If our hypothetical website was designed to appeal to middle-aged men with families, we could easily target only those respondents with a panel.

We have several extensive guides to running successful surveys with panels, so you can make sure you’re spending this portion of your budget wisely.

#5 Piping

– noun \pī′pĭng\

1. to automatically populate content in a survey based on answers or data previously collected in the survey

Piping can be a very confusing term, because it’s an online survey that doesn’t have an actual pipe anywhere in sight. But piping simply means pre-populating a survey question using information you already have.

This could be pulling in data from a Salesforce record, or grabbing the name your respondent entered on the first page and using it to thank them for completing your survey.

Piping is a great way to reduce survey fatigue because it reduces the amount of data entry that a respondent needs to do. They only have to enter their name and favorite color one time; you can pipe their response into future questions to speed things along.

What Other Survey Terms Should We Cover?

Let us know what survey terms you’ve found confusing in the past with a quick comment. We’ll continue expanding our glossary to help demystify the survey process for everyone.