Quantitative research is a critical component of any product management team. It can drive growth, improve retention, and provide product direction.
Many software companies make a habit of tracking the quantitative performance of their product. It’s easy enough to monitor the use of your product, the growth of the user base, customer retention, etc. This quantitative data is informative and definitely tells you what is happening in your product and your market. What quantitative data doesn’t tell you is why something is happening.
A friend of mine recently shared a story that illustrated the importance of using qualitative research. As a product manager at his company, he was seeing a trend in the data—customer acquisition was up, but so was attrition. Their customers weren’t renewing their contracts. The CEO went into a panic. My friend decided to send out a survey to the customers who didn’t renew to get more insight into what was going on. As it turned out, a part of their product was malfunctioning and not charging the customers who were up for renewal. Crisis (mostly) averted!
As a product manager, it’s common to panic or get stuck when presented with aggregate data that doesn’t expose the “why” of customer behavior. Sitting and analyzing the quantitative data isn’t going to get you any closer to the why. You need qualitative data to get there.
As the pragmatic marketing NIHITO principle states, Nothing Interesting Happens In the Office.
Qualitative research is personal, field-based, and iterative. This type of research requires you as a product leader to connect with your market, ask them questions, and think deeply about their answers. As insights are collected through these methods, they are organized and patterns begin to emerge. When these patterns are paired with quantitative data, incredible insights are realized.
I’m not telling you that you should go out and conduct surveys and one-on-one interviews every single day. But there are some key situations where qualitative data makes a difference.
Before we dive in to the why and how, let’s take a quick look at the difference between qualitative and quantitative data.
Qualitative Versus Quantitative: At a Glance
Qualitative data collection helps to influence product direction, identify problems, assess sentiment. When paired with quantitative data, you’re much better suited to make decisions.
Quantitative research is deductive and hinges on the presence of a hypothesis, which is identified before research begins. Qualitative research is inductive and does not require a hypothesis in order to start the research process.
The data collecting process in qualitative research is personal, field-based, and iterative or circular. As data are collected and organized during analysis, patterns emerge. These data patterns can lead a researcher to pursue different questions or concepts, in a manner similar to rolling a snowball downhill.
Got it? If you’d like to go deeper into the discussion of these two data types, read our blog, Quantitative Versus Qualitative.
Ready to keep going? Let’s look into when and how you should use qualitative data in your research.
When Should You Use Qualitative Data?
- Service or product improvement needs have surfaced: If you find that your NPS score is decreasing, call volume for your support team is increasing, or that general product usage has decreased, qualitative research is a great method for problem identification and uncovering why these things are occurring.
- There are brand sentiment uncertainties: By understanding how your customers see your brand, you can react to problems and brand awareness issues. Alternatively, through qualitative research, you can surface social proof in the form of testimonials and case studies that can be used drive new customer acquisition and customer loyalty.
- Demands for product development and innovation: Customer (and non-customer) feedback can be used to understand trends. This insight can support the development of new products to drive business growth. Google does an excellent job of this—they develop ancillary and complementary products that support their existing users and products to increase their market share. Examples are Google Flights, AdMob, Google Play, etc.
- Need to improve marketing effectiveness: Customers insight and buyer/user personas that are developed through qualitative research help your marketing team create better and more targeted content. Additionally, those personas can be used to improve the language and usability of your product.
- Identify market fit and clarify positioning: If your sales team is losing deals, your product team should be conducting Win/Loss analysis. These kinds of conversations can surface critical information about market needs, gaps in your value proposition, which competitors customers are considering, and much more. This insight can help to clarify and inform your positioning and packaging and your general go-to-market strategy.
Some Easy Qualitative Research Methods
This is not an exhaustive list of methods for gathering qualitative data. Instead, knowing that many product managers are seeking fast feedback, here are few accessible options for conducting research that we recommend you try.
- Phone interviews: This is the simplest, fastest, and most universal way to explore your market. It’s useful to go in with a hypothesis and to ask open-ended questions that allow the interviewee the opportunity to fully share their experience, needs, and observations. It helps to have a list of questions to guide the conversation, but you don’t need to stick to a script. In fact, don’t. Let the conversation flow, follow its path, and practice active listening. You will inevitably learn something.
- Use a survey tool: By creating a short three-to-five question survey with open-text fields and disqualification questions, you can collect the answers and feedback that you need quickly. If you’re blocked by not having enough survey recipients, you can leverage a panel service to provide access to a larger audience. If you’re not familiar, these services help field respondents for you, so you can gather enough information for statistical significance.
- Use existing feedback systems: If you are surveying about product or customer service satisfaction, you may already have a wealth of data available. Ask your service team for ticket content and feedback forms that have open text fields in them. You may also have past surveys and exploration studies. Reduce, Reuse and Recycle — it’s not just for the environment anymore!
Of course, as with the other methods there is a “gotcha” here too. Just because the data is available doesn’t mean it’s appropriate or unbiased. Customer support transcripts and tickets are a wealth of information, but the people that contact support may not be representative of your entire customer base. Remember to qualify the data before you base decisions on it. - Ask the front lines: In some cases you simply may not be able to access your customers or target audience directly. Consider interviewing the front-line customer service and sales teams that interact with your audience. It may not be as great a solution as going direct to the source, but these folks usually have a good understanding of customer needs and issues in an organization. We recommend getting as close to the end-customer as possible for these interviews. Avoid interviewing management because they tend to pre-filter their responses.
Qualitative Research Can Be Used to Clarify Quantitative Findings
Should some quantitative information not be completely clear during the data analysis phase, the researchers can screen and identify respondents they believe can shed more light on the results.
With online surveys, you can ask respondents directly if they would be willing to speak with you about their answers. Give them the option of leaving their email or other prefered contact method, and your researchers can then reach out to gain additional clarity.
Using a modified member check technique common to qualitative research, the investigators can discuss the responses that participants provided in order to learn why they responded as they did.
The qualitative data can help answer the questions that clients and decision makers have about the quantitative research, providing rich descriptions and specific illustrations that enhance the quantitative data, and which can strongly add value and utility to the survey. And, by extension, bring clarity and direction to your next business move.