In the world of Product Management, quantitative data is of immense importance, which shouldn’t come as a surprise. Product Managers spend a great deal of time researching and analyzing what is often referred to as ‘hard data,’ meaning statistics, demographics, and website analytics. This information can be gathered through application metrics, A/B testing, and surveys. It is precise and measurable. Information gleaned through quantitative analytics may include statistics on how frequently customers visited the help page, what they were searching for, or how often a specific feature is used. There’s no doubt that this information is vital to Product Managers; however, it isn’t the only necessary data to develop a quality Product. The truth is, quantitative research is only one part of the equation. Quantitative analysis must be combined with an appropriate balance of qualitative research to get the full picture.
Qualitative Research: The Basics
Product Managers tend to focus heavily on quantitative research because decisions based on this research are supported by the numbers. It’s easy to overlook qualitative research when there’s a plethora of statistics and concrete data sitting out there waiting to be analyzed.
the insights gained from qualitative research aid Product Managers in advocating for their customers and their market. Qualitative research can tell the story behind the numbers; it’s detailed, descriptive, and provides the context behind customer behaviors.
Methods of Gathering Qualitative Data
Qualitative research might gather observations from numerous or varied resources, such as:
- Customer interviews
- Reviews
- Phone calls
- Email exchanges
- Help desk tickets
- Focus groups
- Surveys
- Casual conversations
Deciding which method to use will likely depend on the number of resources and time available. Larger companies often have more resources to dedicate to gathering quantitative and qualitative research; however, smaller businesses and startups will need to look to one or two of these cheaper and more efficient methods.
How Product Managers Can Use Qualitative Research
If quantitative data gives us the ‘what,’ then qualitative data gives us the ‘why.’ It allows Product Managers to see that several users have downloaded and uninstalled their app only a day later and gives the why behind that uninstall. “It wouldn’t load on my phone, so I gave up!” or “It didn’t do what I thought it would” indicate two very different problems.
Qualitative research provides context, reveals user’s opinions, and exposes the emotions the Product elicits in users. In the example above, if we solely rely on quantitative data, we might assume the Product isn’t any good and scrap it. However, by adding the context found in the user reviews, we get a clearer picture of the problem. We may discover a bug that needs to be fixed or realize we need to improve the Product description in the app store to reflect its purpose more accurately. Without this qualitative data, Product Managers don’t have all the information needed to make good decisions. As Pinterest’s Product Lead, Lulu Cheng, states: “If you just see a number go up or down week to week, that tells you what’s happening, but it doesn’t tell you why users are doing that particular action.”
Product Managers benefit from incorporating both types of information into their decision making process. Knowing how users experience the Product can even allow Product Managers to discover new features or opportunities that lead to greater consumer buy-in for the Product.
Strengths and Weaknesses of Qualitative Research
Quantitative and qualitative research both have a place in Product Management. There isn’t a one-size-fits-all approach that works for every situation, every time. This means that Product Managers must be flexible and have a comprehensive understanding of each type of research’s strengths and weaknesses.
Qualitative research is an excellent way to uncover the user’s thought process. It’s best utilized when Product Managers need to:
- Detect Product defects, flaws, or problems
- Branch out and explore new opportunities or directions
- Discover trends or patterns that would otherwise go unnoticed
As great as it is, qualitative research rarely provides a clear or direct path forward without quantitative data to back it up.
Quantitative research, on the other hand, appeals to many Product Managers’ desire for a straightforward course of action. Quantitative analysis is excellent for gauging user interest, determining the best phrasing or colors to use in an email marketing campaign, revealing customer preferences, or tracking metrics to discover patterns in data. Where qualitative research fails to deliver a clear-cut course of action, quantitative analysis often provides direct, actionable data for improving the Product or feature.
A Balanced Approach
Even though it provides Product Managers with facts and statistics, quantitative research doesn’t expose the user intent or motivations behind those statistics. And while we gain a great deal of contextual insight from qualitative research, it lacks a definitive course of action. Hence, a balance of quantitative and qualitative research is necessary.
Product Managers are charged with gathering and analyzing as much data as possible on their customers to direct resource investment. The difference between relying solely on quantitative research and incorporating both types of information is the difference between using a map and compass to navigate and having an actual guide who has traveled where you’re headed to lead you on your journey. You may get there using the map and compass, but you won’t have the insider information about shortcuts, hazards to avoid, and amazing views worth seeing just off the trail.