Ask a product manager if they prefer qualitative or quantitative feedback and they’ll probably tell you that quantitative feedback is essential, and qualitative is annoying at best. Even though most product teams recognize that user feedback is vital to their work, they still treat quantitative feedback as legitimate data and qualitative feedback as “mere” opinion.

The problem is, all feedback is opinion, whether users express it through their comments or through their bounce rate. The difference is, only qualitative feedback lets you ask users why they’re behaving as they do. When product teams rely solely on passive, quantitative metrics, they’re merely monitoring their users, when they could be communicating with them.

Many product managers are justifiably skeptical about qualitative feedback because they’ve only seen what it looks like at its worst: anecdotes without context that leave teams chasing the whims of one user. But dismissing the whole concept of qualitative feedback because you’ve only seen it done badly is like saying you don’t like music because all you’ve ever heard is a kazoo.

There’s a right way to collect, manage, and validate qualitative feedback, and it can yield better products and deeper relationships with your users.

Why Product Teams Fear Qualitative Feedback

The reason so many product teams are dismissive of qualitative feedback is profoundly personal: interfacing with users is scary.

To put it simply, developers tend to be introverts at heart. So rather than talking to our users human to human, we’ve built systems that let us hide behind our computer screens and watch users because we find it less painful. And, of course, there’s nothing more painful than actively soliciting criticism of the product our teams made. Tracking your Daily Active Users (or some other numerical metric) stings a lot less than knowing that a human being clicked 😡 instead of 😍 on the feature you made.

So rather than engaging with difficult feedback, we created customer support teams to talk to our users for us. And while the work they do is no doubt valuable, it also acts as another barrier between the people making the product and the people using the product. The more barriers we erect, the less clear we are about who actually owns the voice of the customer.

When product teams seek out user opinion rather than simply monitoring their behavior, it doesn’t just yield valuable insights—it encourages customer loyalty.

When your users feel like they have the ear of your team, and you’re basing your choices on their input, they’re far more likely to stick with you.

The Right Kind(s) of Qualitative Feedback

Interpersonal challenges aside, the main challenge in getting useful qualitative feedback is choosing the right tool to measure that feedback. A lot of companies lean too hard on either deep feedback (like user interviews) or broad feedback (reddit-style reactions), but neither one in isolation is a silver bullet.

Product teams have practically limitless choices for collecting qualitative feedback: a thumbs-up/thumbs-down, a vote, a comment, a rating, a phone call. Each of these should be treated as a specialized tool that requires a different level of investment and yields a different type of insight.

For example, a user interview lets you dig deep into a particular user’s needs and suggestions. But unless you have a method of validating what that customer says, it’s just an anecdote. And if you attempt to verify those results with more interviews, it’ll take hours of work to get even a little bit more color around the original opinion.

In contrast, if you conduct a thumbs-up/thumbs-down user survey, you may get a lot of responses, but little understanding. Let’s say 50 users hit “thumbs-down” on a new feature: 10 of them might be upset you’re releasing this feature instead of the one they requested, 20 might oppose a new feature because they liked the old way of doing things, and 20 might be worried about how this feature will integrate with their existing systems.

With a binary survey, all these varied concerns are reduced to a single response, creating the illusion of consensus. Furthermore, if all your survey responses land in one undifferentiated pile, you can’t separate the opinions of the users you most value from the ones you can afford to ignore.

A Programmatic Approach to Qualitative Feedback

To effectively use qualitative feedback, you have to design an approach that combines deep insights from individual users with broad surveys that validate those insights. It’s that second part, validation, that makes for an effective program. Otherwise, there’s no way to differentiate between widespread, legitimate concerns and one person’s opinion.

The first step in designing a qualitative feedback program is dividing your users into cohorts based on their needs, their behaviors, and their value to the company. Then, choose the appropriate mechanism to acquire deep insights from the users who matter most through focus groups, usability testing, users interviews, etc.

It’s essential that product managers be involved with this part of the process so they hear user problems (and praise) firsthand. After collecting this information, product teams need to organize and summarize the feedback they’ve received and then pick the information they want to pursue further.

Once you’ve homed in on a particular piece of feedback, it’s time to crowdsource the validation of that feedback. Use a mechanism that allows you to talk to a larger population of users, like an in-app survey. If one user told you in a user interview that they found a new feature cumbersome to use, see if that’s been a common issue by measuring the customer effort score.

If you’re considering removing a feature, but you know from interviews that it has some die-hard fans, take a feature fit index to gauge the overall sentiment.

Both are forms of qualitative feedback, but they’re much easier to scale and organize than meandering comments or interview transcripts.

The beauty of combining deep and broad qualitative feedback into your program is that it makes all of your users feel heard, and it makes your most important users feel special. The chance to provide feedback humanizes your team and your product. Even if you can’t give everybody what they want, your users will know that you care enough about their experience to ask.

Love Your Users? Then Listen to Them.

Every product manager believes their work is driven by users, and that every new feature they release is in response to a genuine user need. But if you’re not willing to directly engage those users through qualitative feedback, you won’t get the full picture of what their needs are or how you could be meeting them better. You might even start blaming users when they don’t use your product as expected.

Or you could try something new. Take down the barriers, communicate more openly with your users, and start truly taking responsibility for their sentiment. It takes thought and care to collect, organize, and respond to qualitative feedback. But your users’ happiness and your team’s success are worth it.