You and your product team have invested countless hours building a killer product (at least, you want it to be a killer product). But how do you know if your users are getting value from the product? How do you know what parts work well for your users and which could be improved? What important functionality is your product missing? Product feedback from your users answers these questions and more. In this post, we’ll explore product feedback, explain why it’s important, cover different ways to collect it and explain how to use it.

 

What is Product Feedback?

 

Product feedback is the process through which you receive input about your product from users, prospective customers and even other members of your team. Product feedback can be as focused as input on one specific feature or as broad as a user’s overall satisfaction with their interactions with your company. While we focus primarily on how product feedback can be used by product teams, this feedback can also be used by sales, marketing, customer teams and others.

 

Why Does it Matter?

 

The market for SaaS products is more competitive than ever before. There are more products in the market, meaning more competition for your users’ attention. For your SaaS product to be successful, you need to keep your users engaged and delight them with a great product that they’ll continue to use, pay for, and recommend to others. To do this, not only do you need to build a great product, you also need to continuously improve your product to defend against competition, protect against churn, and ensure long-term customer loyalty.

 

This is where product feedback comes in. In order to ensure you’re consistently innovating in the right direction without overstretching your resources, it’s critical to engage your users in an ongoing process of product feedback, leveraging that feedback to inform and validate your product roadmap. Beyond validating your roadmap, the process of feedback collection is arguably the most effective way to keep your users engaged while simultaneously making them feel valued in process.

 

Ways to Collect Product Feedback

 

Although there are a number of different channels and tools that can be used to collect product feedback, we find it helpful to think about the different methods as either being proactive or reactive. The best user feedback programs will adopt a collection of efforts across both of these different approaches.

 

Proactive User Feedback

 

Proactively collecting product feedback means that you are actively soliciting feedback from your users and customers, rather than waiting for them to reach out to you. While there are many different proactive methods, the most common and accessible include user interviews, usability testing, focus groups and surveys.

 

User Interviews and Usability Testing (One to One Interaction)

User Interview

While we view user interviews and usability testing as two different things, they are similar in that they’re typically one on one conversations with users that allow you to go very deep into a single user’s perspective and experience.

 

Usability testing is focused on a specific interactive product deliverable, where you want to observe the user going through a set of tasks. This interactive deliverable is used as the basis for the questions and session agenda. There are many different methods for usability testing including usability benchmarking, lab studies, A/B testing, clickstream analysis and more, but we won’t cover them in detail in this post, as they often focus on a very specific or nuanced aspect of the product experience.

 

Most product teams adopt user interviews as part of their product feedback strategy. Interviews are a free form format, not focused on interacting with a specific deliverable. Some advantages are that interviews allow for in depth conversations that can range from the broad value of the product to specific feedback on individual product features. Because the conversation is happening in real time, the interviewer can ask the user to clarify responses to questions or go into greater detail on particular areas of interest.

 

There are some challenges with these approaches. Interviews and usability testing can be a very time-consuming way to gather feedback, both for the product team and the users who participate in the interviews, so they’re not scalable. This makes it difficult to gather enough data points to ensure that the feedback is representative of your broader user population.

 

In addition, because of the high-touch nature of user interviews and usability tests, there is a natural self selection process when it comes to recruiting participants. In particular, users who are already negative about your product are unlikely to participate, meaning you’re missing valuable feedback from some of your most powerful voices.

 

The free form nature of interviews also means that information collected risks being unstructured and disorganized. Many product teams that we’ve spoken to admit that, because of this, they often fail to effectively summarize and classify the information gained from interviews. Ultimately this means that while information gained during interviews has the potential to be very useful, teams often fail to take any meaningful action on it.

 

Do these issues mean you should engage in these efforts? Absolutely not. But it’s important not to overreact to a single user’s opinion just because the medium was so personal. You also need to complement these efforts with more scalable methods, such as surveys.

 

 

Focus Groups (One to Many Interaction)

Focus Group

Focus groups are a way of gaining feedback similar to interviews, but at a slightly larger scale. For early stage or concept-stage products, focus groups will be made up of individuals that are representative of the ideal customer profile and target persona. For products that have already achieved product-market fit, focus groups may be comprised of current customers, often the highest-paying or most strategic – referred to in many cases as a Customer Advisory Board

 

Advantages with focus groups are that, as with interviews, the conversational structure presents an opportunity to ask follow up questions for clarification or further detail. And the communal nature can unlock valuable threads of conversation that you otherwise wouldn’t have discovered in a one-on-one conversation with a single participant.

 

Some challenges with focus groups are that, although you may get additional perspectives, you don’t necessarily get deeper perspectives than in user interviews. In fact, participants are often less likely to openly share their perspective, and you’re not able to follow threads of conversation as deeply given the sensitivity to other participants’ time. However, you can always start with focus groups and request follow-up deeper dive user interviews with users who had particularly interesting insights in the group.

 

Focus groups also have a very high overhead cost from planning and logistics. The average cost of running a focus group is $18,000.

Surveys

 

Surveys are a widely used method of collecting product feedback. Advantages of surveys are that they can be conducted at greater scale, while requiring a much lighter investment of time for both the company and survey participants. There are a variety of tools that enable teams to easily build surveys and categorize responses in an easy to digest format, making it more likely that teams will take action on the feedback received.

 

Surveys can be distributed through different channels, but we’ve found that both participation rates and quality of responses are much higher when surveys are delivered in app. Here are some examples of product feedback surveys that can be delivered in app:

 

  • Customer Effort Score – Used to understand how difficult it was for a user to complete an action or task inside of your product. For example, asking a user “how difficult was it for you to complete onboarding?”, immediately after they complete onboarding. We recommend a seven point Likert scale with representative values from low to high of effortless, as expected, and impossible. By including a value of ‘as expected’, the user understands that not all tasks should be easy. For example, running a marathon isn’t easy. However, many people still do them knowing that completing a marathon will be a challenge, and the amount of preparation a runner did will change the difficulty of completing a marathon.
Customer Effort Score
  • Customer Happiness Index (CHI) – Asks a user’s satisfaction with a feature or flow after they have engaged with it. It could be customer support, any knowledge base articles, or specific features within the product. At Parlor, we like to use a 3 point scale of positive, neutral, negative, but it’s not uncommon to see 5 point scales as well.
Customer Happiness Index

Feature Fit Index (FFI) – Asks a user how disappointed they would be if a specific feature or functionality were to disappear. Use FFI after the release of a new feature (e.g. 30-90 days) or after the user has engaged with a feature a certain number of times. This doesn’t have to be complex. In fact Superhuman uses this question and only provides two inputs, thumbs up or thumbs down.

feature_fit_index

Regardless of which type of proactive feedback approach you’re using, it’s important to consider the motivation of the participating users or customers. An engaged user base will likely be intrinsically motivated to provide feedback and actively participate in efforts to improve your product. If you need to leverage extrinsic motivation to drive participation, the quality of the feedback you receive will likely be much lower. Feedback from an engaged user who is intrinsically motivated to help improve your product because it’s a valued tool for doing their job will be worth much more than an unengaged user who only agreed to take a survey because they’ll earn a Starbucks gift card.

 

Reactive User Feedback

Chain reaction

Reactive product feedback is any feedback that comes unsolicited. This feedback will usually be one of a few types – requests for something new, requests for improvements to an existing feature or a report that something is broken. You may also receive general sentiment feedback, for example someone who is unhappy with their overall experience with your company.

 

Channels for this feedback include other customer-facing teams within your company (e.g. sales and support) and the channels that they manage, as well as social media. Feedback may also come through an in-app system that allows users to submit feedback or specific feature requests. Other reactive feedback may not come directly to you. Customers and users may post feedback on online forums, review sites (e.g. G2 Crowd) or on their own social media accounts.

 

In addition to the above examples, you may also receive feedback in the form of signals such as product engagement (as seen in your analytics tool), renewals (or non-renewals), or referrals (or lack of referrals).

 

While you certainly shouldn’t ignore feedback that comes in through these channels, it’s important to consider the context in which it was received. Reactive feedback helps provide insights worth considering or pursuing, but it should be validated in a larger context before being acted on. Your product roadmap shouldn’t be driven by a request from a particularly influential salesperson, nor your CEO’s latest idea. Sure they should certainly be thoughtfully considered, but you should validate these inputs using the proactive and more representative channels and approaches outlined in the proactive section.

 

Some Guidelines for Incorporating Product Feedback

 

Consider who the feedback is coming from. Not all users are created equal, and you’ll probably want to put a higher priority on feedback coming from paying customers – particularly your most valuable customers – over free customers.

 

A high volume of feedback related to one aspect of your product doesn’t necessarily mean it’s the most important thing. High volume doesn’t necessarily indicate that it’s wanted by the most people. You may have a silent population that wants something else, but not every user or user group is equally vocal.

 

While much of the feedback that you receive will be related to the product itself, you may also receive feedback about areas such as onboarding, support or evening billing. Make sure to share this feedback with the relevant teams, and they should make sure to follow up directly with customers.

 

Don’t overreact to any one data point, since feedback collection isn’t the same as feedback validation. While it’s great that you’re receiving feedback from some of your users, you probably will need a different process to validate that feedback with a more representative group of users. For example, a customer may churn, telling you they switched to a competitor because that competitor has a better feature set. But it’s possible the real reason that customer churned was because a close friend works for the competition. Remember, feedback isn’t truth, it’s evidence to help you make decisions

 

Closing the Loop

Once you’ve incorporated your users’ feedback into your roadmap, don’t miss the opportunity to delight your customers. As you release new features that were driven by customer feedback, be sure to announce them and thank the users who submitted them. Your customers will be thrilled to see your product evolving based on their input, and they’ll feel that their input is valued. This process makes for happier customers, who will continue to provide you with valuable feedback and help boost your brand and reputation.