A guide for product managers to using NPS customer feedback

Image
Nov 2020

NPS for product managers:

Using customer feedback to create better products


As a product manager, you’re always listening to your customers, understanding the problems they’re facing and translating this insight to improve your product.

Whether you really are ‘CEO of the product’ or not (from Ben Horowitz who started this key discussion over 20years’ ago), it’s going to be part of your responsibilities to understand and track how customers use your products - and if you’re anything like us, that means being obsessive about analytics, about usage, about asking for feedback, and seeing how small fixes and changes to your product can improve the experience.

Measuring the experience

The good news is there are plenty of proven ways to help ask for feedback with most also generating a business metric you can use to track how your product changes impact customer experience. 

You can go deep into user testing to pinpoint and fix UX issues, for example, or use heatmaps to understand placement of function, or analytics to track user flow. All of these measures also work well with A/B or multi-variant testing as you try out several different designs, features or wording to see which works best for your core audience.

Away from analytics, there are also a number of metrics you could use - these are normally derived by asking your users to answer one or two quick questions. For example, you might use Customer Effort Score (CES) to gauge how easy it is for your customers to use a particular product or service. 

And then there’s Net Promoter Score (NPS) that is one of the most commonly used customer-focused business metrics. NPS is generated by asking your customers for their feedback, specifically if they would recommend your product. It’s a reliable measure of customer experience and satisfaction and helps track how your changes to the product or services resonate with your customers.

NPS for product managers

This article is about NPS and how it can be used by product managers and product development teams. We’ve got similar articles on the other business metrics and feedback systems you can use, and how to connect these directly to your analytics platforms to track user journeys for different types of user.

Define and track your business metric

The business metric generated from this, your NPS score, is calculated by taking the percentage of detractors away from the percentage of promoters. The passives don’t come into it at all, because they have no net positive or negative effect (though their comments will be valuable). 

So, if 50% of your respondents came back as promoters and 20% came back as detractors, your NPS score would be 30. If 20% were promoters and 50% were detractors, the score would be -50. Your NPS score can range from -100 to +100.

And the key to monitor and track is the trend of your score rather than the score itself. If you are trending up, you’re making positive improvements to make it a better customer experience.

What Outcomes Can You Expect When Tracking NPS? 

NPS gives you an immediate indication of whether people really back your product or not. Asking them whether they would recommend your product to colleagues or friends ups the stakes a little from simply asking whether they themselves would use your product. 

As a product manager, this indication of strong feeling either way about your product is invaluable - especially if you take the respondents’ comments into consideration alongside the score itself. NPS can help you identify what exactly people love about your product, find inspiration for new features and releases

To start, you’ll need to plan and run an effective NPS campaign, identify key takeaways and commit to implementing them in real life, which is what the rest of this article will focus on. We’ll cover: 

  • Planning your NPS campaign to maximise data reliability
  • Measuring your results effectively
  • Prioritizing and acting on areas for improvement
  • Using NPS trends to measure success

How Customer Feedback Can help you prioritise your Backlog

You and your product development team will have set out a development strategy and roadmap and you’ll be looking at the backlog of features and fixes on a regular basis. It all changes when you add customer feedback to the mix (as legendary boxer Mike Tyson memorably said that "everyone has a plan until they get punched in the face").

It’s fantastic to build an initial MVP based on customer feedback - but it also comes with a warning when you start to look just a couple of bits of feedback rather than that of the majority of your target market.

Beware of outlier opinions

Asking customers for feedback will give you incredibly valuable insight - but make sure you get a good response. Remember that the first-respondants are normally outliers - very happy or very unhappy about your product. You absolutely must hear these views and act quickly to fix any problems (or work with your fans if they’re promoters!). 

But for a product manager trying to prioritise a roadmap or backlog of features, it’s really hard if there are just a handful of incoming customer comments that force you to re-prioritise the requirements of a few users over the majority.

To help avoid this, make sure your response rate is as high as possible to provide a good level of all views from across your customer base. For example, if you have just ten results from 1000 surveys and half want a new feature that’s currently way off on the roadmap, what do you do? If you have a decent 20-30% response rate you’ll get a much more representative view that can help you prioritise.

Enhancing the customer profile is a good way to help categorise the feedback - for example, if you have three product plans (gold, silver, bronze) and you can identify that all the feedback comes from bronze users, it’s worth looking at other aspects such as pricing, onboarding and so on that might be different for this group.


Plan Your NPS Campaign to Maximise Data Reliability

For accurate results - and by extension useful, actionable data - think about how you put together your campaign. It’s important to think about when in the customer journey you ask a for feedback as well practical matters like at what time or via which format/medium.

Make sure that all your users, whatever their usage levels, give you feedback. If you have power users, why are they using your product so much? What’s their feedback and this could expose a feature set or specific use for a segment that you had not looked at.

As a suggested first step, think about why you want the feedback. Here are some typical reasons:
  • To understand satisfaction by customer segment or product type
  • to test a new feature 
  • to improve an existing touchpoint such as customer support
  • to improve onboarding
  • to better understand a process such as delivery
  • for broader goals such as looking for feedback about the overall product 
  • understanding brand perception
Each of these will lead you implement your feedback campaign very differently. For example, if you want to better understand the checkout process, make it of-the-moment so use a web pop-up survey and don’t send an email survey two days after the event. If you want to understand a process such as delivery, don’t ask for feedback at checkout but introduce a delay. And if you send a survey two days after a product has been delivered you’ll likely get product feedback rather than delivery feedback. All obvious, but worth mentioning!

The aim is also to look for channels or formats appropriate for actively engaging a wide range of your customer base. As a starting point, for example, you could send out an NPS survey to everyone who makes an online order (assuming GDPR preferences allow for this). Perhaps you could record NPS scores using tablet terminals at a particular event. If your customer base is highly mobile oriented, you could embed a survey into your customer app. 

Essentially, you’re planning your customer feedback approach like any other outreach campaign. Here’s a quick checklist of things to consider before you go live with your campaign

  • Which channels should we use to send out the survey? 
  • If using email, what existing insights do we have into subject lines, open rates, response rates etc from previous marketing campaigns? 
  • Are we targeting specific products or asking for an NPS score for the company in general? 
  • Are we tracking NPS as an ongoing process, or doing so via shorter, targeted campaigns?

Note on incentives

We’re often asked about incentives and how to use them to improve response rates. Yes, they normally work if you offer a voucher or discount or even a charitable donation. However, watch out as you’ll set yourself a high bar and to have ‘clean’ data comparisons you’ll need to continue offering the incentive long term, which could easily wipe out your budget. If you run with an incentive for the first two months then have three months without, you’re not compare like with like.


Using the right tools to ask for feedback

You can track feedback and even implement basic NPS-style surveys using a Google Forms and a spreadsheet if want to go the free route and work exclusively using email. However, survey software makes asking for feedback and analyzing the results significantly easier.

You might have a general purpose survey tool in place already that you can use - for example, both SurveyMonkey and MailChimp can be adapted for NPS email surveys use if needed. 

However, to give you the flexability of using feedback forms across formats (email/SMS/website/events) and for the specialist reports and tools to help you analyze and act on the themes and trends, you can implement specialist NPS software, which offers the following benefits: 

  • Multi-channel support so that you can push your NPS surveys where respondents are most interactive for example on your website or in-app or by email or SMS.
  • Advanced segmentation options so that you can view results via customer, product or brand segment.
  • Identifying themes and sentiment in comments, which helps you identify key feedback themes and areas for improvement. 

These features are built in to specialist NPS software products. And, given that the average price for a basic NPS software package starts at £40 per month ($49), this functionality doesn’t need to be a major investment. 

Try SightMill’s features for free to see how they could help you improve your product design 

What is NPS and what does it track? 

In a nutshell, Net Promoter Score is a measure of how much your customers love your products. It does this by asking them, on a scale of zero to ten, if they would recommend your product to colleagues or friends. 

There’s then a follow-up question where the customer has an opportunity to explain why they gave a particular score. 

What’s important is that this follow-up is an ‘open’ question ie it doesn’t ask what did you think of a particular feature or service. The moment you limit the follow-up to specifics, your customers no longer have a chance to comment on the entire experience.

Based on the score a customer gave they are arranged into three groups: promoters, passives and detractors.

  • Those that score 9 or 10 are called promoters. They love your product, and actively recommend it to others.

  • Those that score 7 or 8 are passives. They won’t be shouting about their experience with you, but they won’t have much to complain about either. These are people you have satisfied, but not wowed.

  • Anyone scoring a 6 or below is a detractor. This group has had a broadly negative experience with your company and can decrease your public image through complaints or warning peers off your product. 

When should you ask for feedback in a user journey?

Where you place your survey on your product, or when you ask for feedback in the user journey are incredibly important and can give you very different responses since, without realising it, you could be asking about different elements or features.

For example:
  • If you run an online store and pop up a feedback form just after an e-commerce transaction has completed, you’re really going to get feedback about the checkout process. If you send an email survey a few days later, once the goods have arrived, you’ll get feedback about the product. And if you send a survey a month later you’ll get feedback on either brand recognition or customer support (depending if they had problems).

  • Similarly, if you only send your NPS survey via email to customers on your ‘company news’ mailing list, your results might skew positive since it could only be reaching the account manager or the person responsible for purchasing, rather than the end users of your product or service. This might give you a warm fuzzy feeling inside, but you could be missing out on some important insights about what needs fixing too. 

As we’ve worked with a whole range of different businesses in different sectors, it’s interesting to see what works for different product managers. As a couple of very different examples: 

  • A clothing e-commerce site wanted to improve each step of its order process using NPS feedback. We worked with the product manager and developer to start with a survey after the payment completed (the ‘thank you page’) and tied this to their analytics to view customer acquisition vs feedback. We then moved the survey along the user journey to introduce a delay after the online order to get feedback on the delivery process, then a week later to get feedback on the product itself. 

  • A very different example is a catering supply business was sending email surveys to its mailing list, but this only reached the procurement teams. To help get real feedback from the chefs and catering teams, the delivery drivers were equipped with tablets and our mobile surveys to get immediate feedback.

Impact on user journeys

For website surveys, it’s good to remember how these impact the user journey. A pop-up survey has good response rates but interrupts the user journey, so don’t place it just before your ‘money’ page (eg a checkout page). 

Similarly, a tab survey has low response rates because it’s unobtrusive which also means its doesn’t impact user journeys. And you could always trigger an email survey from a website action (survey software supporting webhooks API will let you set this up easily) so you can send an email survey after a customer has completed an action on your website.

Timing

If you use SMS surveys to reach your customers, but they don’t look at their phones during the day, then make sure you send out surveys when they get home. You’re likely to get a better response rate but the downside is you’ll also likely get higher unsubscribe rates.

Segmentation of users

You can use website NPS surveys as another tool to help A/B test a new feature or design: display a pop-up survey after the customer has been using feature for a few minutes. And, as before, if you include customer profiling data from your CRM or CMS with your survey, you can then slice the results by segment or product category to give you a richer view.

Measure Your NPS Survey Results

Calculating your numerical NPS score is the easy part. We’ve covered that above - you simply take your percentage detractors from your percentage promoters. 

What’s more important is that you use the comments left alongside scores to understand why your score is where it is. 

Identifying The Feedback That Matters To You

Your survey respondents’ reasons for their scores broadly fall into two camps: 

  • Product-related: they enjoy using your product, love the great features it offers and will tell everyone about how it has made a certain aspect of their life much easier - or they dislike how cheaply it was made, how it differed to their initial expectations and how it broke within two weeks of receiving it.
  • Customer service-related: your customers love your short wait times - or they have been put off by inefficient service.  

As a product manager, you’re interested in the whole product experience, but you can normally directly fix issues from the first category as you focus on ways to create better products. Your customer service team can use the feedback to manage questions and continuously improve service levels. 

After calculating your numerical score, it’s vital that you mine the verbatim comments your respondents have left to establish the reasons for your score. Look for recurring product themes - people might love being able to open an account in 10 minutes, or the reliability of your biometric security login. Alternatively, they might be put off by how glitchy your app is or how it’s not optimised for their model of smartphone, or disappointed by the comparative lack of features in comparison to competitor brands.

If you add customer profile data - such as the user cohort (the date they signed up or bought your product) or segment or relationship manager - you’ll also be able to get additional insights. For example, new users with poor scores might be having problems with onboarding; long-tenure users with poor scores might be having problems with training on new features.

As a product manager, keep your eye out for the following comment themes. This is where you will get the most out of NPS as a tool for improving your product creation processes: 

  • Onboarding: how easy is to complete the first task?
  • General design: is the product designed well for its intended use? 
  • Build quality: are the materials used hardy, secure and at low risk of breaking? 
  • Value for money: does the functionality and reliability of the product match the price?

From here, you’ll be able to identify areas for improvement. Perhaps your product is designed well, but made out of poor-quality materials. Maybe your app wasn’t optimised for tablets, despite most of your customers preferring tablets over mobile. Perhaps you can identify one or two components of your products that wear out quickly. 

Wherever you’re falling short, identifying key issues is the first step to improvement.  

Prioritize Your Areas For Improvement and Act Accordingly

Whilst in an ideal world you’d be granted unlimited time, budget and personnel to make all the improvements to your product you would like, we all know 

This means that prioritization is key. Ask yourself:

  • Which issues can you fix within the constraints of your role and the political landscape of your business?
  • Which of these fixable issues is costing you the most in customer satisfaction, according to your NPS results? 

No-one likes to admit that office politics has any bearing on anything, but let’s be honest here. If your procedures for switching to a new supplier are so hellishly bureaucratic that it will take months to just start on any sort of meaningful change, or voiding your existing suppliers’ contract will take a lot of expensive legal work, this probably isn’t a place to start. 

By all means keep the pressure on - but see it as a long-term project that you can work on in the background. Partly because there will be plenty of meaningful improvements you can make in the meantime. Partly, to be blunt, for your own sanity. 

Which Improvements Should I Prioritize? 

Some teams love to get the quick fixes out of the way first. Whether that’s low-level debugging work, rewriting product instructions so they are clearer, improving packaging so that there’s a lower chance products arrive damaged, taking care of the small stuff all adds up! 

Alternatively, you could use your NPS results to determine which of your fixable issues are the most detrimental to customer opinion. If a complaint about difficulty of opening an account on your app correlates with a lower numerical score than other issues, it makes sense to prioritize reworking the account opening user journey.

Lowest effort/most difference are both excellent places to start - it’s up to you as product manager to strike the right balance for your organization. 

Use NPS Trends as an Indicator of Success

Ideally, as you implement your improvements, you’ll see your number of promoters increase and your number of detractors decrease as you continue measuring NPS. 

In other words, if the improvements you implemented to your product design processes, your customers will like your products more. If they don’t...well, it’s back to the drawing board. 

That’s why, as a product manager, you should be tracking NPS after major improvement drives as well as before. Keep an eye on those themes you identified in stage two - have they become less prevalent after you’ve fixed your priority issues? How have the issues cited by detractors shifted since new feature launches and debugs?

Don’t expect magical turnarounds immediately - you know that improvements can take a while to bed in (and in some cases you might see a dip in user satisfaction as people get used to them). So, cut yourself some slack with your NPS score and, if you’re going to work with targets as an incentive, keep these gradual.

Progress often seems incremental until you look back at how far you’ve progressed. A good cadence for your team might be:
  • real-time: act on responses and problems
  • weekly backlog prioritisation: consider feedback themes when prioritising backlog items with the business teams
  • monthly trends: look at how trends are changing and how this should impact resources and prioritisation
  • quarterly review: full review of themes and trends and the business impact on sales, retention, costs, etc

Bringing it together

Customer feedback is essential for the development of successful products and asking, analyzing and acting on feedback is key task for any product manager. 

Understanding how, when and where to ask for customer feedback, how best to use this in your product development team, and working up a simple schedule with your product marketing, business, sales and finance colleagues to review feedback should be really helpful for driving commercially successful product development that delivers a great experience for your customers.