‘Customer satisfaction scores cover all manner of sins’

Customer satisfaction scores such as NPS hide your brand’s problems. Instead of focusing on a simplistic number, take a big data approach to analysing customer comments says University of Cambridge researcher Mohamed Zaki.

Customer satisfactionEvery CEO loves numbers. So when you receive the results from your company’s latest customer satisfaction survey showing most of the performance ratings are at the top of the scale between seven and 10, you might start patting yourself on the back. What you don’t realise is that the glossy chart-riddled report on your desk has lured you into a false sense of security.

Customer satisfaction is now the major differentiator between competitors, so it is not surprising that improving customer experience is a top priority for businesses looking for a competitive advantage. Even better is when your company tops the chart in sector-wide independent research, such as the annual Which? consumer survey. However, the results do not actually reflect the true situation.

READ MORE: Why US brands are crushing the UK on customer experience

This is because most of the survey tools being used to measure customer happiness are simplistic numerical ratings that not only do not provide particularly valuable insights. They are also masking unhappy customers.

In fact, the surveys most commonly used by companies around the world, where customers are asked to rate a firm’s performance on a scale between zero and 10, are frequently mislabelling customers as ‘satisfied’ or ‘loyal’. Our research at the University of Cambridge has found that up to 42% of the supposedly satisfied customers who have given a rating between seven and 10 are unhappy with one or more aspects of the company’s products or services, and may already be considering changing their allegiance to the competition.

Net promoter score does not correspond to customer behavior or even correlate with changes in company revenue.

There are a number of single-question customer satisfaction metrics being used, such as the net promoter score (NPS), customer satisfaction (CSAT) and customer effort score (CES). NPS is one of the most popular and claims to measure a customer’s overall satisfaction with a company’s product or service and the customer’s loyalty to that brand. It simply asks customers a single question: on a scale of zero to 10, how likely are you to recommend this company’s product or service to a friend or a colleague?

But customer loyalty is multidimensional and it is not enough to rely on a single measurement alone to predict whether your customers are going to stick by you. Even worse, we have found NPS does not correspond to actual customer behaviour or even significantly correlate with relative changes in company revenue. The simple fact is that companies are losing customers despite having high NPS scores.

I am not saying that we should completely dismiss single-question metric tools like the NPS, but companies should also consider using other data sources.

Focus on customers’ comments

So where do you start? Taking a deep-dive into your customer accounts is a good first step. Your customers’ transaction history, such as spending value and frequency, holds valuable insights. Any change in behaviour may indicate a change in loyalty status.

And back to that customer satisfaction report sitting on your desk, how do you identify the 42% of ‘false satisfied’ customers that are actually dissatisfied and find out what their problem is? The answer can be found in customers’ verbatim comments within the open-ended answers of surveys.

While comments in the open response section provide more in-depth feedback, the information they contain is often not utilised very well by businesses. The sheer variety of comments can be daunting and difficult to analyse. In contrast, numerical scores are easy to compare and turn into impressive looking graphs, but they give only limited insight into underlying concerns or suggestions for improvements.

The standard process to recover service failure may not be the right way to make all your customers happy again.

Manually analysing open-ended answers and comments in surveys is time-consuming and it’s easy to miss things. This is where big data analysis can help. If you have a fully automated model that can capture emotion and the complaint status (whether it is an initial complaint or an ongoing issue), then you have a valuable tool that can tell you if customers are starting to become dissatisfied.

Also, you have to consider and understand your customers’ culture and language differences. People from different backgrounds have different ways of expressing complaints and compliments about a service. Thus, the standard process to recover service failure may not be the right way to make all your customers happy again.

READ MORE: Memorable experiences set brands apart

Combining these insights from direct customer comments with analysis of customer transactions and other sources can provide companies with a bespoke 360-degree view of the loyalty status of their customers.

The goal is to use those insights to diagnose the underlying factors causing customer dissatisfaction and then write a prescription to cure it. I have been working with a number of UK B2B companies through research undertaken at the Cambridge Service Alliance to generate this kind of prescription.

So, next time your customer survey spits out a number suggesting everyone’s happy – raise a quizzical eyebrow – and reach for some big data. Your customers will thank you for it.

Mohamed Zaki is deputy director of the Cambridge Service Alliance and senior research associate with the Institute for Manufacturing at the University of Cambridge.



There are 2 comments at the moment, we would love to hear your opinion too.

  1. Jim Norris 14 Jun 2017

    Rig the question and you can get any answer you want. Put the money into helping those who take the trouble to complain rather than silently never buy from you again. Learn from them, look after them and the good publicity will go viral. Opinion companies will tell yo what they want you to hear to keep the business. Customers looked after will be your ambassadors. Like sport, it’s not how you play when you are doing well, rather how you play when you are doing badly – and you will be doing badly for a reason.

  2. Jason Conrad 14 Jun 2017

    I work for a company (ForeSee) that measures CSAT, NPS, Customer Effort, and more. I once had a customer tell me that all the peer reviewed research he had seen showed that CSAT, NPS, Etc., was not predictive of changes in financial performance, so why should he measure it? Great question! He’s right…I’ve read the same research. Most measurements of CSAT fail because measuring an unobsevable concept like CSAT is hard. NPS has statistical “challenges” that are well known in academia. As you correctly point out, satisfaction is a multi-faceted and multi-dimensional concept. There is a way to measure it correctly that has been peer reviewed with proof of driving financial results (See the University of Michigan’s ACSI). If you do it right, then measuring success from the customer’s perspective enables management of that experience.

    If you are measuring it right, you then should benchmark that score against yourself and your peers over time which puts the data into perspective. Segmenting the data based on those who are less satisfied can be powerful as well. Then combine it with text and sentiment analytics to add color and more perspective. There are ways to get this right, but it takes a sophisticated, rigorous and scientific approach.

Leave a comment