The unspoken truth behind quantitative data: what they tell and don’t tell.

Companies want to know that what they are doing is right, that the service they offer caters for the needs of their clients and that the product they market is user-friendly. But most importantly, that want to be sure they offer the customer an effortless customer journey and an experience that will turn them into loyal customers. To prove to our CEO that we are doing the right thing, we need statistics and hard evidence. Nowadays you can’t buy anything without receiving an e-mail about how likely you are to recommend the service to a peer, or how satisfied you are with it. Companies are eager to know what a customer thinks of them.

Luckily there are a lot of different ways to find out what our customer thinks. In the past few weeks I documented the different techniques I encountered as a customer myself. And spoiler alert, it was always in the form of a quantitative metric.

Some examples

I went to a telecommunications company for some billing information and received an invitation to share how satisfied I was with the service they had just offered me. Now, that question in itself is more complex to me than my just having to give a score from 1 to 10. Since there was so much more to my experience, I wanted to justify the number I would give. However, there was no room for that. In the store there was a big plate showing the scoring rules: if I was satisfied I had to give a score of 9 or 10, 8 or lower meant I was dissatisfied. I found that a bit harsh and decided not to give any opinion. Later that week I saw a similar picture on LinkedIn. Since when does 8 resemble an unhappy customer? I had the feeling that they just wanted to show a very high score to their manager and say, “look how great we are doing”.

Source: LinkedIn

Another example: When I booked a flight recently, once I had paid the ticket I received an email asking me to describe my booking experience. I was faced with 2 pages of questions on how likely I was to recommend the company to my friends, how I got to know them, how I booked, they literally wanted to know everything. This is perhaps understandable, but there was little space to explain everything, the multiple choices were limited and filling in this form completely would have given them a wrong impression of my experience.

The pitfalls of quantitative metrics

Both these encounters with scoring bring me to the queen of all scoring metrics: the Net Promoter Score or NPS. NPS measures how likely it is that you would recommend a certain product/service to a friend or colleague. The rules for NPS are that a score of 9 or 10 are promoters of your product, 6 or lower are detractors who will criticize your product. The people who are moderately satisfied but probably won’t recommend your product because they feel neutral, score a 7 or 8. In the calculations of the NPS, these passive people are not included and therefore don’t count as a detractor.


Getting a score of 7 or 8 is not necessarily a bad thing. In a 2016 research, Nielsen Norman states that the average satisfaction score of a website lies around the 7 or 8 mark. This means that if your website is user-friendly and enables the customer to perform his tasks, you are doing well. But in order to turn your customers into promoters, you need to exceed their expectations.

In these cases there was no real conversation between the user and the company when it came to giving an opinion. In the above examples, the user has the opportunity to quantify his experience, leaving no room to explain why he feels the way he feels. And even if he did, it was his opinion rather than his behaviour.

NPS, or alternative metrics, are a great way to quantify loyalty, satisfaction and likelihood to recommend. Yet these scores don’t show the full picture since it’s not that a good score equals a usable website. It may occur that users struggle with your site and still give you a good score. The NPS does not take into account the task success rate or time spent on completing the task. As said before, this metric shows you what is happening, not why it is happening. If you want to exceed your customer’s expectations, you will need to find out what those are.

Getting an holistic overview of your customer

What I want to say is that there are different techniques you can use to learn different things about your customer. But if you want to have a holistic overview, it is best to combine different techniques. CEO’s need numbers, so this example is definitely good to quantify the satisfaction and the likelihood of a product being recommended. This quantitative example tells us what is happening: is the customer satisfied? Is our service good? Do we receive complaints? By tracking the NPS, gathering feedback and complaints and sending out surveys we get a lot of data. It tells us what is going on at this moment. But what it doesn’t tell us is why this is happening.

How can we solve that? By allowing the user to explain himself. However, the best way is to combine this metric with other qualitative techniques such as usability testing. You will see first-hand where a user struggles, what he wants to achieve and what his expectations are. It will allow you to turn that 7 or 8 into a 9 or 10.

To get an even better overview of your customer, it is best to combine a set of observational and attitudinal quantitative and qualitative techniques.


I hope this will help you in your next research when you’re trying to figure out who your customer is and what he needs. We can also help you by going into more detail.

Are you ready to get a holistic overview of your customer or would you like more information? Contact us via or call us on +32 478 40 51 86, we will be more than happy to talk to you. Or, should you be in the neighbourhood, why not pop in and visit us at Karel van Lotharingenstraat 4, doorbell 3.2, 3000 Leuven (Belgium). We do pride ourselves on our excellent coffee and tea!