Market research results should be interpreted carefully – very carefully

I received a fair bit of pushback on my column a few weeks ago, when I challenged some of the sacred cows of the research industry.

The Secret Marketer

Now don’t get me wrong, the Secret Marketer is a huge believer in good quality market research, and I have probably helped finance a few palatial homes and farmsteads for research agency owners, in my time.

But my contention is that research results should be interpreted carefully – very carefully.

Steve Jobs never rated market research – he thought it was the job of Apple to know what customers wanted before customers knew it themselves. Henry Ford is (probably misquoted) as saying that if he had asked customers what they wanted at the start of the last century… they would have asked for a faster horse.

I was talking to someone in the Energy Sector the other day, who was recounting that they had discovered some interesting results around door-to-door selling – the shorter the driveway, the higher the propensity to switch supplier. At one point, they thought this was to do with socio demographics (longer drives = higher net worths), but after careful analysis, they discovered that salesmen were reluctant to walk up long driveways, so rarely bothered and instead marked their call sheet as “not interested”.

Then there are the restaurant waiting staff who only hand over customer satisfaction slips to customers that they know are going to say good things about their experience.

And then there is the holiday industry – I saw a graph showing customer satisfaction (and net promoter scores) for customers on package holidays, against a whole host of metrics – airport check-in; airline food; comfort of bed; service from tour rep etc. – but there were two scores for each metric – one a good 10-20% higher than the other. When probed, it turns out the two scores related to the weather, and people marked airline check-in experience and mattress comfort higher… if the weather had been better on the whole holiday, than if it had been dreary throughout.

Now, as I said at the outset, I am a fan of market research – especially where NPS is used – but good marketers need to consider the results it produces very carefully indeed…

Recommended