Measure what matters

Measurement is central to market research, says Luke Sehmer, and knowing the right questions to ask and how to ask them is essential .


Sponsored by Research Now

As consumers our brains are constantly measuring the world around us. Will my car fit into that gap? How long will it take me to get somewhere? How many calories are in this? As marketing professionals we also measure what is going on around us, including measuring key performance indicators to evaluate performance, determining the likely uptake of a new product and calculating return on investment. We seek meaning from our measurement and look for the edge from our understanding.

The vast majority, if not all, of market research starts with the aim of gaining greater knowledge and understanding to make better decisions. It is measurement that forms the foundation which enables insight to be derived. 

In other words, if you are conducting research to help you make better business decisions, make sure you measure what really matters. 

Identify business objectives 

To make better business decisions, we need additional knowledge and understanding. We need to establish the information that will help mitigate risk and help us make the right decision to achieve the desired outcome. Determining the elements that matter is essential for an efficient route to better business decisions. 

There are many approaches to identifying what matters, ranging from scientific assessments to intuitive judgement acquired through experience. Regardless of the technique, a combination of analysis and judgement is required.

Not asking what matters at the start of any research may limit your thinking to what you believe is important rather than what actually is. For example, in customer experience research, if you begin with what you think matters to your customers, you might miss something. Instead, allow customers to identify your weaknesses – although they can do that only if you ask them the right questions in the right ways.

Establish specific research objectives 

We should feel encouraged to engage our brains and question what matters in each case. Measuring everything in the hope that we will be able to derive insight at the analysis stage is inefficient and risky. Thoughtful consideration of the commercial context and the decisions to be made will help us to form specific research objectives and measure what matters.

Start with what the business is going to do with the information and translate this into specific research objectives – yes, they are different. Once we have these we are on a clear road to agreeing what matters for that specific piece of research. 

Avoid asking unnecessary questions

We have a duty to those who are taking part in our research. Even though they may be incentivised, we should not take them for granted; do not ask unnecessary questions, try not to bore them and value their input. Questions that are irrelevant or repetitive will undermine respondent engagement and may lead to incoherent, poorly considered responses and could even compromise the data. 

Do a sense check 

Part of good research practice is to probe around the parameters of research objectives by engaging in lateral thinking. Do your hypotheses stack up? Is there anything in a different industry that we can learn from? Am I asking the right people my questions? How would I answer the questions?

Determining the elements that matter is vital when making business decisions

Furthermore, is there anything already out there that might help? Are you already measuring it? Has someone else measured it before? 

A quick common-sense check can uncover errors in design and ensure the project is on the right track.

Take an objective-led approach to questionnaires

Measurement is usually associated with quantification, and while the term should not restrict our thinking to quantitative research, a large proportion of market research is conducted via surveys. Intelligent questioning of enough of the right people can often provide the information we need. Given that we know what we want to ask them, we must also consider what the right way to ask them would be.  

It is accepted by most that intelligent questionnaire design is both an art and a science. We can apply particular formulae, there are some governing rules and specific techniques, but the question construction itself and flow of the questionnaire also relies on artistry. 

In conjunction with this, we must insist on objective-led research design to help us keep the measurements focused on yielding information which supports greater understanding of the research question.

Choose your tools wisely 

In addition to intelligent questionnaire design and the careful selection of samples, survey technique and survey tools are crucial to get the most out of research. The market research industry has been talking for years about the benefits of shorter questionnaires, and we all know that something should be done about hideous grid questions. 

But is anything actually being done at the coal face? Data collection agencies would argue the answer is yes. We should be making the most out of sexy survey formats and rich media question designs, and we must allow respondents the freedom to complete questionnaires on their terms, irrespective of their chosen device, location or time of day. 

Best practice questionnaire design is currently specific to the data collection method, and given we want to give the respondent the choice over their response method, it is a very real challenge to accommodate those varied needs in questionnaire design.  

I do not pretend to have all the answers and the industry will continue to work towards technical solutions to offer device agnostic approaches. A large amount of work has been done, is being done and will be done to assess the differences in data collected using different methods. 

However, we can start by working hard to focus our questioning on what matters and living and breathing the objective-led approach.

A final piece of advice

Research techniques will continue to evolve, and we should continue to make the most of them. Right now, though, ask yourself this: What matters? If you can answer this then you are much closer to success and it will also help you accept any compromises in your research approach that you have to make along the way. 

Luke Sehmer
Senior research director
Research Now

160 Queen Victoria Street
London EC4V

T: 020 7084 3000