The single sheet solution
Marketing metrics might look easy to obtain, yet the huge volumes of data generated by reporting systems do not always add up to a true insight. David Reed finds out how marketing can give the board a more unified view of its performance
You know how much of your marketing budget has been spent. You know how many sales have been made. You believe the first was responsible for the second. But can you prove it?
Measuring the effectiveness of marketing has been the biggest head-scratcher for almost a century, ever since Claude Hopkins published Scientific Advertising in 1923 (and even he failed to include a chapter on metrics). Reporting to the board demands clear, top-line numbers. Whatlands on the marketer’s desk every week are hundreds of pages of data about each channel in use.
Somewhere in between there must be a way to show that marketing is delivering against business objectives, providing drilldown into the line-by-line detail as well as roll-up to some kind of index of performance. But if you feel that is beyond your grasp, you are not alone.
According to an interim report from a survey carried out by the Chartered Institute of Marketing with Deloitte, 56 per cent of companies rate their marketing measurement capability as non-existent or basic. Less than 10 per cent always set clear KPIs for marketing initiatives, while 19 per cent say that they never establish clear accountability for marketing.
In a sign that senior executives and marketers may be embarrassed by these failures, Nick Turner, head of marketing effectiveness services at Deloitte, says that, “response rate for the survey has been dreadful. That just goes to show where the centre of gravity in marketing is – if you ask about creative or brands, you would get ten times as many.”
The initial findings are still statistically valid and build on a picture first drawn by a Deloitte’s study in 2007, Marketing in 3D, based on 220 interviews with chief executives, marketing directors and finance directors. This found that 70 per cent did not strongly believe the role of marketing was clearly articulated within their organisation and 77 per cent said their staff did not fully appreciate the value of marketing.
If your metrics are not telling you something useful, don’t bother. If they are, do more.
“There is a big disconnect between marketers and non-marketers,” says Turner. “Chief executives are incredibly positive about the role of marketing in driving growth – 86 per cent said marketing is central to business growth.” This understanding is not being shared across organisations, however, with a lack of metrics showing just what marketing brings to the table.
It was this insight that led directly to the CIM research project which is confirming the findings from the first study. “As far as I can tell, there is little pressure to change measures to make them more effective and better able to demonstrate what marketing does,” says Turner.
His company has consulted with clients on building marketing scorecards. The first step always has to be “linking corporate measures of success to the activities of marketing”. This is often overlooked, even if marketers believe they are addressing corporate goals – the objectives stated in marketing briefs are frequently marketing-related, rather than business-aligned.
Changing that does require a considerable investment of time, especially as the slate of measures that emerges from this process will be highly specific to the individual business and its marketing resources. “We don’t go in with a raft of 50 measures and say, pick the ones you want to go after,” notes Turner.
Identifying how the business creates shareholder value should lead to clear areas in which marketing’s contribution can be identified, both on the growth side and also in terms of cost-efficiency – those media measures that all too often become the sole yardstick of marketing performance.
Response rate may look like a measure of effectiveness, but it is not – conversion rate is what matters to the business, for example.
If the cause-and-effect chain between marketing and shareholder value can be identified in this way, it enables a scorecard to be developed which has that all-important single sheet top line, but with detailed drilldowns beneath. “You need to go top down, not bottom up,” insists Turner.
If that looks difficult, it is. But it only requires hard work and concentration, rather than being impossible or needful of process re-engineering. The links are there as is the data – the challenge is understanding where to apply your measuring efforts and insisting that you get data from those points.
Compared to how the public sector needs to measure its marketing effectiveness, the private sector has it easy. Whereas commercial organisations are looking at value (sales, costs, margin), Government and its agencies are trying to achieve behavioural change.
Yet it is here that the most sophisticated measurement approach has been developed by the Central Office of Information and applied to complex objectives like smoking cessation and Change4Life for the Department of Health. “What we encourage the people we work with to do is ensure they have a clear goal in mind and some kind of line of sight to that goal,” says Marc Michaels, director of direct and relationship marketing and evaluation at COI.
COI has developed a framework that assists its clients to evaluate campaigns even when the desired outcomes might not be directly observable, for example if a smoker decides to give up on their own without seeking help through the NHS.
Michaels says that formalising measurement is important. “You get a one-off value from justifying your activity, but then you need to take the learnings or you will do the same thing again, but less efficiently. You might find you are buying a particular medium very efficiently, but what if it is not effective?” he asks.
The COI approach is made up of eight building blocks that create an holistic measurement methodology. The first of these involves putting together a clear picture of the marketing output to ensure the impact of each is visible and their interactions on each other can be accounted for, such as where PR might be extending the reach of an above the line campaign.
Accounting for factors such as word of mouth is another key step and one of the most challenging. “We are looking at better ways of analysing buzz and reputation management. There is a lot of work still to be done in trying to pull that together and understand it better,” says Michaels.
Digging into what the audience takes out of the campaign is an important research component. Michaels also stresses that it is vital to understand people who did not respond to the campaign. “The marketing journey is a series of triggers, enablers and barriers. People drop out on that journey – why? There may be some on the way you could have got if you did the marketing right,” he says. COI’s Artemis tool helps to map out that journey, identify those critical points and take metrics to see if people are getting through to the next step. “That is why I disagree with the view that marketing measurement should be focused solely on media Marketing is also operations. So marketing should know what happens next in the journey because otherwise it won’t know where it is getting it wrong,” says Michaels.
While other building blocks look at direct indicators of response to campaigns, one issue COI has to wrangle with is that of control groups. Marketers in commercial companies routinely exclude specific groups from campaigns in order to benchmark what happened organically and therefore how much incremental shift was achieved.
“We can have control groups around behaviours, potentially, but it is difficult in the Government space because you can’t stop somebody if they want to do something. You have to offer them the opportunity equally,” he says.
External factors need to be considered, from highly interruptive events like terrorist attacks or volcanic eruptions, to recent market shifts. When 9/11 happened, COI saw a drop off in uptake of information about pension provisions because people stopped thinking about their future, for example. Without accounting for these, any historical review of marketing performance risks a false assumption about the causes and outcomes seen.
“When one of the Big Brother contestants was trying to give up smoking, we ran ads in the commercial breaks and response rates went through the roof,” recalls Michaels. “There is a lot of data coming in from a lot of sources. You need to focus on your KPIs. If your metrics are not telling you something useful, don’t bother. If they are giving you a lot of insight, do more.”
You can have a single page report based on your database performance
Artemis is a sophisticated approach to marketing measurement that brings together hard and soft metrics as well as modelling the impact of non-marketing actions. By creating it, COI has been able to point to some dramatic benefits achieved for Government (and society) by its marketing but it is only the start of the journey. Commercial organisations may not have the same interest in behavioural changes of this sort, or the extensive resources available to undertake this range of measurement. Even so, it is possible to create a set of measures that do provide a clear indication of whether business value is being created and how marketing is achieving this.
Robert Colquhon, managing director of Direct Consult, has brought his experience in mail order and online retailing to bear on the issue. “The point I make to marketers is that you can have a single page report based on your database performance telling you how well you are recruiting customers and what they are yielding,” he says.
This report is created using key measures from the database for a specific period – such as this week or month – with comparisons to the same period 12 months ago. The database is segmented by active and dormant customers, with the active one split into groups based on the recency of their last purchase – 0 to 12 months, 12 to 24 months and 24 to 36 months. The average order value for each segment is then included.
“You build up a picture of how your database is growing and what it is yielding. The other element is how many new customers you have recruited in the last 12 months and how this compares to the previous year. It is very simple,” says Colquhon.
By including measures of dormant customers as well as the number of multi-buyers, the health of customer relationships is clearly indicated. “It is a complete top line that gives you conversion rates, revenues and return on investment,” he says. “If you know your business is making money, this is all that matters to you.”
In marketing, other measures will rise and fall. Cost per acquisition, for example, is rarely stable since media inflation, competitor buying and other external factors will push it up or down. There may be a target for CPA, but it does not tell you if marketing is driving the business forward.
“Those are secondary considerations if your database is growing and getting better,” says Colquhon. This simple dashboard should indicate shifts in critical customer behaviours, such as average order value rising or frequency falling. If these are outside of expected parameters or targets, they can be investigate by drilling down. If not, relax and watch the money roll in.
One reason why marketing has often shied away from creating value-based measures, as opposed to output-based ones, is that the organisation may accept performances that show up as negative. Many catalogues and charities will recruit new customers and supporters at an initial loss because they are confident of cross-selling and up-selling them into profit during their lifetime.
Looking at a suite of indicators that are permanently red could be dispiriting. Even so, there will be actions that the marketer needs to take, such as optimising the channel mix to get the breakeven point as low as possible.
This is where marketing KPIs driven by activity, rather than outcomes, can prove to be useful, but are still challenging to realise. “My big thing is the you need to create an optimisation score,” says Gavin Sinden, digital strategy director at Equi=Media. He notes that this is likely to involve a completely bespoke dashboard since no two companies have the same marketing profile.
The root of this dashboard is the classic flow of data from digital channels showing page impressions, clicks and visitors. By combining these with the budget spent achieving those clicks and running the data against the CPA model for each channel, the scorecard will present a hierarchy of how efficiently marketing is being carried out.
What Sinden points out is that optimisation is not an infinite gains process. “If you only had £10,000, first spend it on the most efficient medium – usually natural search. Then move to the next most efficient – probably paid search, followed by affiliates – and so on,”
In a perfect model, this process would just carry on until the marketer runs out of money. In reality, it hits the law of diminishing returns, typically at about 85 to 90 per cent of budget. That non-optimisable 10 per cent should then be directed towards testing new channels, argues Sinden.
The other issue with this approach to measurement is that the type of metrics provided in each digital channel can be very different from each other. “For example, in social media, it is how many people you are linked to and how many have signed up to follow you,” he says. You can build that into a figure for social media reach, but is it really comparable to how may people your banner ads have been served to? And do either of those help you to understand whether that marketing effort has generated valuable new customers for the business?
Finding data feeds is not the problem for marketing. Understanding which ones matter and building them into a cause-and-effect model is much more demanding. The irony of creating a single sheet report on marketing is that it requires hundreds of pages of data to create. But then, the best rewards often require the most effort.