There’s no perfect method for measuring creative

As the Cannes Lions Festival of Creativity draws to an end for another year, many marketers will be patting themselves on the back for wins at the awards. But are they really measuring the effectiveness of their creativity?

creativity

If you’re a marketer you measure creativity. This is not a ‘should you?’ question. The alternative would be – what? – to explain to the board you intend to invest tens of millions of corporate dollars in an activity that might – or might not, who knows- bring some kind of return. That would be like the operations director announcing the multi-million pound purchase of a new machine line but declining to say what function it is to serve, if any. So measure you must.

That’s the easy bit out of the way. The questions from here get tougher. When? How? With what objective in mind? With what tolerances for the inexactitude that characterises the creative process?

READ MORE: Are brands measuring the impact of creativity?

Contemplating that last point, it occurs to you, if it hadn’t already, that the interface between ‘measurement’ and ‘creativity’ isn’t an interface at all – since that sounds far too clean and clinical – but a messy collision of opposites. Precision and tidiness on one side; chaos and serendipity on the other.

Bringing the mindset of metrics to the domain of free expression is like peeling off a wide city boulevard to find yourself abruptly in the Latin quarter. The straight lines and rectangle turns are now behind you, and have given way to winding alleys and knotted junctions, with the turnoffs all on the slant. It would be fun if you didn’t mind getting lost, but in the wrong clothes, with the wrong attitude and an A-to-B objective propelling your steps, it is oddly uncomfortable.

That said, your first big decision is binary enough, or as close as it gets in this marketing neighbourhood. Do you measure after the creative work has been commercially exposed? Or before? Or both?

The tricky case of isolation

The virtue of the post-activity option is that it assesses the actual thing – the actual pack design, the actual YouTube spot – not an early approximation of it. And the metrics can have real-time, real-market validity – perhaps even getting as crudely commercial as looking to see if sales went up after the new work was unveiled.

Any marketer with a capacity for nuance would pause here, however, and consider the downsides. There are two. The first is the notoriously tricky issue of isolation. New creative material is often accompanied by substantive changes in the product or service offer. If sales rise, how do you know which bit did the work?

And no matter what post-activity metrics you use – awareness, net promoter score, volume sales – the data will always be confounded by the crazy asymmetries of the seething marketplace, and especially by competitor activity.

The second problem with measuring after the work has appeared is that the work has appeared. It’s too late to improve it now.

This stuff is hard. It’s exposing. You start out wanting to test some work, only to find yourself being tested.

Not that pre-testing is any cleaner. The first question to vex even the experienced and diligent is the level of veracity of the test materials to the final piece. Some marketers pride themselves on stripping out all ‘executional’ elements to gain a truer assessment of the deep thought within.

Yet one of the finest marketers of recent years – Unilever’s former CMO Simon Clift – maintained that marketers spend too much resource assessing the ‘concept’ and not enough on what they consider to be the ‘incidentals’. In his work on Olivio and Lynx he pioneered pre-testing of music, wardrobe and casting, since in his view they could violently polarise.

It’s your choice, but know this: whether you opt for simplified approximation or more realistic expression, you are going to have to live with imperfection. There are advantages and flaws to both, and zealots insisting you go one way should be heard with caution.

Chasing the metric

As big a test of your judgement will be the chosen methodology. It used to be simpler. Research agencies would, from a position of neutrality, report back on whether consumers understood, enjoyed and were motivated by prototype creative work.

Today, research companies are rarely agnostic. They come with their own views about what ‘works’ and assess your creative content against that preconceived ideal. One of the big sexy ones out there claims to know that advertising creative needs to be ‘emotional’ to perform at the highest level. Its single-figure metric is devoted to crystallising that.

The naive marketer will be seduced; it’s a good spiel. The sceptical marketer will probe the source of their certainty – and be told that it relates to analysis of past IPA Effectiveness Awards winners. Hearing that, and checking how those winners are chosen, the sceptical marketer is likely to remain sceptical.

These ‘single-metric’ research firms usually provide ‘guidance’ on how to attain the make-or-break number that deems the work worthy to go ahead: this kind of narrative, that kind of slogan, a certain kind of track.

But once a marketer starts chasing the metric you know that the spirit of creativity has died. The logic should be drumming away inside your subconscious: if big research agencies advise all their clients how to do it, and they all comply, convergence must be the outcome. Yet that breaks the number one rule of commercial creativity: be different.

None of these measurement decisions would be beyond the skills of a normally competent practitioner were it not for another factor: tick, tock. Testing eats time, and a crunch moment will inevitably loom. Mild panic is understandable: this idea did better than that one, and it may not be perfect, but – tick, tock – let’s go with it.

Making time to reflect

The better leaders remain calm, and insist on 48 hours they don’t really have, for reflection. The very best call for reflection even when the news has been positive; like elite sportspeople, they somehow find precious moments to eschew the good and go for the great, sometimes making it all seem easy.

It’s not. This stuff is hard. It’s exposing. You start out wanting to test some work, only to find yourself being tested. You hope the research will decide, only to realise the biggest decisions will be all yours.

You embark on a process with the laudable aim of measuring creativity – only for the truth to dawn, somewhere towards the end, that creativity is measuring you.

Recommended

Comments

    Leave a comment

    Subscribers get unlimited access to unrivalled coverage of the biggest issues in marketing and world-renowned columnists, alongside carefully curated reports and briefings from Econsultancy. Find out more.

    If you are an existing print subscriber find out how you can get access here.

    Subscribe now