Seeing isn’t always believing in the world of media metrics

What do food, sex and TV have in common, apart from being rather enjoyable? If I add exercise, is that easier? Quite simply, you cannot trust what people tell you about how much they do any of the above. All are notoriously subject to distortion and self-delusion when people talk about their consumption of, or participation in, these four activities.

TV has managed to acquire layers of guilt that are not evident when people talk about other media consumption. Even if you’re reading about Lindsey Lohan, listening to Terry Wogan or trying to flog your old garden chairs on Ebay, these media activities have acquired more respectability than watching telly.

Some of this "distortion" is perfectly conscious self-image management, but just as much is happening unconsciously. This makes researching media habits through the use of questionnaires or self-completion diaries of limited value. If all you want to do is compare Grazia to Red, or TalkSport’s performance this year to last, the shortcomings of this approach matter much less and indeed the National Readership Survey (NRS) and Radio Joint Advertising Research (Rajar) do a very good job for those purposes.

Inter-media decisions are much trickier. Even good old TGI is full of traps; it’s common to see a media plan favour a heavy cinema-goer (at least two visits a month) over a light TV viewer (seven to 20 hours a week).

So the arrival of the Institute of Practitioners in Advertising’s Touchpoints was welcome. Understanding how media work together is vital for properly integrated planning. The IPA’s Touchpoints makes consumers the focus and tracks how an individual moves between media throughout the day. Employing PDAs to capture media usage as near to the moment of consumption as possible, this single source study is startling in the brutality with which it exposes the time spent with each medium.

Touchpoints is not intended to give a complete answer; time spent with a medium conveys neither the scale of brand opportunity in that medium nor the quality of that exposure. I’m sure you will have sympathy with me when I say that I don’t think a minute of watching TV is comparable to a minute of radio or of e-mail. But, ironically, Touchpoints has a better chance of picking up time spent watching the newer forms of television, via mobile and broadband, than Broadcasters’ Audience Research Board (BARB), which does not yet incorporate out-of-home or non-broadcast TV.

Moving away from media metrics to response and sales, the dangers of linear analysis mean that response/sales-driving media – such as TV and outdoor – are in danger of being inadequately valued compared with response/sales-capturing media – like online and direct marketing. Only thorough econometric analysis will properly attribute effects to causes.

But between the opposite ends of media measurement and advertising effectiveness, there are important intermediate indicators of marketing effectiveness/ brand awareness, consideration, image, and recall of advertising being just a few. But the more we learn about how the brain functions and the difference between explicit and implicit memory the more question marks emerge about some approaches to ad-tracking.

Last year Thinkbox undertook a major ethnographic study to find out how people really watch TV. We studied 22 families for six weeks, placing cameras inside TV sets in different rooms and monitoring the viewing of 16,000 commercials. Apart from underlining the continuing centrality of television in people’s lives and the ability of good TV advertising to engage and captivate people, even when prior attention seems low, it has taught us a great deal about the vast differences between real and claimed behaviour.

One great benefit of the research was being able to present people with their own real behaviour, which often contradicted what they claimed to do (for example, "I never watch ads" or "I always fast-forward time-shifted breaks"). This provoked all sorts of memories that would otherwise have stayed beneath the surface. We also followed up the ethnography with a memory study that used new techniques to uncover implicit memories.

Having invested so much time, effort and money in this project I now can’t help but get irritate when yet another flimsy piece of media research is trumpeted, often via a self-selecting online questionnaire. People are barely aware of what they do. The reported levels of explicit recall of ads only scratch the surface of what has entered the long-term memory to influence perceptions and behaviours, but it takes research techniques that go beyond simply asking straightforward (and often misleading) questions to unlock these effects.

In 2006 there was a plague of studies, all relying on claimed behaviour and carried out by online companies, which fed the "death of TV" hysteria. And yet, by the end of 2006, real viewing behaviour caused TV impacts to increase to an all-time high and the biggest brand successes of the year, Marks & Spencer and Magners, were significant users of television. I hope journalists – and marketers – ask as many questions about the methodology employed before they give credence to sensational research headlines in 2007, whether it’s about TV or not.

Tess Alps is chief executive of Thinkbox