RAJAR meter testing needs industry synergy

It is right for industry-wide surveys to come under the spotlight. And at the moment, the limelight is on RAJAR, courtesy of Kelvin MacKenzie.

RAJAR organises the UK’s radio research, which is based on a diary system and is the most complex media survey in the world, the largest outside the US. We began testing two metering devices in January, the Radiocontrol watch and the Arbitron pager. The trial is both wide ranging and time consuming. It will look at whether the devices can be used from the moment of waking, and whether respondents will accept the devices, as well as the implications for broadcasters that use alternative media, such as the Internet. The testing should be completed in spring next year.

MacKenzie seems not to want a detailed test programme. For instance, his Radiocontrol watch system has not been used to measure child-listening. This audience is important to some stations, so one of our tests examines whether children aged over four can, or will, wear meters.

When our tests are complete, the facts will be presented to the board, along with cost implications. The radio and advertising industries (not RAJAR) will then decide whether to implement meter research and in what form.

It is difficult to comment on MacKenzie’s tests because, despite attracting publicity, he has released little information about them. However, there are some points that can be made both about his test and his views.

It is no surprise that different methodology (meters versus diaries) produces different results. That is precisely what has happened elsewhere in the world.

What is surprising is the degree of the variations. Some can be explained by a range of factors, from the different research periods to technicalities. For instance, RAJAR defines a listener as “having listened for five minutes”, change that to “listened for one minute” and variations would increase significantly. So perhaps MacKenzie presented one-minute data?

One can only speculate on why MacKenzie has chosen not to be transparent and why he is repeating misleading statements such as: “The watch meter cannot lie.”

This assumes that the difference between a diary and a watch is that the watch is accurate and the diary isn’t. This is untrue – a meter can easily produce inaccurate data if, for instance, it is not carried all the time. Also, he implies that meters are the only way to collect event-led listening. This is also untrue, the diary system can deliver this, given a three to fourfold increase in sample, the same size of sample that would be needed for meters.

The bottom line is the diary system has served everybody well, but if the radio and advertising industries decide to move to meters or a meter/diary combination that’s fine by RAJAR. Our job will be to ensure that it remains gold standard research – with or without MacKenzie’s practical or moral support.

Jane O’Hara is managing director of RAJAR