Torin Douglas: No clear picture yet from TV’s new ratings system

Viewing figures are scrutinised to the last 0.01 of a per cent, but BARB’s new data gathering system has yet to prove it can provide such accuracy, says Torin Douglas

Remember what life was like before the publication of overnight TV ratings? Of course you do, because this week we’ve gone back to those dark ages (or those more civilised, less ratings-driven times, depending on your point of view).

BARB and its new team of research contractors have suspended publication of the ratings for up to two weeks while they check the reliability of the data from their all-new panel of homes. The hope is that the new figures will quickly get a clean bill of health, but if previous changeovers are any guide we can expect at best a lively few months and at worst chaos, while the system finds its feet.

Like the euro, the new BARB was due to come into effect on January 1. Like the euro, the switchover had been planned for many months to ensure all went smoothly. And like the euro, the launch may have taken place successfully – it’s just that the central data bank won’t be able to tell us for a few days yet.

“Ratings fiasco wreaks havoc on TV”, ran the headline in The Guardian, the first paper to show interest in the story. But in fact, so far, the industry has been remarkably philosophical about the withdrawal of its daily bread-and-butter. This is partly because the TV and advertising worlds only returned from the Christmas break on Monday; partly because January is a quiet time anyway; and partly because BARB’s partners took a pledge of silence, knowing how easy it has been in the past to conjure up rows and “fiasco” headlines.

Official BBC and ITV spokesmen uttered soothing words: this was the biggest change in the ratings system for years, it would provide better data and it was important to make sure it was right. And this just days after the BBC had trumpeted BBC1’s 2001 ratings victory over ITV by a mere 0.1 per cent (later figures put the lead nearer to 0.01 per cent, which to most people is practically a dead heat), while ITV was pointing out that it still led in peaktime, and the multichannel broadcasters were claiming to be the year’s biggest winners, with a three per cent rise in audience share.

If anyone had doubts about the importance of BARB’s data, this was not the week to air them. The fact that BARB has changed its plans and withheld the figures for a few days may be just commendable prudence. But if a 0.01 per cent audience share is so vital to broadcasters, what are we to make of the revelation, in a press release on December 20, that in early “parallel runs” comparing the two panels’ viewing, the new BARB panel recorded five per cent less viewing overall than the old? And that multichannel viewing in multichannel homes, which was expected to rise under the new, supposedly more sensitive, system, fell?

The answer is “not a lot”, partly because the “parallel runs” are based on a two-week period, which isn’t long enough to form sensible conclusions. Also, the panel is only 4,000 homes strong, rather than the intended 5,300 (the full panel should be running by March). BARB, however, insists that even at 4,000 homes, the new panel is better balanced than the old one and the data will be at least as robust.

“This is the first time in over 30 years that a new panel is being recruited from scratch, and given the complexities of the service it is prudent to ensure that the system is thoroughly checked and meets BARB’s high standards before releasing data,” it said in a statement.

It wasn’t meant to be like this, of course. From the moment 20 months ago when BARB decided to replace Taylor Nelson Sofres, its main research contractor which had handled the contract for more than 30 years (originally under the name of AGB, not to be confused with the other AGB Group that owns the new main contractor, ATR), there have been fears about the impact of the changeover.

ATR did not even have an office in the UK, and was proposing a different type of viewing meter. There’d be a new and larger panel. The question was, would it be set up in time? And, if so, how would its figures compare with the previous system? BARB chief executive Caroline McDevitt said then: “There’s always risk attached to change, but it’s a managed risk.” Despite the delays, she remains confident that the new system will prove robust and points out that huge changes have been achieved in a short time, with staff working over Christmas.

But the initially proposed six-month trial period, including a three-month parallel run, hasn’t happened. During BARB’s last major upheaval, in 1991, a three-month parallel run went fine for two months (with only a slight difference between the old and new panels), and then, in the last week, a gap suddenly opened up, widening to a 13 per cent difference between the two panels’ figures for all adults and an even larger amount for some sub-groups.

Three months later, BARB’s then chief executive Bob Hulks told the Media Research Group conference: “BARB had not foreseen these problems and in hindsight would have wished the parallel run to have been longer.”

This time BARB has undoubtedly been better prepared and has foreseen most of the problems. But its parallel run has not been long enough and only time will tell how different the new figures will be – and which channels, if any, will be the winners and losers.

Torin Douglas is media correspondent for BBC News