Paper spat shows problems of online media measurement

Complaints have been made to the Advertising Standards Authority, and Simon Waldman, group director of digital strategy and development at Guardian Media Group, and former director of digital publishing at The Guardian, has posted a lengthy entry on his blog demolishing the Telegraph’s claims (www.simonwaldman.net).

But what this really shows up, as Waldman himself acknowledges, are the problems surrounding the measurement of online media. It’s ironic that a medium that can deliver such a wealth of usage data should find itself constantly criticised for inaccuracy, particularly when the methods of assessing other media are so flawed.

The problems are that there is no standard methodology for counting website visitors, and that each of them has technical complications that create differing results.

And of course, website owners haven’t always been above using whichever figures showed their site in the most favourable light.

None of this really mattered too much when budgets were small and online ad campaigns were little more than experiments. But the recent surge of marketing and advertising money online is having a significant effect.

The same circumstances that are seeing digital agencies becoming familiar with the attentions of procurement departments also mean that more attention is being paid to the claims media owners make for their sites.

With online channels becoming business-critical, it’s now crucial to understand the various methods of counting traffic and their individual shortcomings.

The two main approaches are panel-based and server-based. Panel-based metrics will be familiar to everyone from the broadcast world, and suffer from the same problems online as they do offline. Server-based techniques work by counting the number of visits to a website, but that in itself isn’t too illuminating.

For a start you have to split out all the visits from search engine spiders and the like. In addition, server-based techniques don’t tell you how many of those visits were repeats, for example.

So, in order to convert the simple visitor numbers into something more meaningful, sites track their visitors using cookies. These small packets of data are attached to your browser when you visit a site and identify you to that site, allowing it to make you personalised offers, call up your shopping list and so on.

It also lets the site measure how many different visitors it’s had in a certain period.

The picture is further complicated by the fact that people have started to delete cookies from their machines, usually for reasons of personal privacy: they don’t want the site owners to know exactly what they’re up to. And if someone deletes the cookie placed on their machine by your site, the next time they show up there, they’ll look like a different person, thus skewing your figures.

None of this is to say that one approach is necessarily better than the other. But the result is that unless you know how a site is measuring the figures it’s reporting, the discrepancies between the two methods can cause problems.

It’s also an issue of industry maturity. There are, of course, organisations that audit website traffic, most notably ABC Electronic (ABCe).

But unlike offline media, where almost every publication has an ABC certificate, the reach of the ABCe is limited.

In the days of the dotcom boom, few people felt the need of an audit. Since then, the vast majority of online media spend has been concentrated across a handful of properties, so there has been little incentive for smaller sites to be audited.

Now that the days of big online budgets are back, accompanied by a client interest in return on investment unknown at the turn of the century, it’ll be interesting to see whether there is increased pressure for sites to be audited.

If you were feeling cynical, you might also wonder if traditional media’s criticism of the failings of online measurement techniques carries more than a whiff of fear. The dotcom era jibe about no one knowing the click-through rate of a poster must have sent a chill through many an outdoor specialist, with the result that any of online’s failings in measurement have been seized upon in certain, offline, quarters.

So while The Guardian may be spluttering, the rest of the digital media industry could well end up being grateful to the Telegraph.

Anything that raises the debate about how websites measure their audiences is to be welcomed, since any improvement in understanding can only be of benefit to the professionalism of the sector and, ultimately, to clients.

• Michael Nutley is editor-in-chief of New Media Age