Paper spat shows problems of online media measurement

Does traditional media’s criticism of the failings of online measurement techniques carry more than a whiff of fear?

Complaints have been made to the Advertising Standards Authority, and Simon Waldman, group director of digital strategy and development at Guardian Media Group, and former director of digital publishing at The Guardian, has posted a lengthy entry on his blog demolishing the Telegraph’s claims (

But what this really shows up, as Waldman himself acknowledges, are the problems surrounding the measurement of online media. It’s ironic that a medium that can deliver such a wealth of usage data should find itself constantly criticised for inaccuracy, particularly when the methods of assessing other media are so flawed.

The problems are that there is no standard methodology for counting website visitors, and that each of them has technical complications that create differing results.

And of course, website owners haven’t always been above using whichever figures showed their site in the most favourable light.

None of this really mattered too much when budgets were small and online ad campaigns were little more than experiments. But the recent surge of marketing and advertising money online is having a significant effect.

The same circumstances that are seeing digital agencies becoming familiar with the attentions of procurement departments also mean that more attention is being paid to the claims media owners make for their sites.

With online channels becoming business-critical, it’s now crucial to understand the various methods of counting traffic and their individual shortcomings.

The two main approaches are panel-based and server-based. Panel-based metrics will be familiar to everyone from the broadcast world, and suffer from the same problems online as they do offline. Server-based techniques work by counting the number of visits to a website, but that in itself isn’t too illuminating.

For a start you have to split out all the visits from search engine spiders and the like. In addition, server-based techniques don’t tell you how many of those visits were repeats, for example.

So, in order to convert the simple visitor numbers into something more meaningful, sites track their visitors using cookies. These small packets of data are attached to your browser when you visit a site and identify you to that site, allowing it to make you personalised offers, call up your shopping list and so on.

It also lets the site measure how many different visitors it’s had in a certain period.

The picture is further complicated by the fact that people have started to delete cookies from their machines, usually for reasons of personal privacy: they don’t want the site owners to know exactly what they’re up to. And if someone deletes the cookie placed on their machine by your site, the next time they show up there, they’ll look like a different person, thus skewing your figures.

None of this is to say that one approach is necessarily better than the other. But the result is that unless you know how a site is measuring the figures it’s reporting, the discrepancies between the two methods can cause problems.

It’s also an issue of industry maturity. There are, of course, organisations that audit website traffic, most notably ABC Electronic (ABCe).

But unlike offline media, where almost every publication has an ABC certificate, the reach of the ABCe is limited.

In the days of the dotcom boom, few people felt the need of an audit. Since then, the vast majority of online media spend has been concentrated across a handful of properties, so there has been little incentive for smaller sites to be audited.

Now that the days of big online budgets are back, accompanied by a client interest in return on investment unknown at the turn of the century, it’ll be interesting to see whether there is increased pressure for sites to be audited.

If you were feeling cynical, you might also wonder if traditional media’s criticism of the failings of online measurement techniques carries more than a whiff of fear. The dotcom era jibe about no one knowing the click-through rate of a poster must have sent a chill through many an outdoor specialist, with the result that any of online’s failings in measurement have been seized upon in certain, offline, quarters.

So while The Guardian may be spluttering, the rest of the digital media industry could well end up being grateful to the Telegraph.

Anything that raises the debate about how websites measure their audiences is to be welcomed, since any improvement in understanding can only be of benefit to the professionalism of the sector and, ultimately, to clients.

• Michael Nutley is editor-in-chief of New Media Age

Latest from Marketing Week


Access Marketing Week’s wealth of insight, analysis and opinion that will help you do your job better.

Register and receive the best content from the only UK title 100% dedicated to serving marketers' needs.

We’ll ask you just a few questions about what you do and where you work. The more we know about our visitors, the better and more relevant content we can provide for them. And, yes, knowing our audience better helps us find commercial partners too. Don't worry, we won't share your information with other parties, unless you give us permission to do so.

Register now


Our award winning editorial team (PPA Digital Brand of the Year) ask the big questions about the biggest issues on everything from strategy through to execution to help you navigate the fast moving modern marketing landscape.


From the opportunities and challenges of emerging technology to the need for greater effectiveness, from the challenge of measurement to building a marketing team fit for the future, we are your guide.


Information, inspiration and advice from the marketing world and beyond that will help you develop as a marketer and as a leader.

Having problems?

Contact us on +44 (0)20 7292 3703 or email

If you are looking for our Jobs site, please click here