Facebook needs to do more than be open and honest about metrics errors

The social network seems to think that finding and admitting its errors will pacify marketers but they will not be happy until they get full, independent, third-party verification.


Another month, another Facebook measurement error. Its latest, revealed in a blog post yesterday, is the fifth admission since September and the 10th error it has found in its metrics.

This time, the “bug” was found to be misattributing clicks on video carousel ads. What this means in practice is that if a user clicked on a video to watch it, Facebook thought they were clicking on a link to the brand’s website and charged the advertiser accordingly.

Facebook was quick in its blog post to highlight how small an error this was. Because it related to mobile web users, rather than smartphones, and specifically video carousel ads it only accounted for 0.04% of ad impressions. It apologised, said it takes all bugs very seriously, and promised to pay impacted advertisers back.

Facebook launched the blog to provide updates on these errors in November. It is part of a wider plan to make the social network appear open and honest and willing to own up to errors. After all, what marketer could lose trust in a Facebook that admits its mistakes in public and looks to atone for its sins.

Along with the announcement of a measurement council and several new measurement partners, Facebook clearly thinks it is making big progress on allaying marketers’ concerns. It goes on in its blog post to talk about how it “regularly reviews its systems to ensure their accuracy”.

Clearly these reviews are not happening often enough; this latest bug was seemingly running for more than a year. And while Facebook found this one itself (along with most of the others), some have been discovered by the media. It was the Wall Street Journal that uncovered the first measurement error back in September, while Marketing Land happened on another in November. Marketing Land has a handy list of all the errors here.

This latest error is a biggie because it is the first one to have actually cost advertisers money. It might only account for 0.04% of ad impressions, but it will leave many wondering what other errors are out there that Facebook hasn’t found yet.

The errors are also significant because of the huge role Facebook now plays in the ad world. Along with Google it accounts for the vast majority of digital ad spend and almost all the growth. Advertisers spend vast sums on the site and are entitled to know what impact this is having.

Given that almost all the errors are in the social network’s favour, its clear Facebook has been painting itself in a much more favourable light than the data actually shows across a huge range of measurements for quite some time.

Admitting there is a problem, as Facebook has done with the blog, is a  step in the right direction but it is only a step. The social network is still a walled garden with third parties given very little access to its data. The walls might be slightly lower than they used to be, but it is still almost impossible to compare its data with other ecosystems.

This would not be tolerated in any other media. The TV networks have Nielsen in the US and BARB in the UK to ensure all data is correct and that it can be compared across networks. So do radio broadcasters, magazine and newspaper publishers.

Being open and transparent about metrics means letting a company in to verify those measurements, not just admitting when something goes wrong. No media owner is perfect, not Facebook, not Google, not the TV networks. Facebook is not deliberately trying to mislead advertisers but mistakes happen. That is why independent, third-party verification is needed. Only then will marketers truly know that the metrics they are using are reliable and that they’re spending their money as efficiently and effectively as possible.