Tracking: IP and cookies crumble

A new study has raised serious questions about the way online businesses are modelled, suggesting that the management information being passed around many UK e-businesses is “fundamentally flawed”.

Web measurement company RedEye conducted the study in response to client demands for more robust information about customer loyalty and the growing need for accurate customer profiles.

The study examined two of the UK’s busiest e-commerce websites, williamhill.co.uk and asda.com, over a period of 28 days. More than half of all page requests on the websites are made by existing customers who are logged in, providing a robust sample of known data against which to compare the two main approaches to Web measurement: IP-based tracking – which monitors the address of the computer logging on to the site – and cookie-based tracking.

The study found that the most commonly used tracking methodology, IP-based server logs, inflated visitor numbers by up to 660 per cent over the 28-day period. This would lead to a company underestimating its conversion ratio by 7.6 times. IP-based analysis proved no better at identifying the number of distinct visits to the site, over-reporting these by 260 per cent and only managing to accurately follow 14 per cent of visits from start to finish (that is, all the pages visited in the correct order).

The study went on to show that, while cookie-based tracking was perfectly adequate over a short period of time, the number of people who periodically delete cookies or use more than one computer leads to significant inaccuracies over a prolonged period. Cookie-based analysis led to a 128 per cent overstatement in the total visitors figure, with only 50 per cent of visitors tracked uniquely over a 28-day period.

The results will be particularly alarming for companies that rely on IP-based software to analyse the effectiveness of their online activities. The inability of IP-based software to track visitors for even 30 minutes (the length of a visit) means that the only figure these systems can provide reliably is the total number of times a page was accessed.

“These findings blow the lid off what many people have suspected for a long time – that IP-based Web metrics are totally useless and any company claiming to provide accurate marketing and business data on this basis simply has no credibility,” says Paul Cook, chief executive of RedEye. “It’s crazy that so many businesses still make multi-million pound decisions based on bargain-basement management information. Hopefully now that we’ve quantified the problem, companies will change their approach to measurement. Inaccurate measurement only leads to mismanagement.”

RedEye’s clients have greeted the company’s efforts to provide greater accuracy with enthusiasm. William Hill group marketing director Peter Nolan says: “We’ve been aware of these issues for some time and are pleased that RedEye is providing a solution. Having the most accurate customer data possible will enable us to provide a more personalised service to our customers.”

Web analytics expert and author Jim Sterne comments: “Web analytics reports have always been somewhat erratic. Different measurement methods show more visits or lower conversion rates than others. But now that we have some hard numbers, we are closer to knowing how far to bend the lens to bring the picture into focus, and how to weight the data appropriately to get a clearer idea of what non-logged-in visitors are up to.”

The research also suggests that the accuracy of cookie data diminishes significantly over time. People who delete cookies can appear to be new customers, potentially leading to an underestimation of customer loyalty and conversion ratios. A RedEye-commissioned NOP study of 1,000 British internet users (people who spend at least one hour online in an average week, representing 80 per cent of the UK internet base) offers further explanation for these results. Fifty per cent of respondents said they had used more than one computer in the past three months, while 89 per cent deleted cookies periodically, if they knew what cookies were and how to delete them.

“Cookies provide a much more accurate picture than IP addresses, and are actually quite effective for tracking individuals over a 24-hour period,” adds Cook. “However, analysis of customer data over a longer periods should be done with caution. Most of the figures marketers need can be obtained by weighting figures appropriately. Only correctly weighted cookie data will deliver good quality management information on which to base business decisions.”

Cook concludes: “Accurate measurement standards are vital as the internet grows as a business channel. Our study offers a clear basis for the decisions companies can and cannot take, based on the different approaches to preparing Web data. Most senior marketers do not even know on what basis their management information is prepared, so I hope this study will help them to demand data that is worthy of their decisions. The difference between the level of inaccuracy found in the two sets of results varied greatly and we’re now appealing to other large e-commerce websites to let us capture and use their data for a comprehensive study.”