Big data is just a crutch for unimaginative marketers to lean on

With many millions more sensors, higher-resolution satellite imagery, and much faster computers, we now know much more about hurricanes than ever before. Between the size of this data, the speed it comes to us and the power of processing, we typically see storms a week earlier than just 10 years ago.

Yet due to the vast complexity of weather interactions and their near-perfect chaos, we’ve made little progress on predicting the path. So we now have a week longer to be anxious and get more people concerned, but we can’t do much with it.

It’s time for marketers to rise up and take control of product

There is no better metaphor for the current reality of big data in marketing. We want as much data as possible, but we’re rarely better able to make better decisions. We just have more complexity. The data in the world is like a torch in a vast, pitch-black chasm – a more powerful beam of light that shows just how far the darkness goes. Perhaps it’s knowing this futility that means until now we’ve celebrated the process and the potential of data, not the meaning and the decisions we make as a result.

For one, big data is a myth. In the 2000s it was called data, but one day apparently it got big. Now we have more of it and in more places, and the assumption is this can only be good.

A quality issue

The first problem seems to be the quality and the processes used in data storage. We like to celebrate how much we have, but data normally comes from separate systems, measured in different ways, stored in different places.

Most CTOs will embarrassingly admit they have too much data these days, not too little. They find the data dirty, contradictory and in systems that won’t mesh.

If companies know that much about everything, why are we sent credit card offers for credit cards we already have? Why is my bank offering me cheap loans and pitching me funds to invest in at the same time?

Using data in wrong ways

We also typically tend to use data wrongly. We ascribe anything with a high level of accuracy, a high level of importance. Things we can measure and alter most quickly are considered especially vital.

We now often override excellent, gut-based vision with poor-quality and erroneous data or modelling.

Social media listening on Twitter shows us clearly how a tiny number of very unrepresentative people behave in an unusual manner. To drive the metrics that don’t matter but that we can change, every brand asks inane questions to become engaging, or becomes a baby picture publisher because the only metrics they measure, which happen to be entirely pointless, shoot up.

We are using numbers blindly and not empathetically. We see data that toaster buyers often look at other toasters, so websites spurt out endless toasters to buy in addition, rather than showing toast racks or egg slicers or sign-up offers for Blue Apron.

A culture of spinelessness

It strikes me that data has primarily become a tool for taking blame that has no feelings to hurt. Spreadsheets have evolved from support functions to drivers of a culture of accountability, and led to an over-reliance on data.

It means imagination has become neutered, risks are unacceptable, creativity is against the orthodoxy of the linear and logical. Human judgement is now considered a dark art, unproven. We now often override excellent, gut-based vision with poor-quality and erroneous data or modelling, only on the basis that the process looks more advanced.

What the spreadsheet has done is to create in organisations and governments an over-reliance on numbers, by no means always meaningful or even accurate, with the result that target metrics or values invariably override any conflicting human judgement. Power in management now resides with those who own metrics, and we become a rudderless ‘arithmocracy’, fixated only on spreadsheets and never possibilities, customer needs or any coherent vision or future-facing strategy.

Celebrating the complex but useful

Thanks to geographical data we can fairly accurately find people who have walked into a McDonald’s and serve them ads for Burger King at some point later. We can use local weather data and some basic AI to establish when it’s going to be cold in Dayton, Ohio, buy media moments in that zip code at the time it first gets cold and serve them an ad for hot soup. The only thing we’ve not done is think why.

A cynic’s thoughts on why blockchain could change the world

Is the Burger King consideration set some sort of weird niche? Is a lovely type of soup something we should only be told about on a cold day, and online? What’s the matter with broader targeting, using places where creative could be bolder and being patient enough to build brands?

Data doesn’t get people

Big data has the temptation of making the complex seem simple. It may think I like certain movies, when in fact I like certain movies at certain times or in certain situations. The crap I watch after a beer on a plane ruins any algorithm’s chances, let alone an Airbnb guest using my TV.

We endlessly refer to “data-driven” insights when I’ve only ever seen facts, not insights, from data. Insights are found by observation and, where needed, supported by data. No spreadsheet ever revealed anything as beautiful and transformational as an insight.

Data reflects agendas

Contrary to beliefs, data does lie. It lies to our face and unashamedly. Any stock chart cropped in any manner can tell any story. The battle between employment and unemployment numbers reflects the criteria and agenda behind it. Everyone in our industry needs to be trained to be critical of all data and to understand the underlying politics.

Blending the art and the science is key

We think that science is the solution to all. We have modelling that, in theory, shows the ROI for any event of inputs, yet never includes enough variables. What maths would have ever predicted the viral hit of Dollar Shave Club, or that the curious taste of Red Bull would sell by the billion? We have an array of attribution companies and models far more determined to showcase and capture their proportion of success than create it.

Most, and the greatest, leaps in business happen from the remarkable, not the iterative… The shifts come from something data can’t see because it hasn’t happened yet.

If I’m served a mobile ad for a car and go on to buy one, rather than seeing it as the result of billions of brand experiences, a few companies will assume it’s their sale to celebrate and that it’s just ratified that their targeting was right. Causation is way more complex than a spreadsheet affords.

The world of bigger, faster, cleaner, more intimate data will be a world of incredible opportunity. The notion of targeting people based on context – of building sequential stories over time; of upselling, cross selling – will be life-changing. For the moment we need to celebrate those who ignore data, those who challenge it, those who use data to back up insights from their gut, not those who use a spreadsheet to find one.

In today’s world, it’s very hard to be fired for making what turns out to be a terrible but rational decision. Any choice, supported by enough data, is defensible. Yet if you choose the opposite, even a moderately poor decision makes people feel vulnerable.

Yet most, and the greatest, leaps in business happen from the remarkable, not the iterative. Small, predictable but unremarkable success comes from data, trend lines and linear uncreative projection. The shifts come from something data can’t see because it hasn’t happened yet. They come from accepting there is no ROI that can be calculated – and no case studies to provide reassurance – for something that’s never been done before.

Let’s use data for illumination. Let’s study the numbers, invest in the incredible possibilities of AI to transform our roles. Let’s embrace a culture of ROI, of accountability. But let’s also accept our gut, ideas, risks and leaps of faith are what have really transformed the world.