A matter of show, not tell

I use more words than numbers – lots of words.” So said Gary Lorden, professor of mathematics, California Institute of Technology when he was profiled by Data Strategy in March 2007. His descriptive skills are clearly very good as Lorden went on to become a mathematics consultant on Data Strategy’s favourite TV show, Numb3rs.

Those skills are not widely shared by other mathematicians and statisticians, however. An ability to make a model or algorithm come alive for a lay audience is not essential to gain a degree or PhD in those subjects. Yet any of those graduates who go on to careers in data analysis will at some point find themselves in front of an expectant audience which does not share their love of detail.

Rachel Morgan, planning director at Planning-Inc, faces this situation regularly. “We do a lot of analysis on broadband usage. Our client doesn’t need to see that detail – they need to know what customer types to target. They want to know something they will get excited by,” she says.

The raw results from mining huge volumes of data about customers and their product usage could run to many pages on a spreadsheet. In an academic environment, showing how you have arrived at a conclusion is critical, as is showing any mitigating factors, such as error rates.

In the real world where clients are paying for something they can work with, however, what matters is the out-take. Provided a client has confidence in the analyst, there is no need to discuss confidence levels in the data.

“Our planners or account managers will sit down with the analysts to decide what they are going to present,” says Morgan. “They go through the Excel document to understand what has been done and then agree how they are going to tell the story.”

A similar step is critical at the start of an analysis project, she says. The brief needs to be interpreted into a technical specification and the analyst also needs to understand the business context as much as the data content. “It is easier for an analyst to do a piece of work knowing what the client wants to get out of it,” she says.

This is the creative aspect of data analytics which is often overlooked, yet lies at the very heart of the proposition of external analysis agencies. Getting to the point of using skilled consultants does require a level of maturity which many organisations have yet to reach.

IBM recently identified in research among chief information officers that 86 per cent of CIOs in mid-market companies have identified business intelligence and analytics as the keys to optimising decision making in their organisations. By introducing analytical tools, these companies can move towards a “predict and act” culture, rather than the “sense and respond” capability which standard reporting tools tend to support.

An example of this change through an upgrade in resources can be found at Gamesys Group. A fast-growing e-gaming business which handles more than £1 billion in cash bets each year, it was relying on a spreadsheet-based solution for its reporting and analytics.

To sustain business growth, support more robust information management and improve fraud prevention, it recently introduced a solution from IBM. Tom Hillary, CTO at Gamesys Group, says/ “To protect our business, we need quicker access to information we can trust and we want to be able to analyse that information more deeply. As the online gaming market becomes more competitive, we need to work harder at retaining our existing customers and that involves profiling them at a more granular level.”

Deploying an in-house application is the first step towards greater visibility of those critical patterns in the data and their use in deciding where to take the business next. Having better and more flexible access to customer data starts to build an analytical culture.

One of the great breakthroughs of the last decade was the way in which complex statistical modeling techniques were packaged up into user-friendly data mining tools that even lay people could use. Instead of having to rely on a trained data analyst, frontline staff have been enabled by insight-generating applications.

Mark Robinson, managing director of Marketing Databasics, recalls when his company started working with statistical software vendor KXEN. “It was a funny experience because one of our sales team started to try an build a model, came back and said he’d done it, come and look. Our statisticians went in and found he had built a reasonable model.”

The one action which he had taken in building that model was to include the customer ID number. “He said it was a good proxy for tenure,” recalls Robinson. Depending on how that model might get used, this could prove to be a bad decision, even if the model itself is robust.

Such issues arise once analytical teams start their “deep dive” into the data. “You sometimes find, because of the nature of the work, that you are starting with a simple question. As you get into the data, you start to find layers of complexity about what that question means,” he says.

This is the point at which analysis can start to go off track or become academic, rather than commercially useful. Steering the teams towards concentrating on something the client can use is critical, as is rolling that deep analysis back up into an answer to the original simple question.

In-house analysts can sometimes struggle to bridge between the technical dimensions of their work and the business requirements of the departments they work for because they lack objectivity. A company that is being baffled by data findings is no more intelligent than one which does not have any insights at all.

Combining the ability to carry out the technical aspects of data mining and convert them into strategies, “goes to the very heart of our business,” says Matt Hutchison, general manager of Cogent Analytics. “It is the thing that companies such as mine grapple with the most.”

He recognizes that there are fundamentally different skills sets at work in this process. “It is very difficult to find analysts who are good at communicating visually complex ideas. They use different parts of their minds. There are some out there, but they are rare – if you have got one, hang on to them.”

It is the planner who tends to fit into this role, comfortable with the mathematical complexities but also familiar with the business context. Gaining that combination of skills only comes about in relatively limited circumstances – often through working in an analysis agency – which is why there are few of them on the market.

While the debrief to the client is a vital stage when the insights get illuminated, the greatest sin of analytics is to surprise the client. Like barristers never asking a question to which they do not know the answer, the client should have a good idea what is coming.

Hutchison says his agency’s process is designed to ensure this is the case. “During the analysis we try to include the client. Analysis is an iterative process, so we build in way stations where we stop and look at where we are,” he says. One of the most important checks is whether the insights being developed will have any commercial value or are simply interesting patterns that can not be influenced or changed.

“The danger that I have seen happen many times is that the analyst goes off on a route that will not achieve any results and is not applicable in the business,” he warns. “The cardinal sin is to produce something that ends up meeting with the response, so what?”

Gareth Mitchell Jones, director of analytics at Experian Integrated Marketing, says the way to avoid such an outcome is through having the right perspective. “We constantly try to see things through our clients’ eyes. It is a constant challenge every analytics team faces,” he says.

EIM uses a six sigma framework which puts a planner’s abilities to translate at both the front and back ends of the process. That means ensuring the questions being asked of the data are likely to produce a useful outcome and are also capable of being run through the data sets and technology in play. Trying to find an insight from the wrong database or which is based on a poorly-populated variable is as bad as generating an insight that can not be used.

The client’s questions are broken down into a series of significant queries which can be run through the analytics procedures before the outputs get aggregated back up. Along the way there are important sense checks. “We look to ensure the findings are significant. For example, that it is not just a group of 15 people to target, but something that will affect hundreds of thousands of people and add millions to the bottom line,” says Jones.

An example of this might be to crunch the data on mobile phone owners who have downloaded ringtones. Some of them call the provider afterwards and demand a refund because the ringtone is claimed not to have worked. Jones says that a project he worked on identified a group of people who constantly did this because internal processes did not recognise this behaviour. “That single insight gave a £60 million benefit per year to the company,” he says.

To get such an impactful result out of analytics often requires starting with a much longer set of possibilities. Jones says that his company will ask for a list of 100 things the client believes it knows about its customers that analysis can prove to be either right or wrong. It then works through every single one of those assumptions until one is discovered that would make a substantial difference if changed.

“There are all sorts of myths about what is happening in the business. When you look at the data, they may or may not be true,” he says. Presenting back such findings is no longer about how creatively the data can be expressed, but about how to handle the politics of what might end up getting changed. Those are skills which do not make up part of the maths curriculum – they only get learned the hard way.