Number crunching is no way to assess true value

The IT industry is finding that proving return on investment is problematic. That is because ROI ultimately misses the point, says Alan Mitchell

Doesn’t it gladden the heart? The trillion-dollar information technology industry is having to admit it’s got a problem that marketers have been struggling with for decades: it cannot prove return of investment (ROI).

Faced with increased corporate scepticism at the value of huge IT projects, the likes of Accenture, BT, Cisco, Hewlett Packard, Intel, Microsoft and Xerox are getting together to form an Information Work Productivity Council. MIT research and case studies will, they hope, prove how IT investments drive productivity, and help companies decide which investments to prioritise and the levels of investment needed.

A key impetus for this initiative is the post-boom squeeze on IT spending and a stringent new “prove it pays” attitude emanating from the boardroom. In the midst of this increasingly tough environment, more large investments have fallen by the wayside. New McKinsey research shows that total shareholder returns of IT companies have fallen 64 per cent over the past two years, followed hot on the heels by telecommunications companies at 60 per cent, compared to financial services (up 19 per cent), consumer staples (up 21 per cent), and health care (up 29 per cent).

Clearly there is a cyclical factor at work, but is there something else too? No matter what specialism you look at – whether it’s IT, marketing, human resources, the legal department, even finance itself – they’re all struggling with this same ROI issue.

Is this just the economic climate speaking or is there a deeper issue at work here? Perhaps there is. Perhaps there is a problem with ROI itself.

The trouble with numbers is that they secretly generate their own agenda. Numbers assume separation, for instance. You cannot count things unless they come in the form of discrete, separate units. So when the thing you are dealing with has no clear boundaries, or is inextricably tied to a complex set of interactions and relationships, counting can’t capture its reality.

Numbers also assume similarity. A number is only meaningful if it represents X number of the same thing. Number relies on our ability to reduce what we are talking about to a common set of attributes. Yet such a focus on quantity is, at the same time, a decision not to focus on qualities that only come to life via the subjective experiences of people. Nobody has yet put a number to the taste of chocolate or the smell of a rose – but we value them nevertheless.

Here’s a thought. Perhaps the secret of ROI, whether for IT, marketing or human resources, is the quality of the decision-making and implementation. Does its management smell and taste of roses or chocolate. Or something less attractive?

Twenty years ago, quality guru W. Edwards Deming declared that “97 per cent of the circumstances that affect a company’s results are unmeasurable, while less than three per cent of what influences a company’s results are measurable. Nevertheless, managers tend to spend over 97 per cent of their time analysing measures and less than three per cent of their time on what really matters – the unmeasurable.”

Deming himself was a fanatical statistician. He was always looking for meaningful new measures. But many of his 14 points for business transformation, outlined in his book Out of the Crisis, relate to overthrowing the tyranny of the wrong numbers: “eliminate slogans, exhortations, and targets”; “eliminate quotas”; “eliminate management by numbers, numerical goals”.

That’s because he saw businesses as systems where the most important qualities could not be captured by these numbers, targets and quotas. “Every activity, every job is part of a process,” he wrote. “A flow diagram of any process will divide the work into stages. Any stage has a customer, the next stage.” And at each stage, there is a mutual interaction between the customer and the provider: an exchange based on “This is what I can do for you. Here is what you might do for me.”

Crucially, what Deming saw in these interactions was not the flow of material goods or money, but a flow of human qualities: trust, a sense of common purpose, a chance to learn, to save each other time, to eliminate unnecessary work, to eliminate fear, to take pride in a job well done. Whether or not the system worked well or badly depended on these intangibles – human qualities that can never be captured, measured or managed by numbers. And once this system is working there is no use in specifying a goal: “You will get whatever the system will deliver.”

In other words, whether it’s IT, human resources or marketing, successful implementation of projects depends on human intangibles such as use of information (meaning), learning, trust and motivation. And their value can only be properly assessed in the same multi-dimensional way: whether it accelerates learning, increases trust, improves motivation – as well as cutting costs or increasing revenues.

Fascinatingly, something very similar seems to happen with companies’ interactions with customers. As human beings we naturally tend to assess value in a multi-dimensional way, rather than via just one number such as “value for money”. In any interaction or experience, we ask ourselves whether it was hard work, and whether the people we were dealing with made life harder or easier. We think in terms of information exchange: did we gain interesting or useful information from the interaction; did we get a good return on the information we volunteered? We assess return on attention: was it worth paying attention to, or not? We think in terms of value for time spent: did it save us time or waste time? Was it a rewarding use of time? We also do an emotional calculation: was the experience stressful or pleasurable?

As human beings we seem to have an instinctive ability to crunch all these different dimensions of value into one simple, single experiential “profit and loss account”. Was it worth it or not? Companies do not have this capability yet. Instead, we are having to invent it.

Alan Mitchell,


    Leave a comment