Setting the flight path for data migration

Data migrations are usually greeted with dread by business managers. IT-driven projects to consolidate servers, move to a new application suite or platform, or just to upgrade systems are often the trigger for a data integration exercise as well. Months down the line when the business gets handed the new live solution, they often discover problems.

As a result, migrations and integrations can fail because they do not get adoption by the business. Either data has been modelled in a way that does not readily support their process, or business-critical data has been lost in the process of matching and deduplication.

But if these flaws are the fault of IT, perhaps business should bear some of the blame. After all, by sitting out of the project, it has not brought its needs and knowledge to bear on the specification. That changes the critical success factors and also reduces the opportunity to make process improvements.

Where these have been a driver, rather than a hidden factor, the benefits can be significant. Bupa has just completed the largest data migration project in the UK insurance industry which now enables it to implement its One Plan initiative. Lasting three years, the project involved migrating data on 12.7 million members, nearly 100,000 groups and 21 million claims.

With a unified view of this data, Bupa is now able to bring new products to market more quickly, allowing it to expand its product range and reduce premiums. Staff productivity has been increased, customer experience enhanced and obligations under Financial Services Authority requirements for “treating customers fairly” have been met.

“We were very aware that consolidation of so many systems was a risky strategy and that the data migration element presented significant challenges and pitfalls”, says Steve Felton, director of business change at Bupa.

“For the new systems to be implemented successfully, it was imperative that all the relevant customer and financial information was transferred to the new applications accurately. With the migration of data from flat file, mainframe technology to the multi-level hierarchies of relational database systems together with the multiple mappings required to transform data from source to target systems across different lines of business, this was an incredibly complex project,” he notes.

You have to have a business driver that gets the board to approve.

Indeed, the insurer had to gain both FSA and HMRC approval. That meant the company had to account for data being transferred accurately and in its entirety, as well as providing visibility for auditing early on. Although old customer data could have been archived, the business wanted to clean and extract all the data in its legacy systems in order to analyse its business KPIs.

Kognitio worked with the client on delivering the data migration into the new Swift policy administration system and Peoplesoft financial accounting system out of the nine previous solutions. In the process, the number of individual records was reduced by two-thirds through accurate matching and deduplication. That involved 700 million rows of data which were ultimately translated into 2.5 billion rows being loaded in less than two days.

“Involving the lines of business is definitely important to the success of data migration. It is their data – it belongs to them,” says Phillip Magnall, head of data migration at Kognitio. “They understand the data and need to input into what the final solution is.”

“In the Bupa project, there was a great deal of discussion about how to match and dedupe customer records and the setting up of groups for healthcare policies in the new system,” he says. Involving business teams was something Kognitio asked for as soon as it was appointed to
the delivery contract.

Says Magnall: “As the project progressed, there was more business involvement than at the start. I have seen data migration projects where the whole thing is driven from an IT perspective and business is not involved at all. What Bupa did is the only sensible way to approach this type of project.”

Practitioners with similar experience of major data migration projects agree. At KPMG, advisory partner Steve Gallagher says: “I have been involved in both types, unfortunately. If you look at projects that fail, it is always where it has been left to IT to sort out the data integration. Business sees it as a technology issue and believes IT can solve it.”

This is a basic mistake since the business owns customer or product data and will need to leverage it in everyday processes. What data integration should provide is a significantly enhanced view of these critical assets, leading to performance improvements. Without taking the view of business into account, IT can only reach technical conclusions about what data is important and how it should be matched.

“There is an element of naïvety which believes you can leave it to the software to resolve,” says Gallagher. “If business doesn’t take responsibility for the data, it can’t expect anybody else to work out the meaning of fields and how variables should be matched.”

The most obvious example where the business has to engage with a data migration in order to get maximum value is during a merger or acquisition. Only by ensuring data is integrated intelligently and in line with strategic business objectives will a correct picture of the true asset value really emerge.

But a company does not have to be in a takeover scenario in order to extract value from data integration. “You only do it to get better insight into the business to understand what the performance issues are. It may be necessary as part of developing business-critical systems, like MIS or BI or CRM, that will help different parts of the organisation to become more productive,” he says.

Top-down sponsorship of a data migration and integration project is often what makes the difference to whether the lines of business get involved or not – and to whether the project succeeds or fails. Gaining buy-in from senior management is one of the first things KPMG will ensure as part of its methodology. From there, ownership of specific items of data can be assigned, which naturally leads the business to want to ensure they get the right outcome.

Gallagher says that running diagnostic software across existing data management systems is also critical. “It gives you an indication of what the level of quality is and what you want to get it up to. For example, if your data is currently 80 per cent accurate, do you want to make it 95
per cent? It helps you work out what you are trying to do,” he says.

Introducing data quality measures into a data migration project also allows more strategic business KPIs to be run against it. A purely IT-driven project may only be able to show that it has put data in place where the business needs it, not whether that data is fit for purpose and has helped the process to be carried out better. By applying a business perspective, data integration can start to deliver true return on investment.
One reason why business often resists getting involved with data integrations is the seeming technicalities involved. Technical specifications of systems can be off putting for non-technical people. Yet these should only be the outcome of the set of goals which the business has outlined for data availability and purpose.

“IT’s role is around making sure data is available and delivering it to the right place at the right time for the business to use,” says John Poulter, senior vice-president EMEA at Informatica. “When trying to sell data integration as a benefit, you have got to have a business driver that gets the board to approve the expenditure. New chief information officers, even our own, talk about the need to align IT and the business.”

This has been a big shift in thinking for IT departments and reflects the change from focusing on the technology to understanding what the information it contains is actually used for. That requires a dialogue between the two sides, which neither has been historically good at.

Informatica has recently been involved in a major data integration project for Virgin Media which involved bringing together the eight million
customer records that resulted from its acquisition of NTL and Telewest. The net present value of operational savings which the single view was able to provide to the business equalled £1 billion plus.

“If you went to any board and gave them a business case like that, who in their right minds wouldn’t want to save that amount of money? Previously, the business had multiple views of its customers. We helped them bring 24 systems down to one with improved data quality and better ability to cross sell Virgin Media’s tri-band offering,” says Poulter.

According to Stephen Tidswell, head of IT applications support at Virgin Media, the customer data hub is helping with customer service, even though the company has yet to converge service onto a unified CRM solution. “More often than you might expect, customers contact us on multiple occasions during a single day. Not having those details available would cause problems. If a change in service was made or a payment received or a fault fixed, we can now see that,” he says.

Business benefits are already emerging. Says Tidswell/ “We are currently fixing the fundamentals. A key component of that was customer retention and reducing churn. If, as an agent, what you have to think about is running 24 different systems, that is what you will be doing rather than focusing your energy on the customer.”

Although the final data integration often takes place across a weekend, in order to avoid disruption to the business, projects are usually long-term. That can be a cause for business managers to shy away from getting involved, since they do not want to be committed for long periods of time to potentially open-ended working parties.

Having a clear plan and the involvement of all stakeholders does make a big difference to the outcome, however. “There is no one problem you should expect,” says Jake Freivald, vice-president corporate marketing at Information Builders. “But one common issue is that data doesn’t travel well.”

In moving information out of a legacy system into a new target application, there are often changes being made to definitions and data structures. It is these which can lead to the business claiming the data has been harmed or is no longer what they need, especially if they did not provide their definitions and requirements early on.

“If business wants to make a small change, IT can find that hard to do. It might even lead to a revised specification. That is a constant problem you see across IT, especially in application development. That can lead to dissatisfaction among business and even customers,” he says.

Fundamental changes to the data model are often impossible once a development system has gone live. That is why it is vital early on to have a business view. “The data model in itself is a complete thing and is intrinsic to the software. It depends on how much you want to change – there are tools that can help with changes to mature models,” says Freivald.

A key point where business benefits have started to converge with IT goals is around master data management. Data migrations and integration increasingly go through a MDM step in order to align different views of the business. “What’s different in MDM compared to early forms of data integration is the desire to reconcile things across many systems. Early data warehouse projects reconciled everything into a standard model,” he says.

This is where some of the flexibility which business lines might want is able to be provided. Instead of being forced into a new “single version of the truth” within a data warehouse, MDM can allow local data definitions to remain, while still being rolled up into common classifications in the new integrated data set.

Little wonder, then, that vendors of MDM and data quality tools have started to get very involved in migration projects. Even so, they are having to adjust to the new circumstances in which lines of business have an important say over what is done.

“Software companies have taken the tack of trying to sell into executive level, like the CIO or VP of IT,” says Charlie Lawhorn, vice-president of business development at Stibo Systems. “If the business is not involved early enough to make decisions about the integration of data, they may reject those projects.”

He notes that tens – even hundreds – of millions of pounds have been spent by large organisations on ten-year ERP projects which fail to get adoption among their target users. “It is up to technical services to help to define the business benefits of the software and how to integrate it into making decisions that the business gets something positive from,” says Lawhorn.

“It happens a lot that you don’t see the business case fully worked through. The problem is that a lot of vendors are pushing for revenue booking and shorting the sales cycle,” he warns.

With data staying high on the agenda for business investment, pressure to run data migrations more quickly has grown. But the risk remains the same – unless the end users in the business have input and feel ownership of the resulting data, failure is always a possibility.

When you look at flocks of migrating birds, it is hard to work out how so many individual birds are able to work as a group to reach their goal.
Closer inspection shows that each bird is actually triangulating its flight pattern against seven others nearby. As a model for how data integration should be carried out, that is pretty useful.