Information overdrive

Information is the lifeblood of the modern organisation. Critical decisions depend on data from business intelligence systems about customers’ buying habits, product sales or the effectiveness of marketing campaigns.

The quality of that data is all too often taken for granted, yet according to research by analyst firm Gartner in 2004, 25 per cent of corporate data is inaccurate. Imagine what that does to the organisation’s ability to make good decisions. As the old saying goes: ‘Garbage in, garbage out’.

Ian Charlesworth, a senior analyst with Ovum, says it is only recently that organisations have begun to take data quality seriously. ‘Three or four years ago, everyone saw data quality as a cost to the business and very much the responsibility of IT to put it right. The tools and techniques that people would use were very much down to the capability of the individual developer and administrator,’ he says.

Now, while ownership is still with IT rather than the business, at least firms are beginning to invest in data cleansing tools to address the problem.

One reason for the change, he argues, is that data quality vendors are advertising their services more heavily. Another is that, as businesses seek to rationalise and standardise their business intelligence systems, data quality issues become more visible. When data is combined as the result of a merger or acquisition, the subject of data quality becomes unavoidable.

Compliance is another driver, with the Sarbanes-Oxley legislation in particular forcing businesses to find ways of keeping the quality of their financial data. For the public sector, the drive to share data between agencies such as the police and social services has highlighted the importance of having good quality data to share.

But why is so much data inaccurate? ‘The main factor in causing the quality of data to degrade is people,’ says Ted Friedman, vice president of research at Gartner.

‘There is a cultural issue in that people in the business do not understand the knock-on impact of what they do when they are not maintaining high-quality data.’

In a customer relationship management (CRM) system, for example, it is easy for data to be entered incorrectly at source, with names misspelled or date of birth fields left blank. Even if accurate, it can degrade very quickly, as people move house or change their names after marriage. Most systems hold a few duplicate records, and data held on a single customer can also vary between, for example, the CRM and the billing system.

‘Data quality can be an indication of how integrated a company’s data processes are,’ says Umesh Hari, global data management and architecture lead in Accenture Information Management Services.

Friedman sums the main problems caused by poor quality data as ‘productivity loss, customer churn, opportunity cost, lost revenue opportunities and additional or unnecessary waste’.

And it is not just a problem with customer information, as Charlesworth points out: ‘Order entry errors have huge downstream cost implications such as customers being sent the wrong goods because of the order entry.’

It is almost impossible to put a figure on the cost resulting from not keeping data in good order, but a 2002 study by the Data Warehousing Institute estimated the annual loss to US businesses at $611bn (£316bn) a year.

The not-for-profit advisory service Business Link for London (BLL) experienced the full impact of poor data quality when it was formed in 2001, as the result of a merger of nine small individual Business Links. Nine customer databases were amalgamated, says Mike Pratt, now the organisation’s data integrity manager.

‘There were no geographical boundaries on which Business Link could contact which firm in which area, so there was a huge amount of crossover. And all they did was just bring all the records together, give them a unique ID and dump them into a database,’ he says.

By the time Pratt joined the organisation in 2003 to oversee business intelligence, there were already problems. Customer satisfaction was low because people were receiving four or five copies of the same letter. BLL employees paying visits to customers found they were turning up at the wrong address.

Continued on next page

Like this story? Spread the news by clicking below:

 del.icio.us     Digg this     reddit!