Gotcha! Pitfalls in Personal ProfilesBy David F. Carr | Posted 2005-06-10 Email Print
How Real-World Numbers Make the Case for SSDs in the Data Center
How to get it right, the first time.
Because it maintains records such as credit and criminal histories of individuals, ChoicePoint's business rests on maintaining a high standard of accuracy. But so do the businesses of any organization that maintains profiles of employees, customers and business partners. Here are problems to watch.
Problem: Sloppy data entry.
Resolution: Double-check. "You've just got to get it right the first timeit doesn't get better than that," says Tom Redman, president of Navesink Consulting Group and co-founder of the International Association for Information and Data Quality.
You can purchase or create code that will prompt data entry clerks to double-check an address or the spelling of a name, based on a comparison with some reference data source. But such tools can also introduce errors, Redman says. Workers can get in the habit of blindly accepting the software's suggestionsand the reference data can be wrong. Redman's answer is more managerial than technical: Appoint someone to be responsible for data accuracy. Aggressively look for errors and then seek out root causes. If one particular data field is wrong in multiple records, for example, that might mean data entry personnel are misunderstanding what should be recorded.
Problem: You don't know whether to trust your data.
Resolution: Test its accuracy. Develop a benchmark for measurement of the accuracy of your data, such as a percentage of error-free records, and monitor it religiously.
For an organization like ChoicePoint, this might mean thoroughly spot-checking records from a randomly chosen subset of consumer profiles and recording what percentage of records contain erroneous address, employment, insurance-claim or criminal-history data. Addresses can be checked fairly reliably against postal databases.
Problem: Combining data on one person is difficult.
Resolution: Map out where the right data is. Find all the places in your enterprise that contain different records referring to the same individual. Establish guidelines on where to get which type of information. Then establish procedures to ensure that a piece of data, such as an address, is consistently recorded every time it appears in a database. If names and addresses about one customer exist in 10 databases, and the customer calls to update his address, you need to make sure that change is reflected in all 10 databases. One way to enforce consistency is by creating a master data file, where a central database pulls information from various data stores and creates a master record for each individual.
Problem: Data may be consistent but still wrong.
Resolution: Understand the limits of your software. Too often, organizations implement a technical fix in isolation, Redman says. What's more important is a process for identifying errors, determining their cause and measuring your efforts to eliminate them.
Jim Stagnitto, a data quality expert with consulting company Llumino, warns against underestimating the challenge of reconciling data from many sources. The data cleansing tools used to create a master database work by assigning a probability to whether two records refer to the same individual. But by nature, this is computerized guesswork, according to Stagnitto: It's important, he says, to have steps for your staff to follow for unraveling errors that might be introduced in the process.