by Garry Moroney on February 19, 2007 – 7:18 am
Managing the quality of customer data has its challenges: It is typically collected from a wide range of sources and channels and very often those responsible for entering or capturing data have no incentive to do so accurately. Even if they do much of it, including contact data can go out-of-date rapidly. Despite this most customer data has one major advantage over many other types of data which is agreed and accepted standards and reference data. While these standards do vary from country to country, they are at least universally understood and have an enormous impact on the approach and the effort required for managing data quality.
Because of these global standards and references, there is general agreement on what a complete, valid and correctly formatted address should look like – likewise person or business name, telephone number, date of birth, email address etc. So this means that if I am sharing my customer data with my business partners at least we have a common view of what high quality data should look like and the checks we need to make to assess the quality levels.
Another huge benefit is that third party service providers and technology vendors also understand the requirements and standards by which to measure and improve customer data and they know that these requirements are largely the same for all vendors. As a result large numbers of service bureaux and technology vendors are able to offer well developed, generic, out-of-the-box products and services to tackle customer data quality issues. These can deliver a lot of value with minimal or no customization and the effort to acquire and implement these solutions is small.
Read More »
by Garry Moroney on February 2, 2007 – 3:36 am
Determining the aggregated return on investment for a data quality management initiative is notoriously difficult. Typically a minimum or partial ROI can be estimated by reference to the impact of low quality data on one or two key projects or processes. For example in a CRM project data quality ROI can be tied to reductions in customer contact failures and increased sales due to high quality segmentation. But given that the same set of master data will be used more than once in most organizations (i.e. customer master data will also be used in the billing system, the supply chain system and so on) and will add value (or destroy value!) in all of these processes, basing your ROI calculations on a single system or process will always underestimate the true returns.
For an organization trying to estimate the total returns across the enterprise from a data quality initiative, there are two difficult questions that must be addressed:
• How valuable is this dataset to the enterprise – assuming 100% data quality?
• How does its value decrease as quality erodes?
While these questions might at first seem unanswerable, it is worth noting that these are not unusual questions for a business to ask. In fact businesses need to be able to answer questions of worth and depreciation for all their tangible assets – property, stock etc.
Unfortunately data is one of those intangible assets where normal valuation approaches like recorded cost or replacement value are ineffective. But there are other intangible assets such as IPR, work-in-progress, customer and partner relationships (good will) where significant research has been done to develop effective valuation methodologies. It just might be possible to leverage these methodologies to value your data. For example, the value of customer data is directly related to the value of the customers themselves and so “customer lifetime value” methodologies should be applicable in estimating the value of customer data and the extent to which this value varies with data quality.
Have any of you out there attempted to put a real value on your company data in this way? Perhaps you’d be willing to share your experiences with us.
For more information on building a business case for data quality and calculating potential return on investment see the Informatica white papers: Data Quality Profiling Calculating ROI for Data Migration and Data Integration Projects and The Data Quality Business Case—Projecting Return on Investment.
by Larry English on February 1, 2007 – 3:18 am
IQ in the News
Most people have probably heard about the highly reputable Consumer Reports’ recall of its flawed testing of infant car seat safety. The report, issued January 5, 2007, found that many car seats failed the high-speed side impact test it conducted (the government requires passing of frontal crashes of 30 mph. Consumer Reports tested at 35 mph for frontal crashes and 38 mph (so they thought) in side impact crashes. The findings seemed to indicate a high degree of failure with nine failing some or all of the crash tests, and only two doing well in all tests.
However, the government found a problem with the way the testing was conducted. Instead of a 38 mph side crash, the test simulated a side-impact crash of over 70 mph with very inconsistent results that would have come from 38 mph tests. Consumer Reports recalled the entire report January 18.
IQ Lessons Learned From the Consumer Reports Recall:
Negative impact on consumers and their confidence in the organization:
- The impacts of the faulty testing where dramatic and swift. The Executive Director of the Washington State Safety Restraint Coalition exclaimed that “Consumer Reports screwed up….They really upset people and created enormous confusion.”
- When designing tests, as you will with IQ assessments, you must assure you design the tests properly. Measuring validity and accuracy are two distinctly different measurements. You can test validity by defining the business rules, valid values or ranges the data must conform to, and conduct these tests electronically with IQ assessment software or your own validity routine tests.
But to measure accuracy, you must confirm the data values correctly correspond to the characteristic of the real world object or event, the data represents. To perform this test, you must compare the data with the characteristic of the real world object itself.
In the case of car seats, Consumer Reports believes, rightly as I believe, that crash tests should be conducted at high speeds, more representative of actual accident experience.
- When you make a mistake, own up to it and apologize for it. Then do everything you can to ameliorate the error and its impact.
Consumer Reports retracted the report as soon as they determined the serious problem with the study.
Jim Guest, President of Consumer Reports, wrote, “A message to our readers” on the Consumer Reports home page, with important messages to his customers, “I took action when we discovered a mistake in our side-impact crash tests.” “We strive to be accurate and fair, and I regret this error. I want to make sure that our actions are as thorough and transparent as possible so that we preserve your trust as we continue to test, inform, and protect consumers.”
- When you have IQ problems, but must have accurate and complete data, you must pay the price of the process failure and the costs of “information scrap and rework.” Consumer reports is retesting all of the infant car seats to provide the comparable data.
- “Reputation” of an information provider is not a guarantee of the quality of information provided. Even the best make mistakes.
One must error-proof its processes based on root cause of failure. A better measure is the reliability of the processes to provide consistent, quality information based on the kinds of error-proofing provided and consistency of the process results.
- When you have a significant IQ problem, you must analyze the root cause(s) and improve the process to prevent the root cause(s) from causing failure again.
Consumer Reports will be conducting extensive analysis as to what went wrong in these tests to assure they will not recur. This is the same approach when we find critical IQ problems. We must conduct root cause analysis, find the root causes and improve and verify the efficacy of the improvements to prevent defect recurrence.
What do you think?
