Informatica Data Quality

IQ in the News
Most people have probably heard about the highly reputable Consumer Reports' recall of its flawed testing of infant car seat safety. The report, issued January 5, 2007, found that many car seats failed the high-speed side impact test it conducted (the government requires passing of frontal crashes of 30 mph. Consumer Reports tested at 35 mph for frontal crashes and 38 mph (so they thought) in side impact crashes. The findings seemed to indicate a high degree of failure with nine failing some or all of the crash tests, and only two doing well in all tests.
However, the government found a problem with the way the testing was conducted. Instead of a 38 mph side crash, the test simulated a side-impact crash of over 70 mph with very inconsistent results that would have come from 38 mph tests. Consumer Reports recalled the entire report January 18.
IQ Lessons Learned From the Consumer Reports Recall:
• Negative impact on consumers and their confidence in the organization:
The impacts of the faulty testing where dramatic and swift. The Executive Director of the Washington State Safety Restraint Coalition exclaimed that "Consumer Reports screwed up….They really upset people and created enormous confusion."
• When designing tests, as you will with IQ assessments, you must assure you design the tests properly. Measuring validity and accuracy are two distinctly different measurements. You can test validity by defining the business rules, valid values or ranges the data must conform to, and conduct these tests electronically with IQ assessment software or your own validity routine tests. But to measure accuracy, you must confirm the data values correctly correspond to the characteristic of the real world object or event, the data represents. To perform this test, you must compare the data with the characteristic of the real world object itself.
In the case of car seats, Consumer Reports believes, rightly as I believe, that crash tests should be conducted at high speeds, more representative of actual accident experience.
• When you make a mistake, own up to it and apologize for it. Then do everything you can to ameliorate the error and its impact.
Consumer Reports retracted the report as soon as they determined the serious problem with the study.
Jim Guest, President of Consumer Reports, wrote, "A message to our readers" on the Consumer Reports home page, with important messages to his customers, "I took action when we discovered a mistake in our side-impact crash tests." "We strive to be accurate and fair, and I regret this error. I want to make sure that our actions are as thorough and transparent as possible so that we preserve your trust as we continue to test, inform, and protect consumers."
• When you have IQ problems, but must have accurate and complete data, you must pay the price of the process failure and the costs of "information scrap and rework." Consumer reports is retesting all of the infant car seats to provide the comparable data.
• "Reputation" of an information provider is not a guarantee of the quality of information provided. Even the best make mistakes.
One must error-proof its processes based on root cause of failure. A better measure is the reliability of the processes to provide consistent, quality information based on the kinds of error-proofing provided and consistency of the process results.
• When you have a significant IQ problem, you must analyze the root cause(s) and improve the process to prevent the root cause(s) from causing failure again.
Consumer Reports will be conducting extensive analysis as to what went wrong in these tests to assure they will not recur. This is the same approach when we find critical IQ problems. We must conduct root cause analysis, find the root causes and improve and verify the efficacy of the improvements to prevent defect recurrence.
What do you think?