Ed Lindsey
20 Commom Data Errors and Variation
A good friend of mine’s husband is a sergeant on the Chicago police force. Recenlty a crime was committed and a witness insisted that the perpetrator was a woman with blond hair about five nine weighing 160 pounds. She was wearing a gray pinstriped business suit with an Armani scarf and carrying a Gucci handbag. (more…)
Building A Business Case For Data Quality: Create The Business Case
Building A Business Case For Data Quality, 7 of a 7-part series
Finally, you need to create a business case and present the finding of the data quality checkup. There are two levels of presentation that typically take place after the data quality assessment. The first is a technical presentation to IT giving all the details of completeness, conformity, consistency, accuracy, duplication, and integrity characteristics of the data. IT needs to understand the types of issues in order to figure out what needs to be repaired and have an idea what can be fixed and what it might cost.
The more important presentation is what impact these issues are having on the business. Does the lack of accuracy in the data affect the accuracy of business decisions? How does the completeness of the data affect insurance ratings, loan applications, or well drilling decisions? Are your customer’s committing a crime? (more…)
Building A Business Case For Data Quality: Identify Business Impact
Building A Business Case For Data Quality, 6 of a 7-part series
Once you identify the many data anomalies, you need to work with the business to quantify the business impact. If you can’t determine the impact on the business, it either has no impact or you are talking to the wrong people. If you can’t determine the impact, you might as well stop right there or find another area to look at:
No impact = No reason to fix it = No money (more…)
Building A Business Case For Data Quality: Categorize The Anomalies
Building A Business Case For Data Quality, 5 of a 7-part series
Once you finish your initial assessment, you need to summarize a very long list of potential issues you discovered. This is where you should group the issues to make the presentation of the results more meaningful. For example, you can group items by table—these are X number of issues found affecting Y percent of records per table. Sometimes I group them into the following types of characteristics:
Completeness
- Is all the requisite information available?
- Are all the address fields populated?
- Are data values missing or in an unusable state?
- Are the phone numbers populated?
- Do all the inpatient claims contain an admission date? (more…)
Building A Business Case For Data Quality: Analyze Data And Identify Anomalies
Building A Business Case For Data Quality, 4 of a 7-part series
Now comes the fun part, inspecting the data. For this step, automated data profiling will help you identify actual problems with the data as they relate to business client expectations. Here are just a few possible issues:
- Are the phone numbers empty?
- Are the admission dates missing in inpatient hospital claims?
- Are there car loans with durations greater than 10 years?
- Do shipping records lack corresponding billing records?
- Do product descriptions differ only slightly?
- Are you delivering products to many different customers with the same address?
- What business rules are being violated? (more…)
Building A Business Case For Data Quality: The Steps
Building A Business Case For Data Quality, 3 of a 7-part series
Let’s look at the steps in more detail for building a business case for data quality using the bottom-up approach. Where do you start? You need to find a sponsor—someone who instinctively knows there is a problem and wants help in quantifying it. Marketing knows it has duplicate customer records and wants to get a better handle on them. You should look at these systems or business processes that work with the customer data. You must assess how the data in these systems is used within marketing. For example, what is the data used for, what critical decisions are made based on this data, and how many people use it to make decisions? The more users or the more critical the decision, the more likely this data is a candidate for evaluation. Also look at more than the initial decision support system and data. Look at any systems that get data from the decision support system. Data flow diagrams are always helpful in assessing this but usually difficult to find. (more…)
Building A Business Case For Data Quality: Overview
Building A Business Case For Data Quality, 2 of a 7-part series
When building a business case for a data quality initiative, you can either build a case from the top down or from the bottom up. From the top down, you’ll be working with corporate executives who have already identified a problem that is most likely caused by a data issue. Some examples are:
- Fines for violating the do-not-call list
- A large number of mailings failing to reach intended recipients
- Failure to receive volume discounts from vendors because there are multiple item numbers for the same item
- An increase in the average talk time in the call center because agents are spending too much time correcting the data in the system
Building a business case from the top down is the most efficient way of doing so, but unfortunately, it is rare that metrics such as the ones above are clearly defined. Instead, you often need to build the business case from the bottom up. This approach will be the focus of this series of Blogs. The bottom-up approach starts with looking at the data first. Here you need to profile the data with an automated profiling tool, such as Informatica Data Explorer, or you can ask Informatica to assist you with a Data Quality Assessment or Business Value Assessment (which is actually a hybrid of the two approaches) service to get you started. Profiling your data can reveal its content, quality, and structure with minimal effort. (more…)
Introduction: Building A Business Case For Data Quality
Building a business case for data quality is a waste of time. Nobody really cares. Improving data quality for quality’s sake is a waste of money. Sounds funny coming from a data quality specialist, someone who has spent the last decade preaching data profiling and data quality. But the fact is people from the business side do not care about data quality. What they care about is the impact poor data quality has on their line of business.
When you look at how the business measures itself (after you get past revenue and profit), the talk is about key performance indicators (KPI). What are some of the KPIs for a call center? You will hear about goals of reducing talk time. The business wants to lower costs. You will hear about goals of decreasing hold times. The business wants to improve the customer experience. (more…)
Business Value Assessment Versus Data Quality Assessment
Here at Informatica, I provide a number of services for our customers and prospects. Two of them are the Business Value Assessment and the Data Quality Assessment. Customers are frequently confused about what the two services provide. So, let me explain each at a high level.
The DQ Assessment starts with the data. It is targeted toward the IT organization and in mainly looking at characteristics of the data. The characteristics Informatica considers important are:
Completeness
- Is all the requisite information available?
- Are data values missing or in an unusable state? (more…)
Data Quality – An Expanding Ring
I was talking to a customer the other day who is about to embark on a data quality quest. He asked me to explain my view of a data quality initiative. I explained to him that data quality is a never ending process. I said it was like dropping a pebble in the water. The initial ring is small but slowly expands outward. The data quality process is similar. You start small, with one application or one subject area, show some success then look to expand the process with additional applications or subject areas. Then expand further to additional business units or business functions until you have encompassed the entire enterprise. Then just when you think you’re done, you need to continue to monitor and repair data because it will degrade over time.
While what I said was true, he said that the business would reject that description out of hand. Business believes that all IT projects are never ending projects. It is this thought process that always pitches Business against IT. Business believes that IT never delivers a finished project. His analogy is that Data Quality is more like building a building. It takes a lot of time and effort to get the foundation right, then you need to add the structure, electrical, water, communication, and finishing work. Then you move in and begin to use the building but you’re not done. (more…)
Loading ...