Informatica Perspectives

Are BI managers and professionals sometimes too eager to please the business? Are centralized BI efforts slowing down progress? Should BI teams address requirements before the business even asks for them? These questions may seem counter-intuitive, but Wayne Eckerson, director of research for TDWI, says that the best intentions for BI efforts in many organizations may actually result in sluggish projects, duplication of effort, and misaligned priorities between BI teams and the business. [Read more]

Recently I spoke at a User Group Meeting on the topic “Align for Success: The critical part Data Quality plays in complex Business and IT Initiatives.” I began the discussion by polling the group to find out how many of the organizations represented had a data quality solution in place. The response to the question was mixed, with approximately half the audience indicating they either had a solution or were considering one, while the other half indicated they weren’t currently considering data quality (or the person was unaware of any data quality initiatives). Although this was a very unscientific survey, it set the tone for my presentation, as I attempted to explain the concept of a data quality maturity model. By understanding where an organization is today from the standpoint of the model, management can begin to develop plans as to where they want to end up both in the short and long term. [Read more]

Any budget crunches that hit organizations this year may not directly affect enterprise data management initiatives, but EDM and associated middleware will be called upon to help businesses through turbulent times. [Read more]

I’ll admit it, as an older brother, I didn’t want my younger sister borrowing or bugging me for my prized possessions. I still hoard things at work, old computer equipment, mice, cables, all in the name of finding a use for them at some point. I just like to know they are there when you need them as you can see here.

Is data treated the same way within corporations? Do application owners like sharing their data with others? In my experience, no, they don’t. Ask any mainframe or ERP program manager about utilizing their production data for other purposes and I’m sure you’ll receive a litany of questions around impact to production systems, utilization costs, and complexity of access. And IT’s business request list for access to these precious resources is only growing. For many organizations, data access is a cultural problem.
[Read more]

I’m already making my flight arrangements for the 10th Annual Informatica World Conference in Las Vegas this year. [Read more]

Happy New Year! I look forward to discussing a myriad of Enterprise Data Management topics with you this year. My work with customers never stops and I’ve made a 2008 resolution to share as much of their success as possible. I’ll start with one of the oldest but least addressed problems in Data Integration.

Have you ever asked yourself or been asked, “Where did that number come from?” or, if you’re in IT, have you been confronted by your business colleagues with “Those numbers don’t make sense!” I find these to be very common questions that consume hours and days of business and IT analyst time. Think about it, at the grass roots level of every company or organization, the amount of time spent deciphering numbers from reports is staggering.

This challenge starts from the very beginning of intelligence gathering, underlying data from operational systems. It’s why the first step in any data integration project (DW, Migration, MDM, Consolidation, etc…) is to understand and map out the nature and location of the data appropriate for the business problem at hand. An estimated 70 percent of the time spent on any corporate application development is dedicated to finding, identifying, reconciling, and verifying data, and then determining the consequences of modifying the data. This is what makes traditional integration projects so time- and resource-intensive—and what makes metadata so useful in exercising internal control or streamlining a myriad of related activities. The recent Informatica Release 8.5 launch highlighted “data lineage” for helping IT resolve questions for the business as well as providing “self service” for answering data-related questions for analysts and developers.
[Read more]

I recently asked a customer of ours why they invested so much in monitoring and publishing key performance indicators for their data quality. “Believe it or not, the biggest reason we measure data quality is not to correct bad data” came the reply. “The reason we monitor data quality is to detect problems with our business processes.”

Indeed, as I mentioned in my last blog post, business users look to investments in people and processes in addition to technology in order to address poor data quality. For example, if a bank branch manager received a report showing that customer data originating from his branch office had much higher incidents of duplicate entries and was putting the entire bank at risk of massive regulatory fines, he is not going to throw technology at the problem. His response might be mandatory training for tellers or better hiring practices to screen for adequate computer skills.

Experts in quality control methodology refer to this as addressing “root cause.” Common starting points of measurement involve completeness, accuracy, consistency, conformity, duplication, and integrity. Eventually, as the business culture matures its data quality practices, timeliness and data lineage (origination) are used to evaluate quality of data. Of course, software technology that automates the process of parsing, standardizing, matching and consolidating data is of immense value and is an absolute requirement in any data integration project. However, the issue of data quality goes beyond these IT projects. Ongoing measurement and monitoring of data quality provides value directly to the business because it helps them to better manage their people and processes.

I attended my first parent-teacher meeting the other day for my five-year old daughter. Another one of those “life stage” events done and dusted – I remember dreading the annual meeting when I was a kid. The notion of my parents and my teacher comparing notes on my behaviour was too much to bear – somebody was eventually going to put two and two together and find out I was up to no good.

It all got me thinking about a recent blog post by my esteemed colleague Garry Moroney. His post Mobilizing the Data Quality Army outlined the level of effort, thought and planning that the US Department of Education is putting into data quality.

As Garry points out dealing with data quality in a large, disconnected organization such as the US schools system is not a trivial exercise. But if you were to only read that one post you might be overwhelmed by the potential size of the data quality task in front of you.
[Read more]

A recent InformationWeek article* described the growth in IT employment across the US as a result of a shift in skills. Rather than focusing on pure IT proficiency, organizations are looking for talent with “a more hybrid mix of technology skills, along with an understanding of the business and its customers.”

IT departments are highly motivated to increase the level of collaboration with their counterparts in the business. Nowhere is this more critical than in the area of data quality and the trend is causing a shift in the way companies are looking to solve their data quality issues. First generation data quality tools had a natural focus on technology, instead of business. Here are some of the differences between technology focused data quality solutions and business-focused data quality solutions.

Tools vs. Process
Technology focused data quality solutions provide tools that automate data processing. Evidence of this type of focus can be seen in the way that vendors will tout the sophistication and type of their algorithms over and above their ability to support ongoing data quality management processes. While technology is extremely important, its relevance cannot eclipse the overall data quality management process. Even if your data quality tool can automate the correction of 95 percent of the data, if the remaining five percent cannot be managed properly, you will continue to suffer from poor data quality.
[Read more]

As a new contributor to the data quality blog site, I wanted to start by introducing myself and highlighting the types of topics I plan to discuss on a semi-frequent basis. I am a Principal Consultant with Informatica Professional Services and have spent the past 6 years in the data quality space in a variety of sales and post sales roles. During this time I have seen the data quality market continue to evolve and mature. Thus, I would like to use this column to reflect on the types of use cases I have seen and continue to see when meeting with organization’s faced with data quality problems. I hope these posts can start an active dialogue, regardless if your company is trying to tackle their first data quality initiative or looking to build out a formal center of excellence around data quality.

To start, I wanted to pose a common question I am often asked by clients and prospects – how do I build a business case for data quality? Although an organization may think (or even know) there is a problem, the need to justify the cost around procuring a data quality solution often exists. This justification requirements often comes from the idea that data quality issues aren’t necessarily a core business issue (how wrong this is!) or something that can be handled through manual intervention (this is true – if you have unlimited time and money, but even then your results will be limited). Thus, the following points are meant to help start an organization down the path to building the internal business case through a Data Quality Audit. Note - if you have access to Informatica’s Velocity Methodology, I go into these steps in further detail in the best practice document, “Developing the Data Quality Business Case.”
[Read more]