Informatica Perspectives

Over the past several quarters, I’ve had the privilege of speaking with a number of companies involved in data governance. The interesting thing I found: firms who identified both drivers as critical, but only invest in one and not the other.

Case in point: a leading financial services firm implemented a data governance program to improve the comprehension and accuracy of the company’s existing board reports. I learned that one of their goals was to define their business terms and definitions (i.e. business metadata) to help non-technical users improve their understanding of the data used to run the business. What I found fascinating was that this was being done prior to addressing their data quality issues. In fact, when asked, “Do you have data quality challenges?” most business users said “yes”. Unfortunately, no one at this company knew to what extent. Instead, their focus was on defining their business metadata. This leads me to ask, “Can you trust your metadata without addressing your data quality issues as part of a data governance practice?”

If metadata is information about your data which your business users are relying on to drive decisions, but the source data is not clean, how will that affect your business? The answers seem self-explanatory. Of course you can’t trust your metadata if you have poor quality data.  For example, business metadata is defined from an approved list of valid values. Unfortunately, if the data used to define those values are incorrect, the downstream impact is you end up with inaccurate metadata.

Organizations implementing data governance programs need to consider the lifecycle of how data is captured, processed, and delivered to downstream systems— whether that is your data warehouse, master data management application, data hub or CRM system. Creating, defining, and publishing business metadata without addressing your data quality issues may not help companies looking to benefit from data governance.

In Chris Cingrani's recent post the question: "Data quality, does anyone care?" was posed. The answer is yes, of course people care about data quality – in fact, there are a lot of good reasons why a lot of people should care very deeply about data quality. Let’s look at the most recent example of where data quality makes a big difference, and that is in the federal election process. [Read more]

I recently participated in two DMRadio shows that were aired within a week of each other—the first show focused on MDM, the second on data governance.  Not surprisingly, the topics overlapped tremendously.  In fact, during the MDM show, I found myself talking primarily about the need to establish a data governance program to ensure the organizational and process alignment necessary for successful MDM deployments.  And on the data governance show, part of the conversation centered on MDM being a very common driver behind the launch of data governance programs.

While they are two separate concepts, they are closely linked.  MDM is the more concrete of the two: technology is implemented, data is cleansed, reconciled and shared, and there is a direct impact on business processes.  In other words, MDM is the yang.  Data governance can seem more abstract, focusing on aligning process and people to ensure an organization is maximizing the value of its data.  Data governance is the yin.  In accordance with the principle of balancing yin and yang, MDM and data governance each bring something to the table, and they are each improved by the other.  Dave Waddington from the Information Difference also comments on this interrelationship in this recent posting on their data governance survey.

If you are contemplating one without the other, perhaps it’s time to meditate a bit on the value of the two together.

Did you ever have one of those moments where you didn’t know you knew something until you were asked?  I was asked recently to address a question about Master Data Management (MDM) for a Customer Data Integration (CDI) initiative.  As I reflected on my experiences, it dawned on me that over the past 17 years I have been involved in well over a dozen CDI projects, but in the end they all boiled down to three distinctly different strategies.  Each strategy is distinguished by its technical approach, architectural complexity, and value proposition. [Read more]

Can business intelligence benefit from the current excitement around the rise of Web 2.0 and Enterprise 2.0? Some say the intersection of BI and Web 2.0 will advance us into “Business Intelligence 2.0,” which promises up-to-date information and actionable insights about every aspect of the business. Fellow blogger Rick Sherman recently observed that BI 2.0 isn’t just about tools and technologies, but about “getting more comprehensive, consistent, correct and current data…. We can finally interweave data from the data warehouse with real-time and event-driven data via our data integration efforts.”

Can Web 2.0 make the promise of BI 2.0 more of a reality? [Read more]

This blog post is part two of an ongoing series highlighting the importance of data in a Service-Oriented Architecture (SOA). I look forward to hearing your thoughts and input on the subject.

I'm back. It's been a little longer than normal, longer than I would have liked. Perhaps that’s because 'addressing SOA's data-centric pitfalls' isn’t easy. (Really it’s because I’ve been working on other things. But let’s get back to the topic at hand.)

One of the benefits of the SOA approach is the ability to think top-down about problems. The usual approach is to work tightly with the business to define your processes from a business perspective, leading to clearly defined services that the business understands and you can implement together.

This is wonderful and has a clarifying symmetry that Software Engineering has been trying to achieve since the days of CASE. But now, here we are in 2008 with the SOA standards defined and the tools available to potentially achieve this vision. Ah, finally, the integration hairball will be contained and life will improve immeasurably for all!

But as I talked about last time, one of the reasons that things aren’t that simple is the data-centric pitfalls. And addressing this problem is not easy if you want to take a long-term, enterprise-oriented approach.

In talking with folks who have walked down this path, struggled with data problems, and are trying to think holistically about a workable longer-term solution, three themes come up again and again: [Read more]

If banks and financial institutions had invested in more data integration and business intelligence tools to spot issues arising within their portfolios, could they have avoided the recent credit mess?

Perhaps, to a degree. But it is human beings that are ultimately making the risk judgments, and oftentimes, bad decisions may have looked good at the time they were made.

Still, technology has improved to the point where troubles could have been more effectively flagged. [Read more]

Years, ago, I came across this question in an article in Boardroom Reports:  "What do you call a hamburger that’s 99% meat and 1% garbage?"

The answer was a "garbageburger." In other words, even if a small fraction of the burger is tainted, the whole meal is tainted. The original analogy was being used to illustrate the challenges of time management, but it's an apt analogy for data environments as well. That is, if a portion of the information is bad or unreliable, trust in all the data eventually breaks down. In essence, many implementations of service-oriented architecture (SOA) taking place across companies may be garbageburgers because they are serving up unreliable information – an element that has been out of the control of SOA designers.

Sorry if I ruined anyone’s lunch, but the point had to be made. [Read more]

Predicative analytics sifts though current and historical data to predict future events or behaviors. It incorporates statistical and data mining techniques to determine patterns, highlight risks and identify opportunities.

It doesn’t just extrapolate future performance based on past performance, although that is an input. It also identifies the relationships across many factors to lay out not only what is likely to happen, but also why it may occur along with potentially how to alter the outcomes.

Where there’s data warehousing, there’s predictive analytics. It is increasingly in many industries including retail, telecom, pharmaceutical, insurance and financial services. These industries typically leverage predictive analytics to analyze and predict consumer and business behaviors along with economic predictions. [Read more]

Just as they say success is 10% inspiration and 90% perspiration, it can also be said that the success of a data integration project is 10% technology and 90% chemistry. And when I say chemistry, I'm not talking about hydrocarbons and nitrates, but the chemistry of people.

The success of any complex data integration depends on how the people that make things happen - the teams of administrators, analysts, managers, end-users, and business partners - can collaborate in establishing the business case, setting requirements, selecting technology, and putting all the pieces together.

However, two of the key players in data integration - analysts and administrators - don't necessarily see eye to eye, and this is costing enterprises in terms of staff resources and quality. [Read more]