Informatica Perspectives

Over the past several quarters, I’ve had the privilege of speaking with a number of companies involved in data governance. The interesting thing I found: firms who identified both drivers as critical, but only invest in one and not the other.

Case in point: a leading financial services firm implemented a data governance program to improve the comprehension and accuracy of the company’s existing board reports. I learned that one of their goals was to define their business terms and definitions (i.e. business metadata) to help non-technical users improve their understanding of the data used to run the business. What I found fascinating was that this was being done prior to addressing their data quality issues. In fact, when asked, “Do you have data quality challenges?” most business users said “yes”. Unfortunately, no one at this company knew to what extent. Instead, their focus was on defining their business metadata. This leads me to ask, “Can you trust your metadata without addressing your data quality issues as part of a data governance practice?”

If metadata is information about your data which your business users are relying on to drive decisions, but the source data is not clean, how will that affect your business? The answers seem self-explanatory. Of course you can’t trust your metadata if you have poor quality data.  For example, business metadata is defined from an approved list of valid values. Unfortunately, if the data used to define those values are incorrect, the downstream impact is you end up with inaccurate metadata.

Organizations implementing data governance programs need to consider the lifecycle of how data is captured, processed, and delivered to downstream systems— whether that is your data warehouse, master data management application, data hub or CRM system. Creating, defining, and publishing business metadata without addressing your data quality issues may not help companies looking to benefit from data governance.

Now that the $700 billion dollar Troubled Asset Rescue Program (TARP) has been approved by the government, firms on Wall Street are preparing themselves for even more oversight and scrutiny by lawmakers and taxpayers. Survivors from the market meltdown will be required to establish tighter controls, policies, standards, and processes for managing and delivering trusted information for decision making, auditing, and regulatory reporting than ever before. [Read more]

I recently participated in two DMRadio shows that were aired within a week of each other—the first show focused on MDM, the second on data governance.  Not surprisingly, the topics overlapped tremendously.  In fact, during the MDM show, I found myself talking primarily about the need to establish a data governance program to ensure the organizational and process alignment necessary for successful MDM deployments.  And on the data governance show, part of the conversation centered on MDM being a very common driver behind the launch of data governance programs.

While they are two separate concepts, they are closely linked.  MDM is the more concrete of the two: technology is implemented, data is cleansed, reconciled and shared, and there is a direct impact on business processes.  In other words, MDM is the yang.  Data governance can seem more abstract, focusing on aligning process and people to ensure an organization is maximizing the value of its data.  Data governance is the yin.  In accordance with the principle of balancing yin and yang, MDM and data governance each bring something to the table, and they are each improved by the other.  Dave Waddington from the Information Difference also comments on this interrelationship in this recent posting on their data governance survey.

If you are contemplating one without the other, perhaps it’s time to meditate a bit on the value of the two together.

This blog post is part two of an ongoing series highlighting the importance of data in a Service-Oriented Architecture (SOA). I look forward to hearing your thoughts and input on the subject.

I'm back. It's been a little longer than normal, longer than I would have liked. Perhaps that’s because 'addressing SOA's data-centric pitfalls' isn’t easy. (Really it’s because I’ve been working on other things. But let’s get back to the topic at hand.)

One of the benefits of the SOA approach is the ability to think top-down about problems. The usual approach is to work tightly with the business to define your processes from a business perspective, leading to clearly defined services that the business understands and you can implement together.

This is wonderful and has a clarifying symmetry that Software Engineering has been trying to achieve since the days of CASE. But now, here we are in 2008 with the SOA standards defined and the tools available to potentially achieve this vision. Ah, finally, the integration hairball will be contained and life will improve immeasurably for all!

But as I talked about last time, one of the reasons that things aren’t that simple is the data-centric pitfalls. And addressing this problem is not easy if you want to take a long-term, enterprise-oriented approach.

In talking with folks who have walked down this path, struggled with data problems, and are trying to think holistically about a workable longer-term solution, three themes come up again and again: [Read more]

Governance is a tricky and ill-defined area. For example, in the emerging SOA space, listen to the drumbeat of messages from consultants, analysts, and vendors, and the message is clear: Service oriented architecture won’t work without governance.

However, establishing effective governance has been a vexing challenge, with a lot of disagreement and debate amongst governance proponents. [Read more]

In my last posting, I got on a bit of a soapbox about how every “solution” to the data silo problem seems to proliferate yet more silos—SOA, MDM,  and even data warehousing.  Of course, that’s because technology alone can’t solve this kind of problem.

Technologists will often point to the need for a coherent enterprise data architecture in order to rationalize all the bits and pieces of data that can otherwise spring up.   But we are all familiar with the danger of architectures becoming no more useful than wallpaper—a pretty piece of design work that never becomes materialized.

My belief is that the root problem can almost always be traced back to organizational politics.  [Read more]

For anyone in the integration business, the notion that data silos are bad is deeply engrained in the psyche.  It's plain common sense that having multiple copies of data in different places makes it a lot harder to run your business in a consistent, coherent manner.  But we keep committing the same sins over and over again– often with the very technologies that promise to solve the data fragmentation problem—SOA, data warehousing, MDM, to name a few.

First we moved off mainframes to distributed systems.  Of course, no one would doubt that the benefits in terms of access to key business data and application functionality more than outweighed the costs of silo proliferation.  At least in the client/server era, the number of silos was still somewhat manageable.

But then the internet came along, and everyone rushed to get the latest internet/web application up and running, while at best paying lip service to a cohesive enterprise architecture.  As we later learned, this lead to a huge proliferation of new systems and data silos at most companies.  [Read more]