Informatica Perspectives

For anyone in the integration business, the notion that data silos are bad is deeply engrained in the psyche.  It's plain common sense that having multiple copies of data in different places makes it a lot harder to run your business in a consistent, coherent manner.  But we keep committing the same sins over and over again– often with the very technologies that promise to solve the data fragmentation problem—SOA, data warehousing, MDM, to name a few.

First we moved off mainframes to distributed systems.  Of course, no one would doubt that the benefits in terms of access to key business data and application functionality more than outweighed the costs of silo proliferation.  At least in the client/server era, the number of silos was still somewhat manageable.

But then the internet came along, and everyone rushed to get the latest internet/web application up and running, while at best paying lip service to a cohesive enterprise architecture.  As we later learned, this lead to a huge proliferation of new systems and data silos at most companies. 

Then SOA came along and promised to solve that problem– you would now have reusable services, and that would solve the silo problem.  But without strong governance and enforced reuse, SOA just became a way to proliferate more silos really, really rapidly and easily.

Even if we look at disciplines that are specifically focused on rationalizing data– data warehousing, business intelligence and master data management (MDM), sinners abound.  How many times have you seen data marts and unofficial warehouses spring up around the official data warehouse, because users weren't satisfied with their access to data?  (In fact, Teradata has developed a practice around data mart consolidation, it’s so bad.)  How many BI/reporting tools does your organization have, each with their own data horde?  How many organizations have multiple MDM initiatives going on, covering different domains (products, customers, locations, etc.) and owned by different business units?

Is this ever going to end?  Some more thoughts in the next posting.