If you say the words “data integration“, different people may think of different things. Some think of ETL (extract-transform-load) tools. Others think of enterprise application integration (EAI) technologies or message brokers. But in most cases, regardless of which tool leaps to mind, people think about how data is integrated inside of an organization, or enterprise data integration.
In other words, how data is shared between different applications and systems inside the firewall. But this is just one aspect or realm within the broader data integration discipline, albeit an important one and generally the one most people start with.
Companies and other organizations don’t just deal with their own data. They have to deal with data coming in from their business ecosystem, i.e. “the extended enterprise”. Data needed to support day-to-day business operations, such as order, pricing and inventory data, comes from customers, suppliers, distributors, and trading partners. Thus, the ability to exchange data with these B2B partners, aka B2B data exchange, is a second realm of data integration.
In the last few years, the rise of SaaS vendors has also meant that data is increasingly being hosted in cloud-based applications. It’s critical for organizations to maintain a certain level of access to that data in order to retain control. Moreover, difficulty in migrating data into SaaS applications has stalled many SaaS go-lives. So the third and newest realm of data integration is cloud data integration, which will grow in importance as SaaS adoption continues to increase.
The fourth realm is data quality, which is a key sub-discipline within data integration, and which cuts across the other three realms. After all, what is the point of moving around data if it’s garbage? Doing data integration right means ensuring high data quality across the extended enterprise.
Addressing all four realms may seem irrelevant to some folks. After all, a given project or group may only address one or two realms. However, it’s important to think of data integration not just on a project-by-project basis, but across multiple projects, and ultimately across the enterprise. It’s been shown that standardizing data integration approaches can lower costs. According to Gartner, large enterprises can save an average of 30% in integration application and data interface development time and costs, and 20% in maintenance costs, by adopting the common approach of an Integration Competency Center. Taken in the broader perspective of multiple projects, the need to address all four realms becomes more apparent.
The key to addressing all four realms in a cost-effective and reliable manner is to ensure that the underlying technology, the data integration platform, as well as the methodologies you adopt, support all four. If the tool or platform you are using doesn’t do this, you will be forced to integrate the integration, which partially defeats the purpose of taking a common approach, and which certainly adds to complexity and cost. So make sure that you consider all four realms when you think about the data integration needs of your organization. Even if some realms don’t seem critical now, their importance will become apparent soon.