Data integration is reaching a new level of maturity, a level that recognizes the exigencies of corporate standards, but understands the realities of departmental needs. As opposed to a unified, centralized approach, this approach, rather, is a federated approach to data integration. Federated data integration, just like a federated approach to government creates a balance between the need for central control and consistency at the highest level, e.g. a national government or enterprise IT group, and the need for autonomy at lower levels of organization, e.g. states or departments within a larger enterprise. Federated data integration differs (substantially) from data federation. Data federation brings together data from many disparate sources into one virtual data base. Federated data integration does the same things for the mappings that define how data is integrated. The link is in the federated approach to management.
The catalyst for this federated approach to data integration is the tension between Enterprise IT groups and departmental or line-of-business IT. Departments and lines of business “go rogue” to solve an immediate need, only to find they ultimately need to tie back into other enterprise systems, or to extract data from them. Enterprise IT, which faces ever-mounting backlogs of enterprise-wide and departmental requests is unable to service all the requests it receives; and at the same time must acknowledge that those requests (most of them at least) do have pressing business value.
Most of both the spoils and the casualties of this war are data-related. One party needs data for a business purpose, another party can’t share it for fear of negatively impacting another business purpose or policy. Marketing needs data to execute a marketing campaign, legal needs data from marketing to ensure compliance with government relations, sales needs to keep data under wraps to ensure a customer is being treated in a consistent manner throughout the sales process. As data is the source of most of the strife, so too, will the resolution of its mutual and complementary use provide the solution—hence, the advent of federated data integration.
To do this, to create this federation of data, the organization needs, like a federation of states, a common currency. Data is the chief “currency” within and beyond an organization. As such, it must be possible to “transact” that currency, to move that currency from one area of an organization to the next. Data integration provides the platform that makes it possible to transact data. It allows an organization to define what data needs to move where and provides the platform for facilitating that movement. Data integration also allows organizations to specify what conversions (“transformations” is the word used in data integration parlance) are necessary for the people on both sides of that “transaction” to understand and consume that data. Essentially, data integration is the true information broker in an organization.
With an un-federated approach to data integration, different parts of an organization do data integration, well, differently. These different approaches to data integration effectively create different data “currencies” across the organizations. While they provide a consistent data “currency” and brokerage of that data currency within a specific part or parts of the organization, that data cannot be easily moved and understood beyond those areas. In order to move from one “data integration state” to the next, say, from the “Informatica state” to the “SQL state”, the person wanting the data must “exchange it,” or more specifically, perform yet another integration from one data integration platform to the next. This kind of friction results in negative business results, because much data that should be shared and communicated is not due to the difficulty of moving it.
One of the biggest reasons for the creation of these different “data integration states” is that different parts of an organization have different data integration platforms. As such, the mappings—the directions that specify where and how data is supposed to moved and transformed—are not portable from one platform to the next, and this is what creates that spawning multitude of data currencies. This is precisely the reason that Enterprise IT is so focused on standards and standardization. The reality though, is that if departments or lines-of-business are unable to get their data integration done by enterprise IT, they will still get it done—they cannot be in a position where they are unable to move this vital currency. Typically, these departments and lines-of-business don’t have the same level of funding that would allow them to use the enterprise class tools favored by Enterprise IT. As a result, departments turn to hand-coding and open source solutions to meet their data integration needs. And each new approach creates a new data “currency” that will need to be “converted” into the organization’s preferred data “currency.”
Two things now make this kind of fragmentation and friction unnecessary. The first is Vibe. Vibe is the technology underlying Informatica’s data integration platform, the platform that makes it possible to do a mapping –specify where and how data needs to be moved and transformed—and deploy it anywhere. The second is PowerCenter Express. PowerCenter Express is a free data integration and profiling product from Informatica—that runs on Vibe.
Because PowerCenter Express is available free of charge, it offers a way for departments and lines-of-business to satisfy their data integration needs, without creating a new data “currency” that will need to be converted back into the standard corporate currency. The reality (not just Informatica’s reality) is that the majority of large enterprises have chosen Informatica as their corporate standard for data integration. With the advent of PowerCenter Express, departments within those organizations can now create mappings that will tie their data into that corporate data currency—thanks to Vibe. With Vibe, a mapping can be deployed anywhere—on PowerCenter Express, on Informatica Cloud, on PowerCenter Big Data Edition—regardless of where that mapping was developed.
The combination of Vibe and PowerCenter Express is what makes possible this new era of federated data integration. It makes it possible for organizations to achieve a balance between central control—the need to create a common data currency that moves freely throughout the organization on the data integration platform—and autonomy—allowing departments to satisfy their own data integration needs, without waiting for Enterprise IT. In fact, this federation is even broader than an individual enterprise. Already, consultants are developing mappings off-site using PowerCenter Express, and deploying them on Enterprise versions of PowerCenter.