There is a lot of talk about using data federation, Enterprise Information Integration (EII) or data virtualization to deliver new data to the business, on-demand. However, do existing approaches cut it?
I have been following the data integration space for many years now, and like many of you, I have wondered about the viability of data federation as a data integration approach. Not because it does not hold promise – it does – it has many advantages as a fast, flexible and low cost approach to integrate multiple and diverse data sources in real-time, without the need for physical data movement.
However, according to the numerous architects that I have had the pleasure of meeting with on the Informatica 9 World Tour, simple or traditional data federation has not been able to live up to its immense promise. And why is that I asked – the reasons were many…
- SQL/XQuery-only transformations – limited support for complex transformations
- Lack of support for handling large data sets
- Inability to certify the quality of data and the propagation of bad data
- Need for separate tools, skills and re-training
- Limited data access for unstructured and mainframe data
- Need for re-building or re-deployment of the same data integration logic for other modes (e.g. batch ETL)
What is needed is a next-generation solution that can overcome the limitations listed above. As TDWI’s Wayne Eckerson says in his blog, “If you could have your cake and eat it, too, you might be able to apply ETL and data quality transformations to real-time data obtained through federation tools. You wouldn’t have to create two separate semantic models, one for data federation and another ETL; you could use one model to represent both modalities. Basically, you would have one tool instead of two tools. This would make it easier, quicker, and cheaper to apply both data federation and data integration capabilities to any data challenge you might encounter.”
Informatica Data Services combines the best of both worlds by delivering the next-generation data integration platform that offers data federation as an integrated technology, along-with inline data profiling and real-time data quality for federated views. This makes it easier, quicker and cheaper to apply both virtual and physical data integration capabilities to any data integration challenge you might encounter and also leverage existing skill sets and re-use data integration logic for any application without rework.
Here is what HealthNow NY did with this next-generation technology…in fact they call it “evolutionary and revolutionary.”
What do you think?