Managing PowerCenter mappings and automating how to build them – it’s something that Naresh Govindaraj, Informatica Cloud’s Senior Director of Product Management, has spent a lot of time thinking about. After eight years in database and data warehousing roles at IBM, Naresh joined Informatica in 1998 to build and manage the metadata repository team. In 2004 he started focusing on data integration embeddability for the first time. Building out the PowerCenter Software Developer Kit (SDK) and the Java Mapping Framework (JMF) ultimately led him to create the concept of Cloud Integration Templates, which is now one of the pillars of Informatica’s integration platform as a service (iPaas) and the Summer 2012 release.
I sat down with Naresh to learn more about Cloud Integration Templates, where they came from and where he sees this innovation going.
Naresh, tell us a bit about the history of Cloud Integration Templates?
In the PowerCenter 7 days, I was leading a number of product management and development initiatives and one of the key focus areas at the time was around automation. In some of the larger customer implementations, PowerCenter projects were getting bigger and increasingly difficult to manage and we spent a lot of time looking into ways to automate how to build and manage mappings. We introduced a Java-based API for generating data integration mappings, the Java Mapping Framework (JMF), which remains a core component of the PowerCenter SDK. Some customers and some partners on the Informatica Marketplace use it to build applications that automate mapping creation. Internal teams like our Data Validation Option team also take advantage of the technology.
At the same time, roughly 2005, we were looking for ways to simplify data integration design in order to meet the needs of different users and use cases, with embeddability as a core focus. The Cloud team had also just been established and they had a similar objective – simplified data integration.
So it sounds like your worlds collided – embedded data integration and cloud-based data integration.
That’s right. The Cloud team didn’t want to rebuild the engine, so the JMF was timely. They were able to use it to talk to the PowerCenter server through common mappings. It was around this time that I started thinking about the Template concept. The need was still to solve the mapping management and automation problem, but a higher-level of abstraction was required. A lot of integration is done by non-programmers and everyone has the same challenge, “How do I establish a best practice to let developers generate mappings?”
We needed to provide a mechanism for DBAs and Data Analysts to solve the automation challenge. By the way, it was around the same time David Lyle and John Schmidt published their first Integration Competency Center book. Some of these concepts have since evolved into Lean Integration.
So what happened next with Templates?
The first step towards automation for a non-developer audience was the Mapping Architect for Visio. The concept of “mash-ups” was also gaining popularity with our ISV and OEM partners – bringing components together dynamically in order to deliver a so-called “composite application.” Data integration had to be embedded into the application and it had to be managed and controlled programmatically. But while new IDEs like Flash emerged, which made it easy to build apps visually; there was still no easy way to bring in data from multiple systems. SOA and Web Services APIs were emerging, but there was still no easy way to deliver consistent information across multiple systems to the composite apps. This was the genesis of Cloud Integration Templates.
So the move to mash-ups and composite apps spawned the concept of Cloud Integration Templates?
That’s right. In the world of mash-ups you have charts, components you bring into your framework. You size them and set certain attributes and it works. If you think of integration processes this way and define a Salesforce CRM to Netsuite integration as a Template for example, you should be able to just drop it in and it works (assuming the cloud connectivity is supported). Mash-up builders make it easy to build apps rapidly. What Cloud Integration Templates do is allow you to abstract integration processes so the developers can work with only what they need.
Why is having a RESTful API so important to Templates?
REST APIs make it easier to build applications compared to Web Services and Java. As a result they have become the de facto standard for cloud applications and platforms. This is why the cloud was ideal for us to deliver on our vision of Templates. What we’ve done is taken integration processes that are abstracted in Templates and REST-enabled them. This allows platform developers to build them on their platform of choice and easily embed them into their development environment and apps.
You also talk about the concept of “self-service integration.” How does this relate to Templates?
Templates enable you to build easy-to-consume and configure integration applications that solve specific problems. There are two ways to consume Templates:
- Search for what you need based on how it was named. Identify the Template and a custom-built wizard will walk users through the steps needed to configure the parameters and execute.
- Build an app on Force.com or Google app engine or some other platform. Use your native development environment to build and embed custom Cloud Integration Templates.
How are Templates related to the concept of integration platform as a service?
I see iPaaS as being about making it easier for application developers to handle the complexities of data integration without having to be experts. Templates have the same requirements. They abstract the complexity of data integration, allowing developers to focus on their application logic. Application developers want to build applications that can adapt easily to end-user customizations. Templates can dynamically adapt to changes in data structures and business rules allowing an application to be built once and deployed to multiple customers without changes. As we go forward we want to give application developers access to more capabilities including data quality, data masking and complex data processing. Cloud Integration Templates can be an easy and powerful means to deliver these functions to developers. And finally in addition to application integration scenarios, Templates will allow for abstraction of other patterns of integration including data warehousing/analytics and Big Data processing, allowing application developers to build apps that address a variety of use cases that require data to be processed easily.
Thanks for the information Naresh! Where can developers get started with Cloud Integration Templates?
There is a new Cloud Integration Developer Community and we’re about to roll out a new Cloud Integration Template Mall on the Informatica Marketplace. You can also learn more about Templates on the Informatica Cloud website.