Cloud Application Integration: Point To Point Or Staged Data?

In his whitepaper, Best Practices in Leveraging a Staging Area for SaaS-to-Enterprise Integration, David Linthicum outlines the difference between direct, point-to-point data integration and staging integration. He defines them as follows:

  • Direct: “Moving information from one data source and data schema to another, and translating the differences in semantics from the source to the target system.”
  • Staged: “A temporary location where the data from the source system or systems is replicated in order to support more complex and valuable data integration operations, including support for many large data sets and data operations that are more complex and of higher value.”

With the Informatica Cloud, we’ve seen tremendous interest in both point-to-point application integration and staging. It seems to depend on how software as a service (SaaS) applications like Salesforce.com are managed and the role of IT and line of business (LOB) in the implementation. More often than not, LOB managers and SaaS administrators are interested in data synchronization between front-office and back-office systems. They’re interested in supporting specific business processes such as:

  • Account, Contact, Product, Pricing Synchronization (“Master Data”)
  • Opportunity to order synchronization
  • Quote-to-cash visibility
  • Order history visibility (and analysis)
  • Data quality assessments
  • Etc.

On the other hand, while they are always interested in supporting LOB requirements, IT organizations are typically tasked with broader data integration and data quality requirements as the burden of ensuring timely, relevant and trusted information across all enterprise systems (cloud-based and on-premise) falls squarely on their shoulders.

I’m interested in learning more about the approach your organization is taking for cloud integration? Have cloud-based applications and platforms changed the roles, responsibilities, and relations of IT and LOB in your organization? How do you provide LOB managers and SaaS administrators with self-service data integration while remaining in control? Have you found a way to embrace both point-to-point and staged data integration approaches? What are your best practices?

This entry was posted in Business Impact / Benefits, Business/IT Collaboration, CIO, Cloud Computing, Customers, Data Integration, Data Migration, Data Synchronization, Integration Competency Centers, SaaS and tagged , , , , , , , . Bookmark the permalink.

7 Responses to Cloud Application Integration: Point To Point Or Staged Data?

  1. Pingback: Staging or Direct – SaaS Integration Best Practices? « In(tegrate) the Clouds

  2. Pingback: Twitter Trackbacks for Cloud Application Integration: Point to Point or Staged Data? | Informatica Perspectives [informatica.com] on Topsy.com

  3. Darren, in most of the cases, if not all, the architecture I put in place or saw implemented by customers was including a staging area (sometime call a persistent area). In the early design phase of the project, the direct point to point interface seems more attractive and looks simpler to implement. But in reality it brings much more complexity to administer, make it evolve or scale. It brings a tight couple relation between applications and maintenance can become a nightmare.

    In fact it is not technology dependent, but an architecture design paradigm. I managed some projects using Informatica technology for direct point to point interfaces as well as using a staging area. My choice definitively goes for the staging.

    And it applies both for cloud integration and on-premise integration. The main reasons I opt for the staging are:
    - It enables better business control before the data is pushed from one system to the other. E.g. in SFDC you can have a prospect that you want to become a customer in SAP, you may need to control the data (match existing customer) or enrich it before you push it to SAP. The staging becomes a fire wall so no corrupted data in propagated into your information systems.
    - It enables tracking and reconciliation of a business process. The staging area can also be used as a logging area, each time a data is manipulated, it is logged enabling the audit of any action and the visibility on any reconciliation process.
    - It enables the addition of new sources or targets with reuse instead of building the spaghetti plate of point to point direct interfaces. It responds to the SOA paradigm.
    - It breaks the dependencies between the two systems enabling asynchronous synchronization or synchronous with different size of data set (single message or bulk). And if one or the other system is down or in maintenance for a period of time, it does not affect the synchronization process because the data can be replayed (or compensated) from the staging area.

    There would be many other reasons, but these 4 are often strong enough to get to the staging area architecture.

    Ultimately, many of my customers extended the concept of the staging area to become an Enterprise Data Integration Hub that can deal with any format (file, XML, direct DB access), any volume (individual messages, large set of data), any latencies (RT, near RT or batch), and any protocol (WS, JMS, FTP, HTTP, REST, etc.). The DIH can work for point to point interfaces or publish/subscribe and becomes the core component of an integration strategy enabling the company to handle any type of data integration to respond to any IT or application maturity.

    Customers having implemented this approach saw tremendous ROI and flexibility for managing any type of data integration needs.

    Hope it helps.

  4. WebWatcher says:

    This is really a awesome web blog. The postings in this blog are very outstanding and i liked it. I shared a link on Facebook.

  5. Da frage ich mich beim groben Durchlesen ja schon, ob man selbst nicht irgendwie bescheuert war. Danke für eure Berichte

  6. Deutsch says:

    Sehr spannender Beitrag. Vielen Dank für die Bemühungen die du dir gemacht hast. Viel von dem Post ist mir neu. Das Thema kenne ich allerdings auch bereits, da haben wir was gemeinsam. Viel Glück weiterhin mit deinem Blog.

  7. Pingback: SAP, meet Force.com #socialenterprise « In(tegrate) the Clouds

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>