I am back after a somewhat self-imposed hiatus during which I have been doing some soul-searching, or rather talking to a number of practitioners, experts, thought-leaders and analysts in the integration space. My singular quest was to uncover some real-world myths about SOA.

I spoke to a variety of integration experts – enterprise and application architects, application developers, data architects and data integration developers. During these interesting conversations, we discussed real-world SOA…or let me qualify that term further as real-world “service-orientation.”

Of course we discussed paradigms such as “loose-coupling,” “modularity,” “services,” etc., but more importantly in many cases, we spoke at length about how they were falling short of realizing the promised benefits of SOA. On probing each usage scenario further, I chanced upon a couple of interesting myths about SOA, which I would like to share with you.

So, here goes:

• Myth # 1: SOA “guarantees” business agility

• Truth: Applications and business processes require timely and accurate information but SOA often overlooks or worse yet, simply assumes this requirement

• Myth # 2: The proverbial SOA technology stack does “not” need to include a data integration layer

• Truth: Traditional technology underpinnings of SOA such as EAI and ESB, attempt to, but in many cases cannot efficiently deal with data volumes and latencies

In my mind, there is a serious problem. The glaring point here is that traditionally, SOA has been viewed as the domain of the application architect or developer and this creates an undesirable separation from the world of data integration. As a result, data integration often becomes a “blind spot,” so to speak, which only becomes relevant when things get quite unmanageable.

Let’s take a real-world example that was described to me by an enterprise architect at a leading financial services company. Until recently, they prided themselves in having what they regarded as a complete SOA stack. Their SOA implementation included a web-based self-service portal which was supported by a set of business services, an EAI and ESB product, a business process orchestration technology and a service-oriented registry. On paper, this seemed like a complete SOA stack that could ensure that the IT organization would meet the business’ needs and guarantee agility. However, once this framework was in place, it was a daunting challenge to get consistent, accurate and fresh customer information to the portal while going across multiple, distributed and varied data sources.

What followed was an IT fiasco as they did not think through their data integration situation in detail.Their back-end data sources processed customer information at varying latencies such as batch, trickle-feed and real-time. Their customer information across these heterogeneous data sources was incomplete, inconsistent, inaccurate and not in the desired format. Further, there was no seamless way to prove who touched the information and when the information was updated to support their requirements for regulatory compliance.

Some of their application architects did try their hand at repurposing their investments in technologies such as EAI, ESB and even hand-coding in an attempt to deliver the desired information to the portal. But alas, they were using the wrong tools for the job which resulted in project delays, increased customer dissatisfaction, eroded profits in a highly competitive market and above all, increased expenses in dealing with data integration issues.

As you can see, it is rather unfortunate that data integration often becomes the “blind spot” in an SOA implementation. Without a complementary service-oriented approach at the data layer, SOA will more often than not, continue to fail. What SOA needs is a silver bullet that can guarantee agility through a proactive, efficient and cost-effective approach to delivering timely and trusted information, as have I covered in one of my previous posts, Data Services – The Silver Bullet for SOA’s Data Integration Pitfalls.