Good architecture is the foundation required for agile systems that are responsive to business needs. We see that with SOA, which has been missing a vital piece of its architectural approach – from the data. Since it came about in its current form in the early-to-mid 2000s, service-orientation (mainly focused on applications) has existed in a separate world from data management.

Problem is, an SOA-enabled infrastructure with bad data flowing through it can be useless and even dangerous. One commentator even compared SOA to a mosquito that can deliver payloads of bad data (”viral data”) at lightning speed all across the enterprise — pandemic style — before it can be stopped.

The recent launch of the SOA Data Integration Architect Community (SDIAC), an online community focused on the value of data integration and data services in agile architectures such as SOA, promises to help bring the data and SOA worlds get closer together. The organization will bring support to SOA proponents struggling to address data quality issues and data architects seeking to service-enable their environments.

But what will the role of the data warehouse be in a data-driven service oriented architecture? Where and how will these behemoths fit? Seeking to address this piece of the architecture puzzle, long-time data warehousing expert Barry Devlin observes that the role of the traditional data warehouse has been supplemented to some degree by initiatives such as SOA and Web/Enterprise 2.0.

More so than data warehouses, SOA and Web/Enterprise 2.0 are re-arranging the relationships and boundaries between operational, informational, and collaborative systems. They bring in data from various sources across the enterprise, and provide new ways for end-users to manipulate and mine the information. He notes that SOA, for example, is bringing modular, plug-and-play approaches into the equation, giving rise to possibilities of new types of analysis applications being integrated into various enterprise applications.

The data warehouse approach fell out of vogue, as it became untenable to bring together so much information exploding all across the enterprise into a single, centralized database.

Devlin urges that this model be revisited, however – in an updated form. He observes, in line with Occam’s razor-style thinking, that a data warehouse architecture is the simplest solution to an increasingly complex issue:

“The simpler the answer, the better the solution. If you want to create a consistent, integrated information resource, you must stop creating duplicates of existing information that have to be managed to consistency, and you must eliminate, or substantially reduce, existing data duplication. The original data warehouse architecture did this. It proposed a logically single data store – the business data warehouse – modeled at the enterprise level as the consistent and integrated source of all information for decision making. This simplicity was ultimately lost with the emergence of the layered architecture, due to a combination of database performance and enterprise modeling issues.”

Devlin proposes an architectural approach he calls “Business Integrated Insight” (or BI 2). This is an architecture that “gathers all the information of the enterprise, hard and soft, operational, informational and collaborative into a single logical component.” A second integration point consists of “a logically single, consistent and integrated set of processes that spans all aspects of business and IT needs is required to enable the flexibility and adaptability that modern business requires.”

Devlin also posted a white paper explaining Business Integrated Insight architecture in more detail.

Essentially, Devlin proposes a three-tier architecture:

• Layer 1 – Business Information Resource: This is the single foundational layer with all information —operational, informational and collaborative information, as well as metadata.
• Layer 2 – Business Function Assembly: “The middle layer consists of processes, workflows, tasks, applications, tools and so on: in short, all functionality and processing that runs on the business’ computers.”
• Layer 3 – Personal Action Domain: This is the interface layer to all business users within the company.

The value of such a simplified view is to provide an new architected approach to support the demands on businesses that  are highly interlinked and interdependent, with entwined processes and pressured reaction times, Devlin says. “Business and IT, in particular, need to take a fully-integrated view of what is required from its IT environment.”

While it’s uncertain whether Devlin’s proposal will catch on, he has pinpointed one of the greatest challenges in bringing together multiple initiatives across the enterprise, involving applications and end-user interfaces. Effective data management can’t happen in a vacuum, and enterprises are begging for simpler, more straightforward architectural solutions to out-of-control complexity.