Informatica Data Services

This blog post focues on the typical data-centric challenges that SOA and BPM deployments face. Without accurate, consistent and timely information, SOA and BPM cannot effectively deliver on their promise. I look forward to your take on this.

 As we know, SOA uses a simple Web services paradigm to address high-level application integration and business process orchestration, but it cannot address more granular data issues. Semantic inconsistencies, inaccuracies, diverse data formats and access mechanisms, varying requirements for data latencies and volume are some examples. Typical SOA and BPM deployments assume the availability of readily consumable information. This introduces the costly risk of data inconsistency and inaccuracy surfacing later and undermining the business and IT value of an SOA or a BPM initiative.

I think that in order to maximize the business and IT value of an application-centric integration strategy, organizations need to look closely at data integration challenges, requirements, and prospective solutions. Focusing on application-centric integration approaches like SOA, BPM, EAI, and ESB which promise agility, unless complemented by sophisticated data integration platform,  most likely will fail to deliver on that promise. The platform has to deliver holistic and accurate information as a service to consuming applications and business processes, exactly at the speed and latency needs of the business. Our Real-Time resource center looks at this in more detail.

What do you think about real-time or right-time data integration? Do you agree that without considering an organization’s flexible latency needs (from batch to real-time), business agility is at risk?

Ted Friedman, VP Distinguished Analyst, Gartner recently stated, "Most important for organizations to recognize is that their data integration will require a mix of latencies — while real-time activity is on the increase, there will always be a need for higher-latency data integration work, since not all data in the architecture changes frequently, and not all processes, teams and roles are capable of harnessing real-time data."

Do you agree? What’s been your experience?

This blog post is part of an ongoing series highlighting the importance of EIM and how a properly strategized and architected EIM initiative can remove the cost, complexity and risk associated with enterprise integration infrastructures.

Thus, in my opinion, in order to effectively enable business agility, businesses need access to information at the speed of business, or what is called “right-time” information. “Right-time” information, as we have discussed, is information that is made available to the business at exactly the speed or latency that it is required, be it batch, near real-time or real-time. When businesses have access to holistic and accurate information exactly when it is needed, it becomes extremely easy to respond quickly to changing compliance laws, roll-out new and differentiated functionality, improve the overall customer service experience, rapidly and effectively support mergers and acquisitions, and hence enable true business agility.

It has been a busy 2 months for me as I have been trying to catch-up on all the post-Informatica World activity, as you can follow in the "Informatica World Blog." As promised in my earlier post in this series, I want to round-off this discussion around Getting EIM Right, with a summary of how I define "right-time" information. I would like to hear from you to see if you see it in the same or different light. As I see it, it is extremely important that accurate and consistent information is available at exactly the time it is needed in order to respond effectively to the needs of the business, supporting timely decision making.

If we look around, it’s a new world driven by powerful macroeconomic conditions such as globalization, growth, governance and risk mitigation. With growing challenges in achieving agility and flexibility under these conditions, businesses are starting to see increasing demand to support sophisticated operational scenarios such as consolidation of customer data in real-time to support a call center, or delivery of timely and precise forecasts for supply chain operation optimizations, etc. People and businesses seem to want to access their information much faster than ever before. Also, in speaking to a number of CIOs, IT executives and IT managers, enterprise IT organizations are increasingly trying to use their enterprise data within their analytic domains for more mission-critical applications.

As we can see around us, enterprise data is constantly being accessed, manipulated, and used by more users, through more applications, in increasingly shorter time spans. I can see this trend being reinforced as businesses increasingly adopt industry standards like SWIFT in the financial services industry, ACORD in insurance and HL7 in healthcare, to exchange information with their partners. While in some use cases it could be sufficient, effective and possibly the requirement to get information using a batch data movement mode, in other more real-time, 24×7 or mission-critical operations, live or current information may be needed to maximize operational efficiency.

Here is a graph that I like a lot and that I use frequently to explain this point, as it succinctly depicts all these factors in what I call the enterprise information latency continuum. This graph showcases both the increase in demand for more current or live information as well as a blend of analytical and operational data for enabling businesses to better respond to macroeconomic conditions all around us.

Click to Enlarge

What do you think?

This blog post features a link to an article that appears in the June 2008 issue of DMReview Magazine, written by David and myself. We look forward to hearing your thoughts and input on the subject.

This article introduces you to a data services platform as the most efficient approach for enabling business agility across the enterprise, through the delivery of right-time information, be it information delivered in batch, near real time or real time. As you will read in the article, with a data services platform you can enable scalable access, integration and right-time delivery of business-critical information to enterprise-wide composite applications. We have also tried to explain how a data services platform can maximize business value using right-time information for driving competitive advantage, lowered risk and cost-effective project implementations.

Read on…
Maximize Business Value through Right-Time Information Using Data Services - DMReview Magazine

This blog post is part two of an ongoing series highlighting the importance of data in a Service-Oriented Architecture (SOA). I look forward to hearing your thoughts and input on the subject.

Last posting, I ranted about the fact that ‘data’ is finally a topic of discussion with respect to SOA initiatives. SOA provides business services that at their deepest level interact with data. What are the data-centric pitfalls that SOA can run into?

First off, data has meaning. While an enterprise ‘meaning’ can be presented by the services to outside consumers of those services, someone has to deal with the fact that the foundational business systems may have different meanings for the underlying data. The ‘transformation’ is frequently very important and complex.

Secondly, the meaning of data can change over time as the business changes. These changes will impact the services and the ‘transformations’ mentioned above. And sometimes these changes will affect the users of the services.

Thirdly, the quality of data is not perfect. How do you deal with these imperfections?

Fourthly, the systems of record for data are not usually neatly compartmentalized. At most complex enterprises, there isn’t just one Order Management system, or one HR system. The concepts of Customer, Policy, Employee, etc., can be spread across many heterogeneous systems, with overlapping responsibilities.

I’m sure there’s a fifth, a sixth, etc. But let’s just elaborate on these four. [Read more]

This blog post is part of an ongoing series highlighting the importance of Enterprise Information Management (EIM) and how a properly strategized and architected EIM initiative can remove the cost, complexity and risk associated with enterprise integration infrastructures, I look forward to hearing your thoughts and input on the subject.

In the last post, I mentioned some of the typical modern day business concerns that were expressed to me by a number of customers and prospects. As I dug deeper and tried to understand how these enterprises were dealing with these concerns, it became obvious to me that in order to effectively deal with the business challenges, the underlying IT infrastructure needs to provide a single, comprehensive view into all business critical information assets. Also, the IT infrastructure needs to seamlessly handle the complexity of all enterprise data—its varying volume, its varying latencies, its many formats and structures.

So, does this mean that there are no existing solutions that can efficiently deal with the all the complexity of enterprise data? The simple answer is no! Existing technologies such as Enterprise Application Integration (EAI), Business Process Management (BPM), Enterprise Service Bus (ESB) and Enterprise Information Integration (EII) have fallen short of dealing with all the complexities of enterprise data. Either they have spent their time addressing only the application integration hairball and forgotten that a similar situation exists in the data layer, or they are the wrong or inefficient tool for the right problem. The problem consists of dealing with the complexity of enterprise data, its varied latencies, volumes, formats and structures.

[Read more]

This blog post is part one of an ongoing series highlighting the importance of data in a Service-Oriented Architecture (SOA) and in Business Process Management (BPM). I look forward to hearing your thoughts and input on the subject.

In 2005, I attended several SOA conferences and tried to discuss ‘data’ with attendees and vendors. Most people looked at me quizzically then ignored the topic, saying that SOA will abstract away concerns about data types, formats, location, and such. While some nodded about the importance of data semantics, there was little appreciation of the fact that without some kind of ‘data abstraction layer’ for services to utilize, everyone will end up solving the same data access, cleansing, transformation, semantic translation, and integration problems again and again, this time within java code buried within the services themselves, creating a complex, new ‘Integration Hairball’. Ouch!

But now, almost three years later, data is front and center. With new technologies, people seem to realize that this new ‘Integration Hairball’ will be created in a fraction of the time it took to create the existing, pre-SOA hairball, unless proper approaches to the ‘data problem’ are taken into account with respect to people, processes and technology around data utilized in the SOA initiatives.

Without taking the data into account from the beginning, SOA is just the next evolution of CORBA, COM, client/server, etc. Certainly, SOA can have benefits by itself, but it’s necessary to recognize that it isn’t complete without having a plan to manage the ‘data problem’. [Read more]