Time to Take Data Integration off the Backburner
Posted in Business Impact / Benefits, Data Integration, Enterprise Data Management, Master Data Management, Operational Efficiency by Joe McKendrick |![]() |
It is estimated that organizations spend more than $37 billion annually on enterprise application software licenses alone - not including maintenance, services or other associated costs. So enterprise applications have been and continue to be huge investments for many companies. In fact, many companies have multiple ERP and CRM systems running across various divisions and business units. So you would think enterprises have all the bases covered, right? Not necessarily - a huge piece of these investments get neglected or overlooked, and that is the data that is tied to these systems.
This is the assertion of a new study from Datamonitor, which concludes that while billions upon billions are spent on enterprise application systems, organizations are not investing nearly enough resources or attention in the data that will be moving in and out of these systems.
The Datamonitor report is based on ongoing enterprise application technology market research and monitoring, as well as the results of a survey of 25 senior IT executives in the US, Canada, Western Europe and Asia-Pacific. The full report, written by Datamonitor senior analyst Vuk Trifkovic, can be downloaded here.
How could such a critical piece of enterprise applications be overlooked? Trifkovic says both enterprises and vendors alike tend to concentrate on making the applications operable, which is often a mammoth task within itself. "Given the high cost of implementing enterprise application platforms, the focus is understandably on making the applications operational, which includes change management and enterprise-wide adoption," he observes. "With the focus on the functionality and user buy-in of new business processes, the data that support those processes are relegated to a secondary concern."
However, the insufficient attention paid to data in major enterprise deployments may ultimately contribute to the failures of these investments, he warns. That's because the data supported with these systems may not be reliable, accurate, or up to date. "Inaccurate, inconsistent or simply incomplete data can severely impair enterprises' ability to understand their internal processes, customers or products," Trifkovic states. "Without data quality procedures in place, organizations can not grasp even the most fundamental business facts."
When organizations have attempted to respond to these challenges, it has been through custom-coded or expensive enterprise point-to-point application integration (EAI) projects. The problem is, these approaches usually come to a dead end, due to the extensive maintenance that is constantly required.
What is needed for today's environments, Trifkovic says, is a common integration platform that can address information requirements on a consistent, enterprise-wide basis. The most immediate solutions to address this challenge - and ensure the integrity and availability of mission-critical data - are customer data integration (CDI) and master data management (MDM), Trifkovic says. "The payoffs are immediate and range from improved customer support to boosted customer loyalty, and the generation of new cross-sell and up-sell opportunities," he says.
Trifkovic makes the following recommendations:
- Adopt a comprehensive data strategy. While data management gets plenty of attention on application rollout stages, it tends to get moved to a backburner in subsequent phases of the application lifecycle, Trifkovic says. "Enterprises need to formulate comprehensive data management and integration strategies that can deal with the issue of data in the enterprise application roll-out phase, operation phase, consolidation phase, and even in decommissioning phase."
- Audit current data integration and management capabilities. "The role of auditing is to establish the existing maturity level regarding the flexibility, data-quality and data latency required," says Trifkovic. "This will provide evidence supporting steps that need to be prioritized and worked out, and clues as to how to make an enterprise application platform successful."
- "Future-proof" your data integration platform. "Elements to consider range from the current maintenance of custom integration approaches, to gauging the scale of increase in data volumes and estimating the future pace of the business," Trifkovic advises.
- Take the longer, strategic view on data integration. Typically, the benefits perceived "are primarily tactical, aimed at making immediate improvements to processes," Trifkovic says. "Enterprises should also take a longer view and consider how data integration platforms can create far-reaching strategic impact through business model innovations, enabled by low-latency delivery of richer data, such as fine-grained market segmentation, personalization of products and services, and better understanding of business environment variables."
Today's business environments tend to be highly diverse, with large assortments of enterprise systems supporting different functions and business units. However, to be competitive and effectively compete on analysis, enterprise decision makers need a right-time, enterprise view on what's happening and what will happen within their businesses. A comprehensive data integration platform should be considered a standard component of any enterprise application architecture to provide the data movement backbone necessary for operational systems such as ERP and CRM. This will ensure that the mission-critical and operational data that businesses depend on is always current, consistent, complete, and reliable.













4 Comments, Comment or Ping
francis Carden
I concur with most of the points in this post but would like someone to give me a definition of Data Integration!
You see, having been build applications since the early 80's (Mainframe / midrange / Client Server / GUI / Web etc.,), I more often find that Data is over-rated! What I mean is, it's the BUSINESS logic that makes Data Integration hard. And of course, all Data is (mostly) pre-processed with business logic. More often than not, 30+ elapsed years (tens of thousands on man years) of legacy code exists around the data.
I think, until we articulate this fact, we will always make the mistake of saying "Build the data and others will come" whereas we should be saying, "share the logic and all will reap the reward".
So, whilst Data Integration is and can be important, we must not forget UI integration which is often the only way to get to REAL data!
Jan 13th, 2009
Joe McKendrick
Thank you for your insights, Francis. I'll give you the Wikipedia definition of "data integration," drawn from Maurizio Lenzerini's book, "Data Integration: A Theoretical Perspective": "Data integration is the process of combining data residing at different sources and providing the user with a unified view of these data."
Business logic has been an important consideration and priority in systems design, but a number of factors have arisen over the past decade that adds urgency to the data integration element as well. First, there's been an explosion in the amount of data being created and stored by businesses, and companies want to do more than simply pay to store it, they recognize that this data provides valuable insights into all aspects of their business. Second, initiatives such as data warehousing and business intelligence call for an enterprise view of data. Third, there is a great deal of regulatory pressure to better manage the data and assure its quality. Finally, the drive for better systems and application integration — and now SOA — requires that data be drawn from across previously separate silos and business units. As with the drive to data warehousing, there's a need to view and manage data at an enterprise level.
Jan 15th, 2009
francis Carden
I hear you Joe. I actually think the definition itself is flawed or the definition is used so liberally to describe how to solve business integration problems. VERY little DATA can actually be integrated, even after my 25+ years of doing this. I may be on my own on this but considering how hard integration is, I am pretty sure I'm right
If only Data were stored as simply as the definition defines. Take a simple piece of data (field) held in an Access Data base or a Cell in a spreadsheet or in an Oracle system or in fact in a data warehouse. Say for instance, even to view that field, I need to build some business logic to let someone know I've looked at it (HIPPA, Compliance). I have to call that code whether I view it from an SOA, from a mashup or from another piece of code! And that's just the simplest of examples! Or as likely, that field is a result of 10000 lines of business logic (code) to represent the data (that a user looks at) and cannot and must not be bypassed.
Yes, there is more DATA today that is clean BUT most is not, it requires millions of man years of code to access it / process it properly.
THAT IS WHY INTEGRATION IS HARD and not getting easier, quickly.
Jan 16th, 2009
Joe McKendrick
Thanks for the follow-up, Francis. Agreed, integration is hard work. Adding to the challenge are all those pieces of unstructured and non-traditional data — graphics files, video, etc.
Jan 18th, 2009
Reply to “Time to Take Data Integration off the Backburner”