The ERP Data Trap

The ERP Data Trap
The ERP Data Trap

ERP systems were a true competitive advantage 20+ years ago, but not so today. ERP systems are a tool that gave people the best view into their business, but that is when there really were only ERP systems and Databases, but today that critical data resides in so many other areas. There are several reasons why ERP systems act as a data trap: technical factors, out of date management theory, and big data trends.  First, let’s talk about management theory.

There are two fundamental concepts that have been driving much of the strategic planning in modern organizations in recent decades.  The idea of economies of scale is deeply embedded in our thinking. The concept was first introduced by Adam Smith in the 18th century and reinforced throughout the 20th century by contemporaries such as Bruce Henderson. In 1968 Henderson wrote “”Costs characteristically decline by 20-30% in real terms each time accumulated experience doubles.”  The basic idea is that bigger is better.

In 1985, Michael Porter introduced us to the Value Chain.  The basic is idea is that organizations are essentially a system of processes made up of subsystems each with inputs, transformation processes and outputs. How well the value chain activities and transactions are carried out determines costs and profits.

So along come the ERP vendors who accept these two fundamental concepts and say “We know how to solve this problem.  Let’s standardize processes across the enterprise and create one large system that everyone’s needs and minimizes transaction costs.” However, the ERP vendors neglected to address a third fundamental reality of the business world.  Specifically that processes are not stable and neither are organizations (not to mention constantly changing technology). The one thing that doesn’t change is change itself.

Philip Evans, a Boston Consulting Group Fellow and Senior Partner, delivered a, insightful TED Talk earlier this year on How Data Will Transform Business.  Evans makes the point that “The nature and roll and boundaries (of value chains) are defined by transactions,” and that due to technology advances “transaction costs are falling and breaking up value chains,” and therefore there is “less need for vertically integrated organizations.”  Evans goes on to talk about disintermediation which we see around us constantly.

The point is that a management theory, which is both incomplete and out of date, has resulted in monolithic ERP systems that change slowly and have tightly coupled processes and data.  The data in these systems makes sense and are useful, but only in the context of the hard-coded processes.  In short, the data is trapped inside a complex web of thousands of database tables whose integrity is solely controlled by a rigid fossilized collection of software algorithms. If you don’t believe me, just ask your SAP support staff for access to directly update (or even read) a data table.

Evans, in his TED talk, provides another fascinating insight. That over 50% of the digital data in the world has an IP address. For data to be truly “free” and not caught in a trap, it should be directly accessible and able to be combined with other data to form new insights. The big data trend is causing the data landscape to explode while most of the ERP business suites are serving as traditional systems of record. In fact most of today’s new app initiatives  are outside the core systems of record. Plus then there are the other types of data – social, IoT, mobile, etc.  The bottom line is that 10 years ago the center of gravity relating to data could be 70-80% in an ERP system. That no longer is the case and the data landscape is now massively shifting to the newer types of things.  Your choice then is to move (at massive technical complexity) everything into the ERP world or consolidate outside of that using modular architectures.

The tight coupling between process and data in ERP systems hints at the technical reason I mentioned.  The tight coupling goes deeper when you look at the middleware that is used by ERP vendors (and particularly by SAP) to integrate its components.  Why do ERP systems even need middleware?  They need to integrate themselves since they are a collection of separately architected and developed systems that need to be coupled to form a comprehensive suite of capabilities. Furthermore, the ERP components need to integrate with other legacy and satellite and cloud systems. For example, SAP’s Process Integrator (PI or XI as it used to be called) which is the primary middleware platform used to bring the systems portfolio together is also highly procedural.  In short, the integrations are based on hard-coded procedural 3rd-generation languages.  The result, when looking at from the outside, is a “black box” collection of integrations which are difficult to change or even obtain a clear picture of the end-to-end data lineage.  Once again, data is trapped in a middleware layer that only makes sense to technical developers rather than being transparent and available for re-use by business users.

To wrap up, there is a better way to address the needs of a modern enterprise that is growing globally and changing rapidly.  You don’t need rigid monolithic ERP systems and hard-coded integrations. You can start with an agile enterprise architecture based approach that starts each strategic planning initiative with an approach that:

  1. Focuses on business functions and capabilities – not rigid processes,
  2. Views the enterprise as a collection of modular business services and operational capabilities that are candidates for improvement, and
  3. Glues all the resulting systems (whether in-house or in the cloud) with a transparent data-driven non-procedural middleware architecture like the Virtual Data Machine from Informatica.

As usual, I would love to hear your perspective on this topic.