Many thanks to Loraine Lawson who wrote an insightful article on her blog Classifying Integration Initiatives as Tactical or Strategic Her posting got me thinking. 20 years ago we didn’t need Integration Competency Centers – now we do. What changed?
From my perspective, there are three key drivers:
First, organizations have gotten bigger. The Fortune 500 companies today are much larger in terms of revenue, number of employees and global operations than Fortune 500 companies in the 1980’s. This conclusion is based on personal experience and anecdotal evidence. If readers of this column have seen any studies or statistics on the growth of large enterprises, please speak up.
Second, data diversity and volume has been growing at an exponential rate. We now have data in multiple formats such as video, images, audio, and text to name a few – and in multiple protocol formats and multiple languages. On the volume front, I recall an article from a few years ago that stated that manufacturers of storage devices would ship more than 22 exabytes (22 million trillion bytes) of hard disk capacity in 2005 – which is four times the space needed to store every word ever spoken by every human who has ever lived. In a relatively short number of years we’ve seen data storage capacities move from megabytes, to gigabytes and terabytes – now we’re talking in petabytes and exabytes. And coming soon are zettabytes and yottabytes.
Third, information technology has become more complex. One of the first principles of computer science is that re-use and decoupling of components is enabled by adding an abstraction layer. When I started my career (roughly 30 years ago), I was loading programs into computers using paper tape readers and toggle switches (I was pretty good at reading octal and hexadecimal code in my prime). But with increasing layers of abstraction – macro code to structured programming languages to 4th generation languages to graphical integrated development tools to cloud computing in a web 2.0 environment – we have increasingly distanced the user from the underlying hardware and micro-code. Computers basically still work the same as they did 30 years (binary logic gates), but they are much more powerful, easy to use, and interchangeable which has been enabled by layer upon layer of abstractions.
The common elements across these drivers is constant change and growing incompatibility and complexity. Business processes are constantly changing with broader global scope adding many complications that are not transparent. Data definitions are constantly morphing and volumes are increasing to massive levels as everything becomes digitized and more regulated. Technology waves are constantly adding new variations and complexities without retiring prior generations.
Ergo, we have seen the emergence of integration as a new discipline which didn’t exist 20 years ago because it wasn’t needed then. In the 1980’s, integration was viewed as a one-time project discipline, not as an ongoing capability that an organization needed to sustain efficient end-to-end operations in a complex pattern of incompatible components that are constantly changing.
We could say that projects require micro-integration techniques while the complex systems-of-systems that emerge over time at the enterprise and supply chain scale require macro-integration best practices such as Integration Competency Centers. The ICC disciplines are a powerful capability, but they are non-trivial and hence those organizations that do it better than others will have a competitive advantage.
