If you are asked “what is the biggest application in your organization”, what would you say? If you’re in banking you might say it’s the Hogan deposit system. If you’re in Telecom maybe it’s the Amdocs Customer Care and Billing system. If you’re in retail, you might say the Retek Merchandizing system. If you are a manufacturer, it might be your SAP ERP system. The list goes on, but you get the point. The prevailing perception is that the core business application of whatever industry you are in is the biggest application. But this is a case where perception is not reality.
As I wrote in A Big Hairy Audacious Vision, Integration Systems are Business Systems. The Enterprise Data Warehouse is a system and so is the Master Customer Repository or the Data Services Bus. Integration systems serve a business purpose which is to present a consolidated or aggregated view of data from multiple systems, or to distribute data internally or externally to other consumers of the data. Consolidating, reporting and distributing data is a business function; if we didn’t have systems to do it, then people would have to do it manually. Collectively, I call these capabilities the Enterprise Integration System (EIS) and it is arguably the biggest application system in most large organizations.
The reason you don’t see the EIS is because, in many organizations, there is no clear ownership and accountability for it. Much of the integration legacy is point-to-point interfaces that were developed as extensions of business systems by independent project teams. You may have someone responsible for the data warehouse, or the enterprise service bus, or the Business Intelligence COE, or the ETL Integration Competency Center, but in many cases these are still silo solutions that don’t have a holistic view of integration across the enterprise. Just like dark matter is something that we can’t see but is a huge force in causing expansion of the universe, so too is the EIS in most enterprises invisible because of its highly dispersed nature yet it is the most powerful force for maximizing the potential of data.
Last week I wrote in The Biggest Dirty Little Secret in IT about how IT processes that are largely manual don’t scale well and result in diseconomies of scale in the largest data-driven organizations. I would go further and say one of the IT processes with the highest percentage of manual effort is data and systems integration. Therefore, the best place to start driving improvements is with Lean Integration.
There is another reason why you could start your improvement journey with integration; it is precisely because it hasn’t had clear accountability for all the years as the hairball has grown. One of my MBA professors once said “If you’re not measuring it, it’s too expensive”. Your ERP system may not be without waste, but at least there is clear accountability for it and with management eyes and attention on it, it should be reasonably efficient. The point is that anytime you have a complex system, if no-one is focused on it, then waste creeps in. Waste might be a result of different project teams re-inventing the wheel for common functions, implementing their own middleware systems, or duplicating data copies due to lack of visibility of what already exists or because it seems too hard to gain agreement on how to share a common copy. Regardless of the reasons, my experience with a number of real-life case studies is that any large enterprise that has been in business for over 25 years has an EIS that is 25%-50% waste.
How can you determine how big your EIS is? While there are some challenges in quantifying exactly how much is at stake, it is conceptually straightforward. First, classify all your applications as either business systems or integration systems (or if a system is a blend of both, apply a percentage based on resource consumption). Add up the direct costs for each system including hardware, software, infrastructure, labor etc. For indirect costs, or shared resources use an allocation scheme. For example, if 30% of the servers in a given data center are doing data integration or data movement, then the EIS should be allocated 30% of the data center costs like electricity, floor space, etc.
Once you’ve identified all the integration components, then you can start rationalizing and consolidating them. For example, the Data Integration Hub is one approach as Bertrand Cariou explains in this Brainshark presentation to eliminate the integration hairball. And check out this case study of a telecom company implementation of a PowerCenter-based Data Integration Hub.
In summary, not only is the Enterprise Integration System the largest application in most enterprises, it is also the one with the most waste and offers the greatest opportunity for dramatic cost savings, not to mention reduced risk, improved performance, and increased agility to adapt to changes. As you start your lean journey, you want to be able to demonstrate some early wins. This doesn’t mean that squeezing the waste out of the integration systems is easy – if it was it would have been done already. It is hard work and requires buy-in from a wide range of IT and business stakeholders. But the opportunity is real and the numbers are huge! So go for it!
If you’re coming to Informatica World next week, look me up so we can talk in person about the biggest application in your enterprise and how the Data Integration Hub can help “lean” the integration costs. Hope to see you there!