21

10 Weeks to Lean Integration

Lean Integration is a management system that emphasizes focusing on the customer, driving continuous improvements, and the elimination of waste in end-to-end data integration and application integration activities.

Lean practices are well-established in other disciplines such as manufacturing, supply-chain management, and software development to name just a few, but the application of Lean to the integration discipline is new.

Based on my research, no-one has tackled this topic directly in the form of a paper or book. But the world is a big place, so if some of you readers have come across prior works, please let me know. In the meantime, you heard it here first!

Lean manufacturing is a management system that emphasises creating value for end customers and eliminating activities that are not value add (waste). Its principles were derived from the Toyota Production System (TPS) which was developed over 50 years ago, but since the 1990’s is simply referred to as Lean. While Lean is rooted in product manufacturing, it is now widely regarded as a management approach that can be applied effectively to a wide range of product and service industries. Lean is closely related to, and borrows from, other methodologies including Value Network, Theory of Constraints, Six Sigma and Statistical Process Control including the works of W. Edward Deming.

Lean Software Development is an agile approach that translates Lean manufacturing principles and practices to the software development domain. It was adapted from TPS and introduced by Mary and Tom Poppendieck in their book Lean Software Development and expanded in Implementing Lean Software Development.

Lean Integration builds on these prior works by applying their principles to the process of integration. The definition of integration used in this blog series is “the process of making independent information technology elements work together as a cohesive system”.

Lean Integration may be applied in various integration domains including:

  • System Integration is the bringing together of component subsystems into one cohesive system and ensuring that they interoperate effectively.
  • Data integration accesses data and functions from disparate systems to create a combined and consistent view of core information for use across the organization to improve business decisions and operations.
  • Enterprise Application Integration enables information exchange and process automation across business applications that were independently developed, may use incompatible technology, are typically based on different data models, and remain independently managed.

Software systems are, by their very nature, flexible and will change over time (except for many legacy systems which, by definition, are not very flexible). Integration in each of these domains therefore is not just a one-time activity and instead is an ongoing activity. In summary therefore, Lean Integration is the application of Lean principles to the challenges of data integration and application integration.

Over the next 10 weeks, I intend to clarify what Lean Integration is and how to implement it in a series of blog postings. The rough plan (which is subject to change as my research progresses and readers submit comments) is based on the seven original core principles of Lean as follows:

Week 1: Introduction of Lean Integration (this posting)
Week 2: Eliminating waste
Week 3: Sustaining knowledge
Week 4: Planning for change
Week 5: Delivering fast
Week 6: Empowering the team
Week 7: Building in quality
Week 8: Optimizing the whole
Week 9: Deming’s 14 Points
Week 10: Practical Implementation Considerations

If you’d like to get Informatica blogs on the fly, just click the ‘Subscribe’ button on the right side of the screen.

FacebookTwitterLinkedInEmailPrintShare
This entry was posted in Data Integration, Integration Competency Centers, Operational Efficiency and tagged , . Bookmark the permalink.

21 Responses to 10 Weeks to Lean Integration

  1. I didn’t call it “LEAN” but in the last 2 months, came up with an analogy based on similar points you make. My analogy was around how “USERS” today, despite using computer systems are actually very manual in their business process. Using Agile integration technologies actually makes users (and IT integration) very lean! So the same analogy I used was taking the old automobile production lines and how automated they are (that we now take for granted). So, whilst not directly the same, certainly interesting.

    I’m intrigued to see your next blogs on this subject as this is something I have been doing for 25+ years. This is very relevant in 2009 and beyond given enterprises are talking lean and agile integration at every today since the downtown. I am not sure yet f I buy your meaning that legacy systems are not very flexible. If my definition of legacy, which is a legacy application is any application written 5 minutes ago, matches yours, then I concur. I see applications often very limited to integration, even year old apps and even SOA. Developers seem to struggle and possible always will, to write stuff that is really replaceable or upgradeable without difficult integration. After all, we haven’t really made a dent in this problem in 40+ years, have we!

  2. John Schmidt says:

    Francis, thanks for your comment. I agree that is some respects, this is not a new practice and that it has in fact, at least to some degree, been in use for 25 years. I have case studies from IT shops where I worked 10 and 15 years that demonstrate the value of doing systematic integration as a repeatable process – I called it an “Integration Factory” at the time. What is missing is a “label” that we can all use and a way to teach it to others – which means we need books, articles and training courses that all support the core set of Lean principles in a consistent fashion. Hopefully this blog series will help fuel the interest.

  3. Bulat Ashimov says:

    John, thank you for the opportunity to be one of the luckies to be the first to be introduced to Lean Integration.
    Could you tell more on how LI can help with Business/IT Continuity?

  4. John Schmidt says:

    Bulat, thanks for your comment. When you speak of Business/IT continuity, presumably you are referring to alignment. There are two of the core principles of Lean that can help. First, Lean suggests keeping the customer in mind and only performing activities that add value to the end customers. Activities that don’t add value are a form of waste and should be eliminated. In any event, if everyone in the enterprise is focused on the customer, then alignment between different functions is much easier.

    Second, as I will discuss in Week 8, Lean suggests focusing on optimizing the whole versus optimizing the parts. By establishing broad-based metrics (such as a balanced score-card) and encouraging cross-functional teams to all support the goal of optimizing the end-to-end process (often with trade-offs that sub-optimize one or more functions), then this also is a terrific driver of Business/IT alignment.

    There are other elements as well, but these are the two that spring to mind.

  5. Rakesh says:

    John,

    This is just incidental that I also have been working on concept of Lean ICC (which includes lean integration) as next-generation view-point. In terms of ingredients of the concept, some of the points exactly match wit what you have pointed out while there are some other new dimensions that I believe will take prominence in coming time. Two key components that I believe are critical are ability to change and secondly self-service capability of the entire eco-system. I’m also putting up the series of blog on this topic and it will be great to see your perspective on it.

    http://www.infosysblogs.com/bpm-eai/2009/02/whats_next_for_integration_com.html

  6. John Schmidt says:

    Rakesh, thanks for your comment on this posting and on Part 4 of the series. It took me a while to respond since apparently your comments were stuck in our spam filter for a while – my applogies.

    In any event, I really appreciate your comments and for directing me to your blog as well. In response, I’ve posted a blog article about change leadership which is a topic that you seem to be writing about a lot. Check it out at http://blogs.informatica.com/perspectives/index.php/2009/03/16/change-management/,

  7. Rob Jacobs says:

    John,

    Sounds like Six Sigma re-packaged for Data Integration. Why re-package it? How about integrating the Data world into the existing business program focused on improving efficiency and effectiveness?

    I believe linking to the business process and focusing on the critical customer requirements is the ONLY way to drive Data Integration.

    Rob

  8. John Schmidt says:

    Rob, thanks for your comment. Lean is distinctly different that Six-Sigma, although some people use the terms loosely and see them as synonymous. Six-Sigma is an “analytical technique” that focuses on quality and reduction of defects while Lean is a “management system” that focuses on delivering value to the end customer by continuously improving value delivery processes. Lean management may at times use Six Sigma techniques for analyzing certain types of process problems.

    So from this perspective, Lean does exactly what you suggest – improve efficiency and effectiveness by focusing on critical customer requirements. But it does so providing a robust framework that has matured over the past 5o years at Toyota and other companies.

  9. Richard Trapp says:

    John,

    I like how you’ve taken that which has worked well elsewhere and adapted it to Data Integration. I am a big believer in the concept that there really are very few original or genuine ideas, rather they are more times than not, variants on a theme. Your blog reaffirms that for me.

    Overall, I find the series well thought out and constructed and it does a good job of telling me what Lean Integration is, its applicability and even high level considerations for deployment.

    I am left with 2 basic questions, though. Perhaps the intent of the blog wasn’t to address these questions, but I offer them as fodder.

    1). What is the “So What?”. As an IT professional or LOB owner, why am I going to make a considerable investment in Lean Integration? How is this different than other methods/approaches/frameworks I have seen or been exposed to? Or, why am I as an IT professional going to embrace and sell Lean Integration internally? As a Business sponsor, what problems will it help me solve?

    2). “Where Do I Get One?”. Assuming I am beyond the first question, I will obviously need help deploying one of these. Does Infa have an offer?

    Realize, my experience in this space is more from a customer standpoint. I have been approached numerous times by application, strategy and system integration vendors who claim to have cracked the code for CDI or MDM. Most notably, there has been much talk about Services Oriented Architecture curing all ills. However, the reality of SOA is that very few have successfully implemented it to any appreciable scale and realized the benefits that were intended. In this sense, it is not unlike the benefits of an Enterprise Architecture Framework….and who can argue with Zachman? But at the end of the day, enterpise wide “deployments” of a contextual, conceptual, logical, physical model on any appreciable scale is rare?

    Rich

  10. John Schmidt says:

    Rich, these are good questions. So good in fact I plan to write a blog posting in order to respond more fully. But in the meantime, here is an initial quick response.

    What’s the “So What?” As an IT professional or LOB owner, why am I going to make a considerable investment in Lean Integration?

    Answer: The answer is different for various stakeholders. As an IT professional, the biggest reason is to do more with less. Budgets are constantly being cut while expectations of what IT can deliver are rising; Lean is a great way to respond because it embodies continuous improvement principles so that you can keep cutting your costs every year. By doing so, you get to keep your job and not be outsourced or displaced by a 3rd party. As an LOB owner, the biggest reason is time-to-market. A Lean organization can deliver solutions faster (just in time) due to automated processes and mass-customization methods. As an enterprise owner (I added this one), the biggest reason is governance, regulatory compliance, and enterprise risk reduction.

    How is this different than other methods/approaches/frameworks I have seen or been exposed to?

    Answer: Two words “sustainable” and “holistic”. Other approaches either tackle only a part of the problem – or only for a short-term or at a point in time. The predominant Integration strategy, even today, is customized hand-coded solutions on a project-by-project basis without a master plan. The result is many “works of art” in production which are expensive to maintain, require a long time to change, and are brittle in operation.

    Why am I, as an IT professional going to embrace and sell Lean Integration internally?

    Answer: Because it will advance your career. For example, in one of my prior roles in a fortune 200 company, over a three year period my team grew from 13 staff to 180 staff. I attribute that success to following lean principles; a clear picture of my customer and what they valued, elimination of non-valued-added activities, continuous improvement and team empowerment.

    “Where Do I Get One?” Does Informatica have an offer?

    Answer: Yes. The Informatica professional services organization has been trained in, and relies on, a collection of best practices called Velocity. The Velocity methodology went through a major overhaul in 2008 including incorporation of eight competencies which fall under the Lean umbrella.

    As a Business sponsor, what problems will it help me solve?

    Answer: Get results faster – and be able to sustain them in operation. Lean delivers results faster because it focuses heavily on techniques that deliver only what the customer needs (no “gold plating”), process automation and mass customization. In terms of ongoing operations, Lean is a data-driven fact-based methodology that relies heavily on metrics to ensure that quality and performance remains at a high level; and you can’t manage what you can’t measure.

    In summary, Lean begins to transform integration from an art into a science; a repeatable and teachable methodology.

  11. Mark Pritchard. says:

    Hi John,

    I am reading with interest your 10 week series on Lean Integration. It looks however that the link on the first article for weeks 6 and 7 seem to have been comnined. So, clicking on either sends me to week 7′s blog. I’d be interested in also reading week 6. Is there a webmaster who can fix this?

    Regards,

    Mark

  12. Dave Venus says:

    Hi!

    I think it would be great if you would make these 10 articles available together as a single PDF file.

    Thanks!

    Dave

  13. Pingback: Cloud Computing: The Emergence of IT Outside of IT? « In(tegrate) the Clouds

  14. sven de bruyn says:

    thanks Des!

  15. Sreedhar Bolneni says:

    John,
    Suppose a company has not ICC setup or has a disfunctional ICC and now we have Lean Integration. Should the company first setup ICC and then move towards Lean Integration or can the company make a gaint stride towards Lean Integration with out even setting up ICC?

    Could you please clarify.

    Sreedhar

  16. John Schmidt says:

    Mark, the link to Part 6 has now been fixed. Thanks for pointing out the defect.

    Thanks Des for providing the link in the interim.

  17. John Schmidt says:

    Dave, the 10 weeks to lean integration blog series is also available as a white paper in PDF format. You can download it at http://vip.informatica.com/content/Downloads?docid=6962&lsc=NA-2009Q2-JS-OE_Lean_Integration_White_Paper

  18. I would like to chat about a methodology I developed, that is complementary to your approach. Could you email me off-line? Thanks!

  19. Ben says:

    John, I fully agree with Rob that for any large (Lean or not Lean) Integration effort to work and succeed it needs to link to an EXISTING business program, even if Six Sigma is not a right one.

    Ben

  20. Pingback: Lean Integration and ICC | Integration War Stories

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>