This got me thinking: What is the biggest bottleneck in the delivery of business value today? I know I look at things from a data perspective, but data is the biggest bottleneck. Consider this prediction from Gartner:
“Gartner predicts organizations will spend one-third more on app integration in 2016 than they did in 2013. What’s more, by 2018, more than half the cost of implementing new large systems will be spent on integration. “
When we talk about application integration, we’re talking about moving data, synchronizing data, cleansing, data, transforming data, testing data. The question for architects and senior management is this: Do you have the Data Foundation for Execution you need to drive the business results you require to compete? The answer, unfortunately, for most companies is; No.
All too often data management is an add-on to larger application-based projects. The result is unconnected and non-interoperable islands of data across the organization. That simply is not going to work in the coming competitive environment. Here are a couple of quick examples:
- Many companies are looking to compete on their use of analytics. That requires collecting, managing, and analyzing data from multiple internal and external sources.
- Many companies are focusing on a better customer experience to drive their business. This again requires data from many internal sources, plus social, mobile and location-based data to be effective.
When I talk to architects about the business risks of not having a shared data architecture, and common tools and practices for enterprise data management, they “get” the problem. So why aren’t they addressing it? The issue is that they find that they are only funded to do the project they are working on and are dealing with very demanding timeframe requirements. They have no funding or mandate to solve the larger enterprise data management problem, which is getting more complex and brittle with each new un-connected project or initiative that is added to the pile.
Studies such as “The Data Directive” by The Economist show that organizations that actively manage their data are more successful. But, if that is the desired future state, how do you get there?
Changing an organization to look at data as the fuel that drives strategy takes hard work and leadership. It also takes a strong enterprise data architecture vision and strategy. For fresh thinking on the subject of building a data foundation for execution, see “Think Data-First to Drive Business Value” from Informatica.
* By the way, Informatica is proud to announce that we are now a sponsor of the MIT Center for Information Systems Research.
Adrian gathered experts and built workgroups to dig into the issue and do root cause analysis. The workgroups came back with some pretty surprising results.
- Most people expected that “incorrect data” (missing, out of date, incomplete, or wrong data) would be the main problem. What they found was that this was only #5 on the list of issues.
- The #1 issue was “Too much data.” People working with the data could not find the data they needed because there was too much data available, and it was hard to figure out which was the data they needed.
- The #2 issue was that people did not know the meaning of data. And because people had different interpretations of the data, the often produced analyses with conflicting results. For example, “claims paid date” might mean the date the claim was approved, the date the check was cut or the date the check cleared. These different interpretations resulted in significantly different numbers.
- In third place was the difficulty in accessing the data. Their environment was a forest of interfaces, access methods and security policies. Some were documented and some not.
In one of the workgroups, a senior manager put the problem in a larger business context;
“Not being able to leverage the data correctly allows competitors to break ground in new areas before we do. Our data in my opinion is the ‘MOST’ important element for our organization.”
What started as a relatively straightforward data quality project became a more comprehensive enterprise data management initiative that could literally change the entire organization. By the project’s end, Adrian found himself leading the data strategy of the organization.
This kind of story is happening with increasing frequency across all industries as all businesses become more digital, the quantity and complexity of data grows, and the opportunities to offer differentiated services based on data grow. We are entering an era of data-fueled organizations where the competitive advantage will go to those who use their data ecosystem better than their competitors.
Gartner is predicting that we are entering an era of increased technology disruption. Organizations that focus on data as their competitive edge will have the advantage. It has become clear that a strong enterprise data architecture is central to the strategy of any industry-leading organization.
For more future-thinking on the subject of enterprise data management and data architecure see Think ‘Data First” to Drive Business Value
“Enterprise Architecture needs to be the forward, business facing component of IT. Architects need to create a regular structure for IT based on the service and product line functions/capabilities. They need to be connected to their business counterparts. They need to be so tied to the product and service road map that they can tie changes directly to the IT roadmap. Often times, I like to pair a Chief Business Strategist with a Chief Enterprise Architect”.
To get there, Enterprise Architects are going to have to think differently about enterprise architecture. Specifically, they need think “data first” to break through the productivity barrier and deliver business value in the time frame that business requires it.
IT is Not Meeting the Needs of the Business
A study by McKinsey and Company has found that IT is not delivering in the time frame that business requires. Even worse, the performance ratings have been dropping over the past three years. And even worse than that, 20% of the survey respondents are calling for a change in IT leadership.
Our talks with CIOs and Enterprise Architects tell us that the ability to access, manage and deliver data on a timely basis is the biggest bottleneck in the process of delivering business initiatives. Gartner predicts that by 2018, more than half the cost of implementing new large systems will be spent on integration.
The Causes: It’s Only Going to Get Worse
Data needs to be easily discoverable and sharable across multiple uses. Today’s application-centric architectures do not provide that flexibility. This means any new business initiative is going to be slowed by issues relating to finding, accessing, and managing data. Some of the causes of problems will include:
- Data Silos: Decades of applications-focused architecture have left us with unconnected “silos of data.”
- Lack of Data Management Standards: The fact is that most organizations do not manage data as a single system. This means that they are dealing with a classic “spaghetti diagram” of data integration and data management technologies that are difficult to manage and change.
- Growth of Data Complexity: There is a coming explosion of data complexity: partner data, social data, mobile data, big data, Internet of Things data.
- Growth of Data Users: There is also a coming explosion of new data users, who will be looking to self-service.
- Increasing Technology Disruption: Gartner predicts that we are entering a period of increased technology disruption.
Looking forward, organizations are increasingly running on the same few enterprise applications and those applications are rapidly commoditizing. The point is that there is little competitive differentiation to be had from applications. The only meaningful and sustainable competitive differentiation will come from your data and how you use it.
Recommendations for Enterprise Architects
- Think “data first” to accelerate business value delivery and to drive data as your competitive advantage. Designing data as a sharable resource will dramatically accelerate your organization’s ability to produce useful insights and deliver business initiatives.
- Think about enterprise data management as a single system. It should not be a series of one-off, custom, “works of art.” You will reduce complexity, save money, and most importantly speed the delivery of business initiatives.
- Design your data architecture for speed first. Do not buy into the belief that you must accept trade-offs between speed, cost, or quality. It can be done, but you have to design your enterprise data architecture to accomplish that goal from the start.
- Design to know everything about your data. Specifically, gather and carefully manage all relevant metadata. It will speed up data discovery, reduce errors, and provide critical business context. A full compliment of business and technical metadata will enable recommendation #5.
- Design for machine-learning and automation. Your data platform should be able to automate routine tasks and intelligently accelerate more complex tasks with intelligent recommendations. This is the only way you are going to be able to meet the demands of the business and deal with the growing data complexity and technology disruptions.
Technology disruption will bring challenges and opportunities. For more on this subject, see the Informatica eBook, Think ‘Data First’ to Drive Business Value.
The conversation at the Gartner Enterprise Architecture Summit was very interesting last week. They central them for years had been idea of closely linking enterprise architecture with the goals and strategy. This year, Gartner added another layer to that conversation. They are now actively promoting the idea of enterprise architects as strategists.
The reason why is simple. The next wave of change is coming and it will significantly disrupt everybody. Even worse, your new competitors may be coming from other industries.
Enterprise architects are in a position to take a leading role within the strategy process. This is because they are the people who best understand both business strategy and technology trends.
Some of the key ideas discussed included:
- The boundaries between physical and digital products will blur
- Every organization will need a technology strategy to survive
- Gartner predicts that by 2017: 60% of the Global 1,000 will execute on at least one revolutionary and currently unimaginable business transformation effort.
- The change is being driven by trends such as mobile, social, the connectedness of everything, cloud/hybrid, software-defined everything, smart machines, and 3D printing.
I agree with all of this. My view is that this means that it is time for enterprise architects to think very differently about architecture. Enterprise applications will come and go. They are rapidly being commoditized in any case. They need to think like strategists; in terms of market differentiation. And nothing will differentiate an organization more than their data. Example: Google autonomous cars. Google is jumping across industry boundaries to compete in a new market with data as their primary differentiator. There will be many others.
Years of thinking of architecture from an application-first or business process-first perspective have left us with silos of data and the classic ‘spaghetti diagram” of data architecture. This is slowing down business initiative delivery precisely at the time organizations need to accelerate and make data their strategic weapon. It is time to think data-first when it comes to enterprise architecture.
You will be seeing more from Informatica on this subject over the coming weeks and months.
Take a minute to comment on this article. Your thoughts on how we should go about changing to a data-first perspective, both pro and con are welcomed.
Also, remember that Informatica is running a contest to design the data architecture of the year 2020. Full details are here.
What would the ideal data architecture of the year 2020 look like?
Informatica want’s to know how YOU would answer that question. For this reason, we’ve created the Informatica Architect’s Challenge, a chance for YOU to share how you would approach enterprise data architecture differently. Send us your proposal and you could win 100 iPad Minis for the school of your choice.
There are a lot of challenges to think about here, but let’s start with these:
- Organizations are requiring dramatically faster delivery of business initiatives and are unhappy with the current performance of IT. Think this is “marketing hyperbole?” See the McKinsey survey.
- Data in most organizations is highly fragmented and scattered across dozens or hundreds of different systems. Simply finding and prepping data is becoming the majority of the work in any IT project.
- The problem is only going to get worse as cloud, 3rd party data, social, mobile, big data, and the Internet of Things dramatically increase the complexity of enterprise data environments.
Data is the one thing that uniquely differentiates your organization from its competitors. The question is: How you are going to architect to deliver the data to fuel your future business success? How will you manage the challenges of increasing complexity while delivering with the speed your organization requires?
It’s a chance make a positive contribution for education, while at the same time gaining some professional visibility for yourself as a thought leader. We can’t wait to see what you’ll create!
For additional details, please visit the Informatica Architect’s Challenge official page.
For years, corporations have tried to solve business problems by “throwing new enterprise applications at them.” This strategy has created ‘silos of data’ that are attached to these applications and databases. As a result, it is now increasingly difficult to find, access and use the correct data for new projects.
Data fragmentation leads to slow project delivery and “shadow IT.” Without a change in enterprise data strategy, is it unlikely this problem will improve. On the contrary, the growth of cloud applications and platforms, mobile applications, NoSQL, and the “Internet of Things” create increasing urgency. Unless a new approach to enterprise data management is taken, the house of cards is going to crumble.
I seriously doubt that this is a surprise to most of you. The question is, “What should we do about it?” The Informatica World 2014 event is a perfect place to find answers. Here are eight benefits architects will enjoy at Informatica World 2014:
- A dedicated track of breakout sessions for architects. This track explores reference architectures, design patterns, best practices and real-world examples for building and sustaining your next-generation information architecture. The track will begin with a keynote on the broader issues of enterprise data architecture. It will also include panel of architects from leading companies.
- Inspiration to participate in defining the enterprise data architecture of the future. (I can’t spoil it by divulging the details here, but I promise that it will be interesting and worth your while!)
- New insights on how to manage information for the data-centric enterprise. These will expand your thinking about data architecture and platforms.
- Chances to network with architect peers.
- A Hands-on Lab, were you can talk to Informatica experts about their products and solutions.
- Updates on what Informatica is doing to bring business subject matter experts in as full partners in the co-development of data-related projects.
- A chance to win a one-hour one-on-one session with an Informatica architect at Informatica World.
- A chance to learn to control your biggest enterprise system: The collection of all data-moving resources across your company.
We believe that architecture is the key to unleashing information potential. To compete in today’s global 24x7x365 economy, business requires well-designed information architectures that can continuously evolve to support the new heights of flexibility, efficiency, responsiveness, and trust. I hope you will join us in Las Vegas, May 12-15!
We are excited to announce the new Potential at Work Community for Application Leaders.
As an application leader, you have a very demanding job. You have to successfully manage issues such as:
- Driving the maximum business value from your company’s enterprise application investments
- Keeping all of your enterprise applications current and meeting user requirements
- Delivering on your service agreements and managing all of the “ilities.”
- Defining an enterprise application strategy that includes on-premise and cloud
- Delivering timely, authoritative and trustworthy data for your enterprise applications
This community is here to help you to do exactly that and to help you to excel in both your current job and your career ahead. Our goal is to provide tips, insights, best practices and information from experts to help you become more successful.
Our first edition is focused on the theme of managing an enterprise cloud application strategy. For those who are in the process of selecting cloud application vendors, I’ve included a very handy Vendor Selection Checklist that is used by Informatica’s Vice President of Applications.
Are we interested in your input to the community? Absolutely! If you have an idea or content to share with the community, please contact us and we will get you published.
Join the community and start unleashing your potential by clicking on this link:
Roger Nolan firstname.lastname@example.org
Julie Lockner email@example.com
Click here for more information on the Potential at Work communities.
IT application managers are constantly going through a process of integrating, modernizing and consolidating enterprise applications to keep them efficient and providing the maximum business value to the corporation for their cost.
But, it is important to remember that there is significant risk in these projects. An article in the Harvard Business Review states that 17% of enterprise application projects go seriously wrong; going over budget by 200% and over schedule by 70%. The HRB article refers to these projects as “black swans.”
How can you reduce this risk of project failure? Typically, 30% to 40% of an enterprise application project is data migration. A recent study by Bloor Research shows that while success rates for data migration projects are improving, 38% of them still miss their schedule and budget targets.
How can you improve the odds of success in data migration projects?
- Use data profiling tools to understand your data before you move it.
- Use data quality tools to correct data quality problems. There is absolutely no point in moving bad data around the organization – but it happens.
- Use a proven external methodology. In plain English, work with people who have “done it before”
- Develop your own internal competence. Nobody knows your data, and more importantly, the business context of your data than your own staff. Develop the skills and engage your business subject matter experts.
Informatica has industry-leading tools, a proven methodology, and a service delivery team with hundreds of successful data migration implementations.
To find out more about successful data migration:
- Informatica World: Visit us at the Hands On Lab – Data Migration.
- Informatica World: Informatica Presentation on Application Data Migration.
Application Data Migrations with Informatica Velocity Migration Methodology
Friday June 5, 2013 9:00 to 10:00
- Informatica World: Data Migration Factory Presentation by Accenture
Accelerating the Power of Data Migration
Tuesday June 4, 2013 2:00 to 3:00
- Bloor White Paper: Lower Your Risk with Application Data Migration: Next Steps With Informatica
- Informatica White Paper: De-Risk Your Application Go Lives