SubscribeData Integration Driving Job Growth
Posted in Data Integration by David Linthicum |
Lately I’ve been getting a few more head hunter calls, and many of them talk about jobs related to data integration. Trends always intrigue me and I wasn’t sure what brought this one about, so I looked into it. It seems that the Global 2000 and the US Government (The Global 1), have slowly but steadily been growing their in-house data integration skills. Thus, while cloud computing and SOA grab the headlines, pure data integration has been quietly keeping pace with that market.
According to Indeed.com which tracks job postings (see Figure), data integration job posting have increased exponentially since 2005. This is not a scientific methodology, but it’s a good metric to follow. I see similar metrics in my practice, typically in the context of big enterprise SOA and cloud computing job growth.
| data integration Job Trends | data integration jobs |
So, why is this growth occurring in data integration? I think there are a few core reasons:
First is the explosive growth of SOA during the same period of time, as you can see from the figure below. This trend tracks relative to the growth of data integration. Clearly, those doing SOA understand they need good data integration talent to make their SOA work the first time: Something I’ve been screaming about for years now. It’s impossible to do SOA right without a good foundation of data integration, including data abstraction and more traditional data integration approaches. Most are only finding that out now, and best practices are becoming better understood.
Second is the availability of technology and tools, such as Informatica 9. Data integration was more of an EAI concept until recently, and the tools sets were expensive and complex. The emerging data integration tools market is much more sophisticated and cost effective, and layers nicely within SOA lifecycles. Moreover, the shift from simple data transformation to data abstraction allows you to leverage existing databases as something more aligned with your business, which is a huge advantage as well.
Finally, data integration is getting much better PR these days within the board room. Like many concepts that include data integration, such as SOA and EAI, which were largely driven by hype in the past, data integration has a much better rep for quickly delivering ROI. In other words, data integration is a simple enough concept to understand, it provides a niche value, and does not create unrealistic expectations.
I would say it’s job security.
| soa Job Trends | soa jobs |
2 Comments, Comment or Ping
Spencer Day
You are correct that data integration is now a more accessible topic than several years ago. There are many reasons, but perhaps more than any other, it is that the COST of maintaining numerous duplicated data stores has become unsustainable. This is also driving data center management and related refactoring programs (i.e., power reduction, server virtualization, data de-duping, various SOA drivers, etc.).
The market implosion of EAI notwithstanding, no one should suggest that EAI was not well on its way toward solving some data integration issues. The problem was that EAI was itself expensive and non-standards based, thereby chasing off companies which otherwise already had a clue that data integration was highly desirable.
So, how much further along are we today? The purported cure for what ailed EAI has unfortunately been largely hype - in the form of web services and SOA, portions of which have been implemented by various companies in various ways. In my opinion, what the web services & SOA combo lacks is exactly what EAI delivered - a cohesive methodology and delivery system for integrating data. Imagine that.
It appears that IT pros can only absorb and react to a small number of major topics and their conceptual hype cycles. When proponents of web services declared EAI dead and the IT projects buying EAI products dropped off, financial resource managers were happy to wait for the cheap alternative. Did it ever arrive?
Nov 30th, 2009
David Linthicum
I really look at EAI as an architecture pattern, not a specific category of technology (I wrote EAI book). Core to EAI is data integration, as I also point out in the book. Thus, no matter if you’re using data abstraction, replication, or ETL, the core integration patterns are there. SOA was more of a core architecture pattern that incorporated aspects of EAI, so we need to make sure to understand the differences.
What has occurred in the in the last 10 years is the advancement of standards, such as Web services, and the development of key integration technologies, such as Informatica 9. Thus, EAI and more specifically data integration has gotten easier and much more cost effective. I suspect this trend will continue.
Dec 4th, 2009
Reply to “Data Integration Driving Job Growth”