Wanted: Data Integration Experts
These days I’m often asked to provide career advice when I speak at conferences. The questions usually run along the lines of, How can I do interesting things, and How can I make the most money? More often than not, my recommendation is to move to a career path that aligns with being a data integration expert.
Why? Data integration is a problem we’ve been trying to solve for the last 20 years, even as things became more complex. The rise of the cloud, for instance, means that data stores are more widely distributed and thus must be synchronized. Moreover, the movement to big data systems means that we must deal with once unheard of volumes of data. Finally, the rise of data-intensive technology trends, such as IoT and Blockchain, are expanding the need for data integration talent and technology even further.
The need for data integration experts is a steady requirement that comes from both enterprises and consulting organizations. The pay is typically higher than average IT salaries. Moreover, and most important to me, the work is rewarding.
The success of a data integration strategy and technology solution provides direct strategic value to the business. If the business processes, automated or not, are able to function with almost perfect information using data integration mechanisms, then the business can provide better customer service, better time-to-market, better inventory control…pretty much anything that’s important to the business.
It’s no secret that businesses are making heavy investments in data integration technology and talent. That being said, there is a shortage of good data integration experts, or those who are willing to learn. This has made data integration a good career choice; the demand is always high and steady.
Those who choose this path understand data integration in the wide: Concepts such as semantic mediation, messaging, streaming, ETL, real-time, near-time, and other concepts are core to understanding what data integration is all about. They also need to understand data integration in the narrow, such as a specific data integration technology, how it’s used, configured, and operated.
The long and the short of data integration as a career path is that there are few things you can do to add value to your company or organization that are more impactful than data integration. I’m finding that it’s like back pain…everyone has it, but it depends on how bad it is, as to whether or not you need major surgery to correct the problems. You’ll find that the number of data integration issues you solve will vary a great deal from organization to organization, but you’ll certainly be solving some problems.
So, where do you learn about data integration? There are on-line and in-person courses, even on specific technologies. However, my best advice is on-the-job training, including reading and learning on your own. This means joining an organization that’s willing to teach you, and show you hands-on how things work. This will be your best path.