Three More Data Integration Best Practices that You May Not Know About
I’m often surprised by what people don’t know about data integration. Data integration is in the top 3 strategic technologies that enterprises employ. The concern is that enterprises are not getting the most for their investment in data integration technology.
Perhaps it’s time to look at both basic best practices that have been around for years, as well as new best practices that most don’t know about. Either way, it’s a good checklist for you to work through as you expand or create your data integration solutions.
Best practice 1: Understanding the data is key.
Those who want to integrate data that they have not defined to the metadata level are bound to make huge mistakes. The mistakes may not be easily undoable, such as missing the collection of key information that’s need to support predictive analytics, or other operations that need access to historical data.
The problem with this best practice is that steps to defining your data are largely unknown in the world of data integration. These days we do have tools and technology that not only helps us define the data found in our source and target systems, but manages the metadata ongoing. As things change, we can redefine, as well as automatically make changes within our data integration technology.
Best practice 2: Security can’t be an afterthought.
“Oh yeah, security.” I hear this too often. Data integration security needs to be systemic. No matter if you plan on encrypting both data at rest, and data in flight, the security approach, models, and technology needs to be pre-determining prior to implementing your data integration solution.
The good news is that there are new options today that were not available a few years ago. Identity and access management (IAM), for instance. While this may not be a fit for all data integration problem domains, there are many instances where the use of identities may be a work well for the types of security services that are needed to support data integration.
The most important thing to understand in this section, is that security needs to be systemic. You need to proactively plan for it.
Best practice 3: Gather skills before building.
So, what are some of the hardest skills to find? Cloud computing? IoT? Nope, it’s data integration specialists. Indeed, the competition for good data integration talent is fierce, and as some of the better technical types transfer their careers to focus on the more hype-driven stuff, such as cloud computing, this seems to be getting worse.
While the market for talent is humbling, if companies are able to start the search for the right brain power before beginning their journey to a well integrated enterprise, they will do okay. Those who try to find the talent just in time, will find that approach won’t work.
No matter where you are with your data integration journey, learning the how-to approaches is part of the game. Also be aware that best practices keep changing. Keeping up with those changes is a sound investment.