Data Management Trends in 2016 – Our Predictions
If I am asked to make a couple of predictions regarding major data trends in 2016, at top of my list would be these: Metadata (data about data) will be ubiquitous and data-centric security will become main stream. The world of data is growing faster than the world of software and this will have a major impact on metadata and enterprise data security in 2016.
Before we dive into our predictions, let me define what metadata means to us. Metadata describes the structure, attributes, logical and physical locations, relationships, lineage, profile, and quality of the underlying data. The underlying data can be on-premise in traditional data stores or Big Data platforms, or in Cloud-based applications or data repositories. Typically, the metadata is stored as a knowledge graph (like we do in the Informatica Live Data Map) enabling the metadata to be consumed in diverse ways by IT users, business users, data stewards, administrators, and metadata-based applications via APIs.
Our first prediction for 2016 is that metadata will become ubiquitous as IT departments scale up to support the data demands of developers, data scientists, and business users.
We see a major movement underway in enterprise architectures to abstract data from applications. Data used to be closely coupled with enterprise applications, allowing limited access and usability. This is no longer the case. The democratization of data is leading enterprise architects to develop and enforce the next-generation architecture built around metadata. Enterprises are also realizing that the metadata-based approach affords them the flexibility they need to manage all their data consistently, whether it is on-premise or in the Cloud.
Designed to support a wide variety of data-fueled projects, intelligent data platforms that catalog all data assets owned by the enterprise will be implemented to improve the agility of data consumers while enabling better governance by IT. For Big Data projects involving growing volumes and varieties of data, organizations are increasingly adopting Hadoop-based data lakes. By taking advantage of intelligent data platforms, developers in 2016 will be able to productively find, ingest, prepare, provision, secure and govern the transactional and interactive data managed in Hadoop data lakes. The metadata-based approach will extend to SaaS and PaaS applications as well, enabling enterprises to treat their Cloud assets as an extension of their environment without creating new silos.
We also predict that data-centric security will go main stream in 2016 as enterprises rely on metadata to discover and protect their sensitive and confidential data.
The democratization and massive proliferation of data are resulting in a massive shift in the world of data security. Without a clear perimeter, the current network and application security solutions are no longer sufficient to protect sensitive data. Because the data is spread so widely, we have greater challenges with security breaches. Security professionals need a solution that provides visibility into where sensitive and confidential data resides, as well as visibility into the data’s risk. The best way of providing this visibility is through metadata.
It is imperative for organizations to understand their data risk – the sources of sensitive data, how it is being used, where it is going, its value and how it is protected. Moreover, data protection needs to start with the data itself and security investments should be mapped to information resources that create and/or consume sensitive data. The metadata foundation enables enterprises to adopt a “Data Centric Security” approach to improving overall information security. In regulated industries like financial services and healthcare, the metadata-based approach to security will have the added benefit of proving to regulators that companies are focusing their resources on their most critical data assets.