Next Gen Analytics Strategies — The Future of Data Management is All About AI
With a successful Informatica World 2018 behind us I’d like to share a recap of my session on Next Gen Analytics (NGA) Strategies in a six-part blog series (one on each strategy). I suggest you read the blog I wrote the week before Informatica World that summarized the strategies.
- Enterprise unified metadata foundation powered by AI
- Data cataloguing is the first step
- Optimize the data pipeline for Big Data
- End-to-end collaborative data governance
- Build in data privacy and protection
- Architect for multi-cloud and hybrid
To illustrate these six NGA strategies, I’ll summarize some of the strategic focus areas from our most successful customers working on NGA initiatives. See if you recognize any of the strategies listed above as you read through these customer stories (HINT: I underlined them for you).
Multi-conglomerate creates personalized customer experiences
Every company wants to be more engaged with customers and provide a better customer experience. An example is a large Indonesian multi-conglomerate which literally touches hundreds of millions of customers. Their goal was to be able to act on insights in what they referred to as “human time”. This meant they needed to create personalized experiences for each customer across every touch point. To do this required analyzing massive data volumes created by multiple sources such as clickstreams, POS transactions, customer service transactions, and devices at properties, including Wi-Fi routers and building sensors. The Chief Data Officer (CDO) preferred the breadth of Informatica’s Intelligent Data Platform and Data Governance solution to scale the entire data pipeline for big data while protecting sensitive customer data.
Global life sciences company delivers shared analytics in the Cloud
A lot of companies started off with good intentions for their analytics programs but quickly became overwhelmed by all the challenges. Many of these challenges were related to taking their proof-of-concept (POC) implementation into a production deployment that would be easy to maintain, flexible to change, and could address a variety of business problems and initiatives across multiple divisions. Moving your analytics infrastructure to the Cloud can address many of these deployment challenges which is exactly what one of the largest global life sciences company in the world has done. They have a Cloud first strategy where they’ve deployed their analytics and Informatica data management solution on Amazon AWS.
Global energy company digitally transforms oil exploration
We’re also seeing many customers with a Cloud first or hybrid strategy move their analytics to Microsoft Azure. Which is exactly what one global energy company has done. They’re building an advanced analytics framework to eliminate data silos, discover insights, and improve productivity, efficiency and performance of an exponentially growing IT environment across cloud and on-premises. To put this in perspective they manage an exponentially growing large IT Environment with ~10,000 Databases, ~70,000 Reports and ~4,000 ETL/ELT Workflows. To manage this huge environment, they first invested in a data catalog that could scale to help discover and curate millions of data assets. One key reason they selected the Informatica Intelligent Data Platform was because it was built on an enterprise unified metadata foundation powered by AI and machine learning – what we call the CLAIRETM engine. CLAIRE helps improve productivity and efficiency by automating thousands of processes and making intelligent recommendations to guide user behavior.
What each of these companies has in common is that they’re thinking about their data from a systems perspective, as an integrated end-to-end data management pipeline that:
- Supports all types of users including data engineers, data analysts, data scientists, data stewards, etc.
- Can be deployed both in the Cloud, on-premises, and hybrid environments
- Ensures trusted data, protects sensitive data, and is governed to support compliance with industry regulations such as GDPR and BCBS2329.
- Scales for big data and real-time latencies
This requires organizations to adopt a best-in-class hybrid data management solution. The Informatica Intelligent Data Platform delivers a single, integrated, and modular end-to-end data management solution enabling customers to start small and grow at their own pace. The platform shares a common user-experience, is built on flexible microservices and is powered by CLAIRE. Which leads me to the first strategy for Next Generation Analytics: Invest in a data management platform having an enterprise unified metadata foundation powered by artificial intelligence (AI) and machine learning (ML).
A data management platform built from the ground up on enterprise unified metadata powered by AI/ML is the only way to scale the massive amount of work without hiring an army of scarce and expensive data professionals. Scanning all enterprise metadata is not enough. To make data assets truly discoverable and manageable at enterprise scale, CLAIRE automatically curates and relates datasets with minimal user involvement. CLAIRE is critical for data management because it increases developer productivity, improves operational efficiency, and improves users (business and IT) data experience. CLAIRE does this through several ways such as:
- Business domain, entity, and relationship discovery
- Semantic search, pattern identification, and data classification
- Anomaly detection and notification
- Deriving structure from messy IoT and machine data
- Cloud bursting to automatically handle data spikes
- Monitor and predict system problems
- Auto-tune jobs for performance
- Suggest data sets, transforms, and rules
- Auto-map, cleanse, and standardize from sources to target
The future of data management powered by CLAIRE includes things like an augmented reality to discover and understand the complex world of your data — think Google Earth for data. CLAIRE is becoming smarter so it can power self-integrating, self-healing systems that detect and automatically provision data. And it won’t be long before CLAIRE can automatically master and govern data such as parsing regulations and mapping them to processes and data.
In my next blog I’ll discuss Next Gen Analytics Strategies: Data Cataloguing is the First Step