Machine Learning and AI: The Hype is Real When it Comes to Data
Virtually every business is planning or executing a major digital transformation involving new business models, new technologies, and new processes. There are several essential ingredients for a successful digital transformation. Data is one of them – data is a critical foundation for every successful digital transformation.
In the world of data, we are experiencing not just the usual level of change. We are in the midst of a generational market disruption that we call Data 3.0. The sheer amount of data available for decision making can overwhelm the human mind. Not only is it hard to keep up with the data, we can’t even keep up with the questions we should be asking. Which is why, in addition to expertise and processes around data, we need machine learning and AI.
Even the simplest questions are hard to answer
When it comes to building the data foundation for digital transformation, there are many simple questions that most companies struggle to answer:
- How many databases do we have? How many tables/schemas do we have?
- Where do we store our customer data? Revenue data? Employee data? Product data?
- What data do we have in the Cloud? How many Cloud apps are we using and what data do they have?
- Who has access to our data?
- Which database is the source of truth? How many duplicate copies of that data do we have? …and so on
The growth of Cloud and Big Data platforms has made these questions even harder to answer. These platforms are providing powerful data processing capabilities to virtually every employee at relatively low cost. And employees are embracing these capabilities rapidly. How can an enterprise maintain control over its data in such a fast-growth environment?
This is where machine learning comes in. It’s not the pursuit of a perfect answer. It’s the search for a good enough answer — the best we can find today, further refined with new data tomorrow. Think search engine queries and Amazon’s “people who bought that also bought this” recommendations. Are those outputs ever necessarily “perfect”? No, but they’re good enough to drive the business forward, which is the point.
Our approach to machine learning, to connecting data at a machine scale with something of a human “intuition,” is CLAIRE(™). The CLAIRE engine delivers unified metadata intelligence across all of your data stores. The key is to apply machine learning to metadata. Metadata describes the structure, attributes, logical and physical locations, relationships, lineage, profile, and quality of the underlying data. The underlying data can be on premise in traditional data stores or big data platforms, or in cloud-based applications or data repositories. By applying machine learning to metadata, you find the signal in the noise more quickly, and start answering the questions we discussed above. We make the CLAIRE technology available to customers through the Enterprise Data Catalog.
Applying AI to your business data – start small, grow fast
My advice to CIOs and Chief Digital Officers is to get started within 1-2 business areas. Many of our customers start implementing their Enterprise Data Catalog in the context of a data-driven initiative like GDPR or predictive analytics or cloud migration. Other customers do it in the context of an efficiency or operational simplification initiative. What better way to save money than to eliminate duplicate copies of data that you have found through CLAIRE?
Once your team learns how to use these new machine-learning based technologies, it will spread very quickly. While enabling your company to build a data foundation for digital transformation, it will also enable you to establish control over your data in this world of Data 3.0. And that’s no hype.