Using Pattern Identification for AI Solutions
I thought I was going nowhere. Now I can see there was a pattern (Kate DiCamillo)
Candy Crush. If you haven’t struggled with that addiction, someone you love probably has, and you may have found yourself considering an actual intervention. There are many explanations as to why this game is so addictive, one of them has to do with how our brain works. Our brain loves patterns; organizing the universe around us in groups or in sets helps us remember, navigate, and operate in a complex world. Candy Crush is all about creating visual patterns and about organization of repeating elements. Our brain finds this activity soothing and comforting, and we, in turn, feel happy and want more.
Pattern-finding is not just about dopamine release, it is about organizing data and information so that we can make better decisions. The more information, the more data, the harder it is to find patterns in the data and organize it in an actionable way. Today’s world, which has technology that not only generates more data than ever before (BTW, did you know that today, in two minutes, humanity takes more pictures than were taken in the entire 19th century?[i]), but is also able to store it. So today, workers who find themselves facing a pile of data are expected to achieve better results because of it, but are unable to do so.
Enter Artificial Intelligence and Machine Learning. One of the goals of this technology is to help us make sense from the overload of data we encounter every day. By identifying patterns and organizing the data into groups and sets we can make better decision and streamline the decision process, and with a streamlined process you can start defining workflows, automate actions, eliminate repeating redundant and necessary steps, and start seeing real productivity gains.
Informatica has been supporting our customers’ data integration and management requirements for over 25 years. During this time, we collected information about the decisions that people take when they move data from one location to another. Using our CLAIRE™ engine, we found that there are many common patterns that exist with certain customers; for example, they have created a singular mapping, which might be highly parameterized, that they are executing thousands of times (with the ratio of some customers in February being greater than 30,000:1 in terms of jobs run vs. assets created) for different sources/targets, or there are specific steps that most customers take when they move data to a given source. So, we started to imagine a new way for data architecture, a workflow that is automated and driven by the target definition; we call it “Target Driven Development”. Informatica’s CLAIRE AI capability is now being used to develop an integration recommendation engine that is based on repeatable patterns, offering functionality such as within-transform recommendations like field mapping, joiners, and filters, and across-transform recommendations such as quality assurance steps and a recommendation to join another source. Join me in Informatica World 2018 to learn more about the power of CLAIRE for data integration.