Chasing Metadata to Improve Decision Quality
You could say that our lives are ultimately a collection of decisions – small and big, personal and business, short and long term oriented. They could have the potential of impacting a number of lives, altering the course of an organization’s trajectory, or influencing our individual happiness. What’s interesting in all of these situations is the fact that the less direct factual knowledge we possess, the more emphasis we place on what else is going on. Enter the context. Circumstance, history, comparative evidence, anecdotes, and even our own gut feelings try to compensate for it. They attempt to make up for the direct contextual intelligence that we may lack in a number of critical decisions that we face. Factual knowledge and evidence remain a veneer that seems to fall a little short of providing a comprehensive layer of confidence.
Presence of a constraint in one or another form, be that time, cost, data, or our own processing ability, tends to complicate this picture. We still face the critical challenge of making the best decision based on imperfect and incomplete knowledge. How can we get help in these situations? How can we increase the confidence in our decisions and as a result, ultimately enhance the quality of our choices? That’s precisely the role of context intelligence. There needs to be appropriate care and focus in this regard. “Contextual intelligence can’t be rushed or mandated into existence,” says Tarun Khanna, Jorge Paulo Lehmann Professor at Harvard Business School in a recent article, “Context Intelligence” at the Harvard Business Review. German retailer Metro Cash and Carry had learnt a few things about local context when it had expanded to other parts of Europe and Russia before it launched operations in China. But China and later India introduced a number of context issues that were result of local tastes, preferences, and situational challenges. It took Metro 14 years to break even in China, which speaks volume to importance of getting context right.
Metadata: The Secret Sauce?
Metadata is really about providing the color behind the data that we deal with. It brings context in to life. Metadata augments our engagement with the data by connecting meaning with purpose and goes on to create a new kind of data in the process – the kind that adds the context. Metadata truly embodies the soul of the data, one might add, so that it can come to life and be useful. But as much as metadata defines what data is and how to get the most use out of it, such context also underscores what isn’t in the scope of the data in question.
Context, in this manner, provides an invisible boundary – a framework, if you will, to help narrate a story better through the data on hand. Context when coupled with content, even when incomplete, provides the foundation for understanding. In turn, our knowledge of the circumstance or task at hand, improves in an incremental fashion with that situational intelligence. When we shop for new products, consume personalized services, provide feedback about our experience, support customer requirements, or make critical decisions to address any health issues – we seek context. We seek the additional knowledge beyond the facts and information in front of us.
Context Is Always Evolving: What To Chase?
Why does context always evolve? Or, does it? Data doesn’t have an end-point any more. It flows through various steps of a decision making process and morphs in to a new life. As various operators consume and operate on this data at different stages, they add new context. Both the data and metadata, data with the context, in this fashion are evolving continuously. If we consider the world of big data of social medial, web log files, or machine generated information of today, we witness the multiplication of data at a speed that’s becoming very difficult to keep up with. May be, instead of chasing after the raw information in this world of big data and struggling to find the needle in a haystack, we probably would be better served by aligning our major efforts around metadata. What path does metadata take as it continues to move through? Who changes it and why? Which events are responsible for it? Can we capture that information and provide that to decision makers at the time of their needs?
Technology advancements around big data, no SQL movement, modern data warehousing initiatives, and even the advanced predictive analytics fields focused primarily on the content side – what data could tell us directly. Few have endeavored to go after the more interesting part – to build not only a data lake – but a reservoir of sorts, made of metadata that can store, process, and make available the purpose of the data and connect so lucidly with the meaning. Metadata that provides us with the rich context to suggest how something similar might have been used earlier, results accomplished, data parts that might be inherently related to each other, and how changes in one area affects other parts.
In this world of metadata, context is king and quality of our decisions is judged based on improvements made that can be measured. Every step of the way, our efforts and initiatives are connected through the rigor of better decision quality. Is your organization ready for such quality decisions? Can you imagine having a visual map of all of your data at your fingertips that can clearly paint a picture of relationships among data components and how each affects the final decision? If you did have such capability, how do you think your decision quality might improve? Live Data Map is such an initiative that is aimed at amassing world’s first metadata reservoir and connecting with an intelligent data lake that Informatica is currently working on – so that we have sanity eventually! Content and context side by side – body and soul – will bring the storyline through data to life. Goal would be to enable us with right support so we can make the most trusted and high quality decisions. Are you ready to embark on this journey of being decision-ready?