Why ‘Usefulness’ of Big Data Triumphs Over Availability?

Treat Big Data as Scarce Resource

Big data Analytics

Discipline of analytics is really about telling a story through the data – stitching together pieces of data from different sources, uncovering patterns – both obvious and hidden ones, and narrating a story in a way that’s easy to understand.  The journey of analytics has evolved in to the realm of self-discovery and what-if scenario analysis from the more basic slicing and dicing of data along different dimensions.  The distance between what’s interesting and what’s useful is gradually shrinking.  Much of that’s possible due to advancement of both storage technology and data management capabilities, linked to the abilities to process data in near-real time or real-time.  But much more is needed to render the full value spectrum of usefulness of data.

All Data Created Equal?

The movement around Big Data and associated technology would almost make us assume that data is ubiquitous, hence easily affordable and almost equally useful.  Cheaper per unit cost of data storage has driven this mindset probably more than anything else.  Not all data is created equal.  As such, not all data is equally useful.  While analytics tools try to address this gap by letting users discover and consume data in a multi-dimensional way, they often fall short of connecting the content of the data to the context, where truly the usefulness bit comes in.

Context of the data is grounded in the meaning and usage of the data.  It’s the environment that provides data with its purpose and defines its role as part of a business process or decision making step.  Usefulness of data, whether a single component, or as a collection, increases when context sensitivity matches trust, relevance, and timeliness of this data with the job at hand.  Such confidence in data improves not only its half-life but also how much it contributes down-stream.  Incremental awareness of what a data component contributes to overall story is the measure of usefulness of it in the bigger context.  So, the story unfolds one step at a time, enriched by the context and meaning that are associated with it.  A recent CMO survey by the AMA and Duke’s Fuqua School of Business found that the number of marketing projects using marketing analytics to drive decisions decreased from an already low 37% in 2012 to even below 30% a year later.  Could it be because most of the data isn’t as useful as once it was thought, despite of the fact that volume and availability of actual data increased manifold during that one year period?

Does Usefulness Matter in Big Data World?

Analytics performed on such rich context-aware data is going to be more in line with what we might want from useful data parts and not just performing analytics for the sake of it.  This is in fact more evident in the big data world.  Recent Capgemini survey reveals, only 27% of respondents consider their big data initiatives as “successful” with a paltry 8% describing them as “very successful.” Big data issues especially present this conundrum where data is both abundant yet presents a needle in a haystack discovery issue for many. 

Correlation of data components among a myriad set of data doesn’t necessarily add up to the causality or meaningful derivation of underlying behavior: a story-line we so want the data to communicate, may be, sometimes too eagerly.  And we almost would like big data capabilities to provide the panacea to understanding of what causes what – simply because more data is available.  But is that the real key in uncovering true meaning or diagnosing a root cause?  Zoomdata CEO Justin Langseth noted similar concern when he mentioned that when it comes to big data, design is as important as performance. Big or volume factor doesn’t amount to much, as it turns out, if it misses the usefulness part.  So, is it still a good enough reason to treat all data as equal simply because we can process, store, and access it?  Or, much more is needed to understand the relative currency value of each data and how does it contribute to overall understanding of business issues?  Is it time to treat big data and other data components for that matter, not as ubiquitous as experts may want you to believe, rather as scarce objects and treat with care and caution?  Is it time to focus more on the usefulness of each data component more than the means of getting and processing it?