I ran across an interesting post by Dr. Michael Wu, principal scientist of analytics at Lithium, a social media strategy firm, about the value of Big Data, suggesting that too many people think the data itself is a valuable commodity. However, it is not the same as “information.”
As he describes it, “the promise of Big Data is that one could glean lots of information and gain many valuable insights. However, people often don’t realize that data and information are not the same. Even if you are able to extract information from your Big Data, not all of it will be insightful and valuable.”
Precisely. Just as gold prospectors of old had to sift through thousands of gallons of stream water to extract a few microscopic nuggets, so do Big Data analysts. The information that really matters is probably just a few microscopic nuggets, buried somewhere within mountains and mountains of records and files.
Adding more fuel to this point of view is Jer Thorp, co-founder of the Office For Creative Research, a data research group. In a new piece at the Harvard Business Review blogsite, he refutes the notion that Big Data is the “New Oil,” a line of thinking that suggests that the huge gushers of data organizations are experiencing will translate into huge wealth. To take this analogy a step further, to be of value, oil needs to be transported and refined and distributed to where it is needed. Otherwise, it’s a pool of sludge.
When it comes to data, more is not necessarily better, Wu points out in his post at TechCrunch. Think about airport surveillance video logs – which are really huge files in and of themselves. This is big, Big Data – “there are hundreds and probably thousands of security cameras all over the airport,” he says. “What happens when you double the number of camera installations? In terms of data volume, you will again get about 2x the data. But will you get 2x the information? Probably not.”
Again, it’s finding those few little gold nuggets in the midst of a sea of sludge. “Because the amount of valuable insights we can derive from big data is so very tiny, we need to collect even more data and use more powerful analytics to increase our chance of finding them, Wu points out. This is where, actually, Big Data can help – the more data that is available, the greater the chances of finding something.
So what separates the gold from the piles and piles of stones? Wu identifies three characteristics of what makes information valuable;
Interpretability. “Since big data contains so much unstructured data and different media as well as data types, there is actually a substantial amount of data and information that is not interpretable. Insights must lie within the interpretable parts of the extractable information.”
Relevance. “Relevant information is also known as the signal, so irrelevant information is often referred to as noise. But relevance is subjective. Information that is relevant to me may be completely irrelevant to you, and vice versa. Furthermore, relevance is not only subjective, it is also contextual. What is relevant to a person may change from one context to another.”
Novelty. Information must “provide some new knowledge that you don’t already have.”
For his part, Thorp takes a more expansive view of the value of Big Data, suggesting it is a type of resource that we really haven’t encountered before, and it may ultimately advance many aspects of our businesses and society. A first step in the right direction, he points out, is to understand and experience data ownership. “While everyone in our society is producing vast quantities of data, individuals rarely see or interact with any of it. When people are given tools to store, visualize, and explore their own data, they gain an understanding of the worth and utility of this information.”
There actually isn’t anything new about Big Data in terms of volume – a decade ago, a terabyte was Big Data. What has changed is how many parts of our organizations and day-to-day operations are now digitized, leaving trails of data all over the place. Business and technology leaders are just starting to understand where the real wealth lies in all these bits and bytes. It’s a matter of digging down and finding the ones that really matter.