Is the Big Data Analytics Market Growing or Slowing?

Big Data Analytics Market Growth
Big Data Analytics Market Growth
At first glance, recent predictions for the big data analytics market appear contradictory. According to IDC’s Worldwide Semiannual Big Data and Analytics Spending Guide, the big data and business analytics market will grow by 13.1 percent to $130 billion by 2020.

But in a news release citing a recent online survey among Gartner Research Circle, Gartner says: “The survey revealed that 48 percent of companies have invested in big data in 2016, up 3 percent from 2015. However, those who plan to invest in big data within the next two years fell from 31 to 25 percent in 2016.[1]

So, which is it? Is the big data analytics market growing? My take on this is that the amount of growth you forecast depends on what you are measuring. We are definitely reaching a slow-down period for hand-crafted, one-off, “science experiment” types of big data projects.

On the other hand, the need to understand customers better, to use analytics to deliver better business outcomes, and to either be a digital disruptor or head off digital disruption has never been greater. It looks like the broader analytics spending will continue to grow and along with it, the Big Data projects that are connected to measurable business outcomes. But the trend will probably be away from stand-alone Big Data projects and more towards Big Data as an important part of a major business initiative.

In particular, we are seeing a lot of investment in customer-centricity types of projects and investment in marketing analytics. What is particularly interesting to me is that, based on my unscientific observation, some of the organizations who are spending significantly on data management for analytics projects fall into two categories.

  • Those with less-than-stellar brand reputation at present. This is a chance for them to fix their customer satisfaction issues and fend off any competitors who might think of moving into their space.
  • Those with fairly undifferentiated product/service offerings. If your product is pretty much similar to what your competitors offer, it makes a lot of sense to compete on better customer experience, or targeted and personalized outreach to your customers (i.e., a “market of one”).

What is it going to take to be successful with these initiatives? Data management is key. You need to be able to manage your data across your on-premise systems, cloud systems, and big data systems. And those that have a single data management platform that work across all three of these will have an edge. Further, if you are doing big data analytics, just throwing all your data in a Hadoop cluster and hoping for insights to emerge is a recipe for failure. The fundamental truths still apply. Big data gives you the option of managing your data after it is ingested into Hadoop, but data still needs to be managed, and now more than ever if it is to be used for business processes or decisions. It will pay to have a plan to be able to understand, discover, transform, and manage your data as well as to be able to attach tags and business metadata to promote understanding and correct use of the data. Finally, you will also need a plan to be able to operationalize your data management steps. A genius-coder might do great work, but that work won’t be very share-able or repeatable. Winning in the data and analytics will be about delivering trusted data at the speed of business. It will pay to design in code-reuse, repeatability, automation, and intelligence into your data management operations.

[1] Gartner, “Survey Analysis: Big Data Investments Begin Tapering in 2016.” Nick Heudecker. 19 September 2016.