Leveraging Big Data to Get Business Value from Across Silos

Data-Silos 2
Leveraging Big Data to Get Business Value from Across Silos

I’ve been plowing through a lot of great reading about data management strategy this month, and following recent discussions of a McKinsey article and some new IDG research, I dove into a Forrester examination of how big data can really help a company’s bottom line.

The report, “Big Data Fabric Drives Innovation and Growth[1],” has a lot of great insights, but the one finding I want to cover here (before you go off to read the entire report for yourself) is this Forrester observation: “Today, most big data deployments are built in silos largely to address specific business needs, such as collecting sensor data for smart metering, web clickstream data for customer analytics, and geolocation data for consumer personalization.”

Such siloing can be okay, in the short term. The important thing is to quickly show measurable results that have clear business value to the organization. That’s step one, and without it, you never make it to step two. But at the same time, it’s also critical to have a defined future state data management architecture in place so that as your big data project matures, it fits into the overall data management architecture and can leverage data to and from the overall data management environment.

Faster and faster bicycles

The potential business advantages from harnessing big data are pretty widely understood. Both business and IT leaders generally realize that the greatest analytic value will come from relating data sets from disparate sources and finding interesting insights that can drive the business to be better, faster, and more engaging. But we haven’t reached that goal yet. Not by a long shot. Many organizations are still struggling to get value from their data projects.

And this isn’t a simple, static challenge that we’ll all master in time, the way a child eventually learns to ride a bike. The data volume, types, speed, and complexity of data are only increasing, so by the time some companies learn to balance on two wheels, they’ll find that their bicycle has morphed into a fighter jet. So it’s not a question of “Just master the complexity of big data, and you’re done.” You have to plan for continuous technology change, proving the value of new tools and technologies to the business and incorporating them seamlessly into an overall data management environment that has been optimized to accommodate change.

Your data sources and analytics tools and engines will also change at a faster pace. As organizations look to leverage digital transformation and analytics for competitive advantage, there will be a significant upgrading of capabilities. In my previous blogs, referenced earlier, we have discussed some of impending changes:

This is important because big data analytics will be an important part of your future success. Your data management architecture will need to enable to your leverage the relative strengths of the current data warehouse/BI environment, big data analytics (including NoSQL), and cloud analytics.

A Data management platform

The Forrester report states that, “Without a big data fabric, firms will spend excessive time and effort to ingest, integrate, transform, curate, secure, and deliver big data insights to business stakeholders.” Your infrastructure and tools must weave together to leave no gap in how you leverage your data. Enterprise architects should approach the big data fabric as a way to accelerate big data initiatives, drive bottom-line value, and respond faster to competitive threats. “Big data” management can’t be separate from structured operational data. One platform must manage data across any scale, source and format, and work equally well with data that’s on premises or in the cloud. Look for a data management platform that has integrated capabilities for data integration (with unstructured data, data quality, data governance and data security.)

Your data management platform and architecture needs to be designed to support key requirements of your analytics strategy as it matures, for example:

  • Data warehouse/BI reporting and diagnostics
  • Real time management and operational decision support
  • Support for business data self-service
  • Support for analytics innovation and rapid experimentation
  • A plan for how to operationalize interesting innovations and bring them in from “the laboratory” to “the factory”—with more strict control over data quality, context, meaning, compliance and security.
  • A plan to leverage the best of available technology (Hadoop, NoSQL, Columnar, cloud analytics) with minimal or no disruption.

Failure to do this will just create new silos of data that will ultimately slow down the organization’s ability to deliver business value.

If you’re just getting started, or want to give your existing vision a careful second look, take a look at our workbook, “Laying the Foundations for Next-Generation Analytics,” which starts with a great checklist for envisioning your general data architecture (and then dives into making individual projects successful).

And, of course, take a look at the full Forrester report. Download “Big Data Fabric Drives Innovation and Growth.”

[1] “Big Data Fabric Drives Innovation and Growth,” Forrester Research, Inc. March 8, 2016