Governed or Self-Service? Centralized or Distributed? Get Ready to Deal With Today’s Vexing Data Dilemmas

We’ve certainly come a long way when it comes to business intelligence (BI) and analytics, Organizations have become very adept and taking data from multiple sources and packaging it into formats for consumption by analytics engines and algorithms. While we have addressed the technical aspects of BI and analytics, the organizational challenges and data dilemmas are another story. For instance, some end-users just want insights delivered to them, while others want to dig deep into their data troves to uncover new business ideas. While some executives want centralized hubs, others see advantages in more dispersed analytics close to the data.

Modern analytics platforms need to be more than technology solutions; they need to be able to assimilate the various cross-currents of today’s organizations. That’s the word from Wayne W. Eckerson with Dr. Carsten Bange, authors of a recent report on the forces and technologies that are converging to comprise the modern analytics platform. The modern analytics platform is “more than just a collection of tools, functions, and features,” they write– it can almost be equated to the role of a marriage counselor, providing a bridge between partners in organizations.

Here are the four most pressing data dilemmas inherent within today’s organizations, and how modern analytics platforms need to be configured to address those dilemmas, as explored by Eckerson and Bange:

Silver-service versus self-service: There are two main categories of analytics consumers in today’s organizations– casual business users and power users, who often have polar opposite requirements. Casual users will require “silver-service” delivery of analytics, seeking “to consume reports and dashboards that make it quick and easy to monitor their performance against predefined metrics, analyze key trends and anomalies, collaborate with others around the data, and act on the findings.” Power users, on the other hand, “are hired to gather analyze and visualize data to answer new and unexpected questions, rather than use data to monitor operations, they use data to explore new opportunities, analyze root causes, and develop predictive models.”

A modern analytics platform needs to recognize these two worlds and unify them into a single platform, Eckerson and Bange urge. “A modern analytics platform provides both silver-service and self-service capabilities: a silver-service environment so casual users can monitor the business, and a self-service environment so power users can change it.”

Self-service versus governance: There is also the inherent friction that arises between self-service provisioning of analytic services, versus governance and a strong case of data dilemma can be made for both. While governance means all the essentials of a well-run data enterprise– privacy, security, data consistency, metadata, data quality, data integration, compliance, data lifecycle management, this must balance against the independence and agility self-service delivers.

A modern analytics platform needs to bring together these disparate worlds, Eckerson and Bange urge. This calls for a common, single standard platform, as well as governance “encoded in a data catalog that is continuously updated by power users as they search and use data sets and reports.”

Enterprise versus business unit: There is also disparity in terms of what the C-suite may want from the analytics environment versus the desires of folks further down in the trenches. “Enterprises, led by top executives, prefers a single BI platform for all users,” Eckerson and Bange state. “On the other hand, business units find a single tool or platform limiting. They prefer a best-of-breed strategy that allows them to use the tool that best meets their local requirements rather than one foisted on them by a corporate patent.”

Rather than foisting unwanted technology from above, Eckerson and Bange recommend a “land-and-expand” approach to analytics tools and solutions, “where the internal market selects the winner.” This is how standards arise within enterprises today, and “a de facto standard is better than a de jure standard that no one implements.”

Scalability versus performance: Often, when it comes to data analytics, greater scalability leads to reduced performance. “Some tools query all the data, giving users the most up-to-date and comprehensive information,” say Eckerson and Bange. “But these tools are often slow, especially when querying large volumes– terabyte and petabytes of data. On the other hand, some tools provide fast query performance, supporting iterative analysis and decision-making, but they often run against a subset of data that someone must capture, model and store in a local high-performance database in advance.”

While a modern analytics platform needs to support the appropriate workarounds to the performance-versus-scalability dilemma, it also requires a “multi-faceted data architecture that can be tailored to optimize a workload or application.” Such an architecture requires the flexibility either to query source systems directly, or to download data into in-memory caches or databases to avoid further data dilemma.

Eckerson and Bange provide additional words of advice for building a modern analytics platform. Make sure it is “designed from the ground up to run on modern computing platforms: namely, the Web, cloud, and mobile devices. It also uses microservices and an open, feature-rich Application Programming Interface (API) that empowers a community of developers to create an ecosystem of third-party add-ons, extensions, and utilities that enrich the platform well beyond the resources and imagination of the platform vendor.”