Analytics Results May Reflect the Wrong Definition of Success


Everyone wants their businesses to become data-driven enterprises. The potential rewards are to compelling to ignore – including improved visibility into customers, markets and operations, the ability to predict what’s coming, and new paths to innovation.

However, as with all good things, there’s a risky side as well – namely, that business leaders may drink too much of the elixir, betting their businesses on analytics that may or may not be accurate, or biased.

Accordingly, one highly regarded big data expert is warning that “the era of blind faith in big data must end,” because decision makers are starting to rely on data-driven insights without even questioning the logic behind them.

Cathy O’Neil, a mathematician and data scientist, speaking at a recent TED Talk, says the algorithms that we depend on to score everything from credit scores to customer satisfaction levels could potentially be wrong – and nobody would even be aware of it.



“Algorithms are opinions embedded in code,” O’Neil explains. “It’s really different from what you think most people think of algorithms. They think algorithms are objective and true and scientific. That’s a marketing trick.”

Too much trust is placed in algorithms, based on the assumption they are impartial and well-constructed, she continues. “It’s also a marketing trick to intimidate you with algorithms, to make you trust and fear algorithms because you trust and fear mathematics.” Worse yet, she says, many algorithms may be concocted with secret formulas not open to critical scrutiny.

An algorithm, for example, may be based on one person’s definition of success – subject to enormous, yet hidden, bias. Many algorithms represent a form of data laundering,” as O’Neil brands it. Another terms she applies to algorithms: “weapons of math destruction.”

“Algorithms don’t make things fair. They repeat our past practices, our patterns. They automate the status quo,” O’Neil says. Shel has advice for data analysts, scientists and developers seeking to remove bias and erroneous results from algorithms:

  • Perform regular data integrity checks
  • Rethink the definition of success and audit
  • Consider accuracy and the errors of algorithms
  • Consider the long-term effects of algorithms, the feedback loops that are engendering.
  • The bottom line is that algorithms are not sacred vessels not to be questioned. They should constantly be reviewed, and the logic behind them constantly questioned