Tag Archives: competitive advantage
Building a Data Competence for a Decision Ready Organization
There has been a lot of talk about “competing on analytics.” And this year, for the third year in a row BI/Analytics is the top spending priority for CIOs according to Gartner. Yet, the fact is that about half of all analytics projects do not deliver the expected results on time and on budget. That doesn’t mean that the projects don’t show value eventually, but it’s harder and takes longer than most people think.
To compete on analytics is to establish a company goal to deliver actionable business insights faster and better than anybody in your industry – and possibly competitors who may be looking to jump industry boundaries as Google and Apple have already done several times.
This requires a competence in analytics and a competence in data management, which is the focus of this blog. As an analytics manager at a healthcare company told me this week, “We suffer from beautiful reports built on crap data.” Most companies do not yet established standard people, processes and technology for data management. This is one of the last functional areas in most organizations where this is still true. Sales, Marketing, and Finance standardized years ago. It is only in the area of data management, which is shared by business and IT, that there is no real standardization. The result is unconnected silos of data, long IT backlogs for data-related requests, and a process that is literally getting slower by the day as it gets overwhelmed by data volume and data complexity.
Analytics Use Cases and Data Requirements
It is worthwhile to think of the different broad use cases for analytics within an organization and what that means for data requirements.
- Strategic Insights are the high level decisions a company must make. Better performing organizations are moving from “gut feel” to data-driven decision making. The data for these large decisions needs to be as perfect as possible since the business costs of getting it wrong can be enormous.
- Operational Insights require quick decisions to react to on-the-ground conditions. Here, the organization might be willing to sacrifice some data quality in order to deliver quick results. There is a speed versus expected benefit tradeoff to consider.
- Analytics Innovation is the process of asking questions that were often never possible or economic to even ask before. Often, the first step is to see if there is any value in the question or hypothesis. Here the data does not have to be perfect. Often approximated data is “good enough” to test whether a question is worth pursuing further. Some data scientists refer to this as “fail fast and move on quickly.”
The point here is that there is a tradeoff between speed of data delivery and the quality of the data that it is based on. Managers do not want to be making decisions based on bad data, and analysts do not want to spend a high percentage of their time just defending the data.
The Need for Speed in Business Insight Delivery
We are moving from historical to predictive and proscriptive analytics. Practically everybody has historical analysis, so while useful, it is not a market differentiator. The biggest competitive payoff will come from the more advanced forms of analytics. The need for speed as a market differentiator is built on the need provide service to customers in realtime and to make decisions faster than competitors. The “half-life” of an analytics insight drops rapidly once competitors gain the same insight.
Here are a couple of quick examples or predictive and proactive analytics:
- Many retailers are looking to identify a customer coming in the door and have a dashboard in front of the customer service representative that will give them a full profile of the customer’s history, products owned, and positive/negative ratings about this product on social media.
- In Sales, predictive analytics is being used today to recommend the “next best step” with a customer or what to upsell to that customer next and how to position it.
- Beyond that, we are seeing and emerging class of applications and smart devices that will proactively recommend an action to users, without being asked, based on realtime conditions.
The data problems
The big problem is that the data internal to an organization was never designed to be discovered, access and shared across the organization. It is typically locked into a specific application and that application’s format requirements. The new opportunity is the explosion of data external to the organization that can potentially enable questions that have never been possible to ask before. The best insights and most differentiating insights will come from data sources across multiple disparate sources. Often these sources are a mix of internal and external data.
Common data challenges for analytics:
- The 2015 Analytics and BI survey by InformationWeek found that the #1 barrier to analytics is data quality. And this does not just mean that that data is in the right format. It must be complete, it must have business meaning and context, it must be fit for purpose, and if joined with another data set, it must be joined correctly.
- The explosion of data volume and complexity.
- More than 50% organizations use is coming from external sources (Gartner). This data is often less-structured, of unknown structure, and may have limited business context as to what the data means exactly.
- The time-value of money. As mentioned earlier, the value of data and insights is eroding at increasing pace.
- Data Discovery: Gartner estimates that the BI tool market is growing at 8% but says that the market could be growing much faster if issues around data discovery and data management were addressed.
Recommendations for the Decision Ready Organization
If you truly want to compete on analytics, you need to first create a competency center around data management. Analytics is a great place to start. First:
- Break down the data & technology silos
- Standardize on data management tools, processes, skills to the extent possible
- Design so that all of your data is immediately discoverable, understandable, and shareable with any application or analytics project that might need it
Pick industry-leading data management tools, or even better, tools that are integrated into a comprehensive data management platform. Make sure that the platform:
- Works with any data
- Works with any BI tool
- Works with any analytics storage technology
- Supports all the analytics use cases: Strategic Decisions, Operational Decisions, and Innovation
- Supports multiple delivery modes: business analyst self-service as well as the more traditional IT delivery of data managed by a formal data governance body.
The past focus on applications has resulted in hard-to-access data silos. New technologies for analytics are causing some organizations to create new data silos in the search for speed for that particular project. If your organization is serious about being a leader in analytics, it is time to put the focus required into leading-edge data management tools and practices to fuel insight delivery.
We are working with organizations such as EMC, and Fidelity that have done this. You don’t have to do it all at once. Start with your next important analytics projects. Build it out the right way. Then expand your competence to the next project.
For more information see:
Just last week, I visited a client for whom I had been consulting on-and-off for several years. On the meeting room wall, I saw their Enterprise Architecture portfolio, beautiful graphically designed and printed on a giant sheet of paper. My host proudly informed me how much she enjoyed putting that diagram together in 2009.
I jokingly reminded her of the famous notion of “art for art’s sake”; which is an appropriate phrase to describe what many architects are doing when populating frameworks. Indeed, when we refer to Enterprise Architecture, we must remember that the term ‘architecture’ is, itself, a metaphor.
In a tough economy, when competition is increasingly global and marketplaces are shifting, this ability to make tough decisions is going to be essential. Opportunities to save costs are going to be really valued, and architecture invariably helps companies save money. The ability to reuse, and thus rapidly seize the next related business opportunity, is also going to be highly valued.
The thing you have to be careful of is that if you see your markets disappearing, if your product is outdated, or your whole industry is redefining itself, as we have seen in things like media, you have to be ready to innovate. Architecture can restrict your innovative gene, by saying, “Wait, wait, wait. We want to slow down. We want to do things on our platform.” That can be very dangerous, if you are really facing disruptive technology or market changes.
Albert Camus wrote a famous essay exploring the Sisyphus myth called “The Myth of Sisyphus,” where he reinterpreted the central theme of the myth. Similarly, we need to challenge the myths of Enterprise Architecture and enterprise system/solution architecture in general – not meekly accept them.
IEEE says, “A key premise of this metaphor is that important decisions may be made early in system development in a manner similar to the early decision-making found in the development of civil architecture projects.”
Keep asking yourself, “When is what we built that’s stable actually constraining us too much? When is it preventing important innovation?” For many architects, that’s going to be tough, because you start to love the architecture, the standards, and the discipline. You love what you’ve created, but if it isn’t right for the market you’re facing, you have to be ready to let it go and go seize the next opportunity.
The central message is as follows: ‘documenting’ architecture in various layers of abstraction for the purposes of ‘completeness’ is plainly ridiculous. This is especially true when the effort to produce the artifacts takes such an amount of time as to make the whole collection obsolete on completion.
If you haven’t updated your B2B integration capabilities in the past five years, are you at risk of being left behind? This is the age of superior customer experience and rapid time-to-value so speedy customer on-boarding and support of specialized integration services means the difference between winning and losing business. A health check starts with asking some simple questions: (more…)
In a recent post: Remove the Restrictor Plate with High Performance Load Balancing, my colleague Jeff Brokaw compared the high performance architecture of Informatica Ultra Messaging to the removal of the carburetor restrictor plate on a NASCAR racing engine to increase airflow and speed. Ultra Messaging has removed the “restrictor plate” in that it provides direct peer-to-peer communication between applications with no intermediary brokers – thereby delivering extremely high and sustained throughput rates at low latencies. (more…)
I recently returned from China and Hong Kong after having met with several CIOs, media and analysts, as well as delivering keynotes focused on customer centricity. When I return to the US after traveling, I’m often asked about the state of IT in the geography I was just in. I’ve been to both China and Hong Kong several times over the past few years, and from my perspective, IT is maturing at a very rapid pace in that region.
During prior trips to Asia, it felt like the old days of data processing. I would speak with senior IT leaders and they were more concerned with the “blocking and tackling” of IT, and not looking at how IT can provide a strategic competitive advantage. Specifically in China, IT leadership was comfortable scaling by applying people to the problem rather than using commercial software. (more…)