Tag Archives: Data Architecture
Building a Data Competence for a Decision Ready Organization
There has been a lot of talk about “competing on analytics.” And this year, for the third year in a row BI/Analytics is the top spending priority for CIOs according to Gartner. Yet, the fact is that about half of all analytics projects do not deliver the expected results on time and on budget. That doesn’t mean that the projects don’t show value eventually, but it’s harder and takes longer than most people think.
To compete on analytics is to establish a company goal to deliver actionable business insights faster and better than anybody in your industry – and possibly competitors who may be looking to jump industry boundaries as Google and Apple have already done several times.
This requires a competence in analytics and a competence in data management, which is the focus of this blog. As an analytics manager at a healthcare company told me this week, “We suffer from beautiful reports built on crap data.” Most companies do not yet established standard people, processes and technology for data management. This is one of the last functional areas in most organizations where this is still true. Sales, Marketing, and Finance standardized years ago. It is only in the area of data management, which is shared by business and IT, that there is no real standardization. The result is unconnected silos of data, long IT backlogs for data-related requests, and a process that is literally getting slower by the day as it gets overwhelmed by data volume and data complexity.
Analytics Use Cases and Data Requirements
It is worthwhile to think of the different broad use cases for analytics within an organization and what that means for data requirements.
- Strategic Insights are the high level decisions a company must make. Better performing organizations are moving from “gut feel” to data-driven decision making. The data for these large decisions needs to be as perfect as possible since the business costs of getting it wrong can be enormous.
- Operational Insights require quick decisions to react to on-the-ground conditions. Here, the organization might be willing to sacrifice some data quality in order to deliver quick results. There is a speed versus expected benefit tradeoff to consider.
- Analytics Innovation is the process of asking questions that were often never possible or economic to even ask before. Often, the first step is to see if there is any value in the question or hypothesis. Here the data does not have to be perfect. Often approximated data is “good enough” to test whether a question is worth pursuing further. Some data scientists refer to this as “fail fast and move on quickly.”
The point here is that there is a tradeoff between speed of data delivery and the quality of the data that it is based on. Managers do not want to be making decisions based on bad data, and analysts do not want to spend a high percentage of their time just defending the data.
The Need for Speed in Business Insight Delivery
We are moving from historical to predictive and proscriptive analytics. Practically everybody has historical analysis, so while useful, it is not a market differentiator. The biggest competitive payoff will come from the more advanced forms of analytics. The need for speed as a market differentiator is built on the need provide service to customers in realtime and to make decisions faster than competitors. The “half-life” of an analytics insight drops rapidly once competitors gain the same insight.
Here are a couple of quick examples or predictive and proactive analytics:
- Many retailers are looking to identify a customer coming in the door and have a dashboard in front of the customer service representative that will give them a full profile of the customer’s history, products owned, and positive/negative ratings about this product on social media.
- In Sales, predictive analytics is being used today to recommend the “next best step” with a customer or what to upsell to that customer next and how to position it.
- Beyond that, we are seeing and emerging class of applications and smart devices that will proactively recommend an action to users, without being asked, based on realtime conditions.
The data problems
The big problem is that the data internal to an organization was never designed to be discovered, access and shared across the organization. It is typically locked into a specific application and that application’s format requirements. The new opportunity is the explosion of data external to the organization that can potentially enable questions that have never been possible to ask before. The best insights and most differentiating insights will come from data sources across multiple disparate sources. Often these sources are a mix of internal and external data.
Common data challenges for analytics:
- The 2015 Analytics and BI survey by InformationWeek found that the #1 barrier to analytics is data quality. And this does not just mean that that data is in the right format. It must be complete, it must have business meaning and context, it must be fit for purpose, and if joined with another data set, it must be joined correctly.
- The explosion of data volume and complexity.
- More than 50% organizations use is coming from external sources (Gartner). This data is often less-structured, of unknown structure, and may have limited business context as to what the data means exactly.
- The time-value of money. As mentioned earlier, the value of data and insights is eroding at increasing pace.
- Data Discovery: Gartner estimates that the BI tool market is growing at 8% but says that the market could be growing much faster if issues around data discovery and data management were addressed.
Recommendations for the Decision Ready Organization
If you truly want to compete on analytics, you need to first create a competency center around data management. Analytics is a great place to start. First:
- Break down the data & technology silos
- Standardize on data management tools, processes, skills to the extent possible
- Design so that all of your data is immediately discoverable, understandable, and shareable with any application or analytics project that might need it
Pick industry-leading data management tools, or even better, tools that are integrated into a comprehensive data management platform. Make sure that the platform:
- Works with any data
- Works with any BI tool
- Works with any analytics storage technology
- Supports all the analytics use cases: Strategic Decisions, Operational Decisions, and Innovation
- Supports multiple delivery modes: business analyst self-service as well as the more traditional IT delivery of data managed by a formal data governance body.
The past focus on applications has resulted in hard-to-access data silos. New technologies for analytics are causing some organizations to create new data silos in the search for speed for that particular project. If your organization is serious about being a leader in analytics, it is time to put the focus required into leading-edge data management tools and practices to fuel insight delivery.
We are working with organizations such as EMC, and Fidelity that have done this. You don’t have to do it all at once. Start with your next important analytics projects. Build it out the right way. Then expand your competence to the next project.
For more information see:
- Home Hubs from Google, Samsung, and Apple (who did not attend the show but still had a significant impact).
- Home Hub Ecosystems providing interoperability with cars, door locks, and household appliances.
- Autonomous cars, and intelligent cars
- Wearable devices such as smart watches and jewelry.
- Drones that take pictures and intelligently avoid obstacles. …Including people trying to block them. There is a bit of a creepy factor here!
- The next generation of 3D printers.
- And the intelligent baby pacifier. The idea is that it takes the baby’s temperature, but I think the sleeper hit feature on this product is the ability to locate it using GPS and a smart phone. How much money would you pay to get your kid to go to sleep when it is time to do so?
Digital Strategies Are Gaining Momentum
There is no escaping the fact that the vast majority of companies out there have active digital strategies, and not just in the consumer space. The question is: Are you going to be the disruptor or the disruptee? Gartner offered an interesting prediction here:
“By 2017, 60% of global enterprise organizations will execute on at least one revolutionary and currently unimaginable business transformation effort.”
It is clear from looking at CES, that a lot of these products are “experiments” that will ultimately fail. But focusing too much on that fact is to risk overlooking the profound changes taking place that will shake out industries and allow competitors to jump previously impassible barriers to entry.
IDC predicted that the Internet of Things market would be over $7 Trillion by the year 2020. We can all argue about the exact number, but something major is clearly happening here. …And it’s big.
Is Your Organization Ready?
A study by Gartner found that 52% of CEOs and executives say they have a digital strategy. The problem is that 80% of them say that they will “need adaptation and learning to be effective in the new world.” Supporting a new “Internet of Things” or connected device product may require new business models, new business processes, new business partners, new software applications, and require the collection and management of entirely new types of data. Simply standing up a new ERP system or moving to a cloud application will not help your organization to deal with the new business models and data complexity.
Architect’s Call to Action
Now is the time (good New Year’s resolution!) to get proactive on your digital strategy. Your CIO is most likely deeply engaged with her business counterparts to define a digital strategy for the organization. Now is the time to be proactive in terms of recommending the IT architecture that will enable them to deliver on that strategy – and a roadmap to get to the future state architecture.
Key Requirements for a Digital-ready Architecture
Digital strategy and products are all about data, so I am going to be very data-focused here. Here are some of the key requirements:
- First, it must be designed for speed. How fast? Your architecture has to enable IT to move at the speed of business, whatever that requires. Consider the speed at which companies like Google, Amazon and Facebook are making IT changes.
- It has to explicitly directly link the business strategy to the underlying business models, processes, systems and technology.
- Data from any new source, inside or outside your organization, has to be on-boarded quickly and in a way that it is immediately discoverable and available to all IT and business users.
- Ongoing data quality management and Data Governance must be built into the architecture. Point product solutions cannot solve these problems. It has to be pervasive.
- Data security also has to be pervasive for the same reasons.
- It must include business self-service. That is the only way that IT is going to be able to meet the needs of business users and scale to the demands of the changes required by digital strategy.
For a webinar on connecting business strategy to the architecture of business transformation see; Next-Gen Architecture: A “Business First” Approach for Agile Architecture. With John Schmidt of Informatica and Art Caston, founder of Proact.
For next-generation thinking on enterprise data architectures see; Think “Data First” to Drive Business Value
For more on business self-service for data preparation and a free software download.
We are way past the point where the architecture needs to be aligned with business goals and value delivery. That is necessary but no longer sufficient. We are now at the point where architecture needs to be central to the creation of an organization’s strategy process. Not to get hyperbolic, but anything less is risky for your career.
The Challenge: Digitization
I just came back from the MIT Center for Information Systems Research (CISR) research forum. One of the leading topics was digitization and how every business is becoming digitized. To those in the High Tech industry, this may be an “of course” topic, but to most other industries it is a wrenching change. Even those who are comfortable with the idea of digitization risk taking this too lightly.
The fact is that most products and services will have a digital component to them in the near future and an increasing number of products and services will be entirely digital. The fact is that digitization and the technologies that enable it are going to bring about a period of increased disruption. This will mean:
- New competitors. Examples: autonomous cars, sports equipment with embedded sensors that provide feedback, personal assistant fully capable of making decisions and taking action. Gartner is predicting that almost everything over $100 will have a sensor by the turn of the decade.
- New competitors jumping across industry boundaries. Examples: Apple iTunes and Google cars to name a few.
Why Architects Are Important
Architects are in a unique position to not only understand the technology trends driving this disruption, but they also to know how to leverage these trends to drive business value within their organizations. The very best architects are going to be those who are deeply involved in defining the organization strategy, not just figuring out how to implement it.
Evidence of Change
Many architects and CIOs currently report very little interest from upper management in IT. That is about to change, and quickly. At the MIT CISR forum I attended last week, they presented research around this area that is very telling:
- Half of Board of Directors members believe that their board’s ability to oversee the strategic use of IT is “less than effective.”
- 26% of Boards hired consultants to evaluate major projects or the IT unit.
- 60% of Boards want to spend more time on digital issues next year.
- Board members estimate that 32% of their company’s revenues are under threat from digital disruption.
That last bullet is the really interesting piece of research. 32% is a huge impact.
The Role of Data in Digitization
Digitization by its very nature is all about data. The winners in this space will be those that can manage and deliver relevant data the quickest. The question for architects is this: Do you have the architecture and agility to take advantage of the coming disruptions and opportunities? Are you actively advising your organization on how to leverage them? As we have documented in many previous blogs, many organizations are poorly positioned to manage their data as a discoverable and easily sharable asset. This will essential for:
- Delivering business initiatives and showing value faster (agility).
- Enabling business self-service so that IT is not the bottleneck in new analyses and decisions.
All of this requires new thinking around enterprise data architecture. For fresh thinking on this subject see Thinking “Data First” to Drive Business Value.
If you build an IT Architecture, it will be a constant up-hill battle to get business users and executives engaged and take ownership of data governance and data quality. In short you will struggle to maximize the information potential in your enterprise. But if you develop and Enterprise Architecture that starts with a business and operational view, the dynamics change dramatically. To make this point, let’s take a look at a case study from Cisco. (more…)
The first thing I would like to do is dispel a myth that many people believe. That is, being information-enabled or competing with data means analytics or BI. This is only partially true.
Analytics is one of the methods an organization uses to compete on information. For example, with analytics you can analyze buying behavior and leverage the information to better promote products. To truly be information-enabled, an organization must control the information across operational (transaction) systems as well as analytic solutions.
In the world of analytics, most organizations invest a significant amount of time and effort cleansing data from operational systems before it moves into a data warehouse. Thus, enabling higher quality analytics where reporting can be performed. However, the “cleansing” effort is rarely reflected back into the source/operational systems. This plays into the unwritten rule of IT that bad data doubles at the rate of good data. (more…)
When I speak with most senior executives at companies, they highlight the “value gap” in information. According to the PriceWaterhouseCoopers 12th Annual Global CEO Survey in January 2009:
“…CEOs still see major gaps in the information they need to survive the next 12 months and make decisions about the long-term success of their businesses. CEOs believe that agility, customer service, talent, management and reputation are the four most important factors in long-term competitive advantage. Not surprisingly, most also believe that data about their customers (94%), brand (91%) and employees (88%) are important or critical to long-term decision-making. However, strikingly low percentages of CEOs say they have comprehensive information in these and other critical areas that contribute to organisational agility. Just 21% have comprehensive information about the needs and preferences of customers and clients. Less than one third feel they have all the information they need about reputation (31%) and the views and needs of employees (30%).”
I would not expect the results of the PWC survey to be a surprise to anyone in IT. With that said however, why aren’t IT professionals surprised? If they truly know this is the reality for companies, why hasn’t the value gap in information been solved?
Here are my views on this: (more…)