Tag Archives: Advanced Analytics
Building a Data Competence for a Decision Ready Organization
There has been a lot of talk about “competing on analytics.” And this year, for the third year in a row BI/Analytics is the top spending priority for CIOs according to Gartner. Yet, the fact is that about half of all analytics projects do not deliver the expected results on time and on budget. That doesn’t mean that the projects don’t show value eventually, but it’s harder and takes longer than most people think.
To compete on analytics is to establish a company goal to deliver actionable business insights faster and better than anybody in your industry – and possibly competitors who may be looking to jump industry boundaries as Google and Apple have already done several times.
This requires a competence in analytics and a competence in data management, which is the focus of this blog. As an analytics manager at a healthcare company told me this week, “We suffer from beautiful reports built on crap data.” Most companies do not yet established standard people, processes and technology for data management. This is one of the last functional areas in most organizations where this is still true. Sales, Marketing, and Finance standardized years ago. It is only in the area of data management, which is shared by business and IT, that there is no real standardization. The result is unconnected silos of data, long IT backlogs for data-related requests, and a process that is literally getting slower by the day as it gets overwhelmed by data volume and data complexity.
Analytics Use Cases and Data Requirements
It is worthwhile to think of the different broad use cases for analytics within an organization and what that means for data requirements.
- Strategic Insights are the high level decisions a company must make. Better performing organizations are moving from “gut feel” to data-driven decision making. The data for these large decisions needs to be as perfect as possible since the business costs of getting it wrong can be enormous.
- Operational Insights require quick decisions to react to on-the-ground conditions. Here, the organization might be willing to sacrifice some data quality in order to deliver quick results. There is a speed versus expected benefit tradeoff to consider.
- Analytics Innovation is the process of asking questions that were often never possible or economic to even ask before. Often, the first step is to see if there is any value in the question or hypothesis. Here the data does not have to be perfect. Often approximated data is “good enough” to test whether a question is worth pursuing further. Some data scientists refer to this as “fail fast and move on quickly.”
The point here is that there is a tradeoff between speed of data delivery and the quality of the data that it is based on. Managers do not want to be making decisions based on bad data, and analysts do not want to spend a high percentage of their time just defending the data.
The Need for Speed in Business Insight Delivery
We are moving from historical to predictive and proscriptive analytics. Practically everybody has historical analysis, so while useful, it is not a market differentiator. The biggest competitive payoff will come from the more advanced forms of analytics. The need for speed as a market differentiator is built on the need provide service to customers in realtime and to make decisions faster than competitors. The “half-life” of an analytics insight drops rapidly once competitors gain the same insight.
Here are a couple of quick examples or predictive and proactive analytics:
- Many retailers are looking to identify a customer coming in the door and have a dashboard in front of the customer service representative that will give them a full profile of the customer’s history, products owned, and positive/negative ratings about this product on social media.
- In Sales, predictive analytics is being used today to recommend the “next best step” with a customer or what to upsell to that customer next and how to position it.
- Beyond that, we are seeing and emerging class of applications and smart devices that will proactively recommend an action to users, without being asked, based on realtime conditions.
The data problems
The big problem is that the data internal to an organization was never designed to be discovered, access and shared across the organization. It is typically locked into a specific application and that application’s format requirements. The new opportunity is the explosion of data external to the organization that can potentially enable questions that have never been possible to ask before. The best insights and most differentiating insights will come from data sources across multiple disparate sources. Often these sources are a mix of internal and external data.
Common data challenges for analytics:
- The 2015 Analytics and BI survey by InformationWeek found that the #1 barrier to analytics is data quality. And this does not just mean that that data is in the right format. It must be complete, it must have business meaning and context, it must be fit for purpose, and if joined with another data set, it must be joined correctly.
- The explosion of data volume and complexity.
- More than 50% organizations use is coming from external sources (Gartner). This data is often less-structured, of unknown structure, and may have limited business context as to what the data means exactly.
- The time-value of money. As mentioned earlier, the value of data and insights is eroding at increasing pace.
- Data Discovery: Gartner estimates that the BI tool market is growing at 8% but says that the market could be growing much faster if issues around data discovery and data management were addressed.
Recommendations for the Decision Ready Organization
If you truly want to compete on analytics, you need to first create a competency center around data management. Analytics is a great place to start. First:
- Break down the data & technology silos
- Standardize on data management tools, processes, skills to the extent possible
- Design so that all of your data is immediately discoverable, understandable, and shareable with any application or analytics project that might need it
Pick industry-leading data management tools, or even better, tools that are integrated into a comprehensive data management platform. Make sure that the platform:
- Works with any data
- Works with any BI tool
- Works with any analytics storage technology
- Supports all the analytics use cases: Strategic Decisions, Operational Decisions, and Innovation
- Supports multiple delivery modes: business analyst self-service as well as the more traditional IT delivery of data managed by a formal data governance body.
The past focus on applications has resulted in hard-to-access data silos. New technologies for analytics are causing some organizations to create new data silos in the search for speed for that particular project. If your organization is serious about being a leader in analytics, it is time to put the focus required into leading-edge data management tools and practices to fuel insight delivery.
We are working with organizations such as EMC, and Fidelity that have done this. You don’t have to do it all at once. Start with your next important analytics projects. Build it out the right way. Then expand your competence to the next project.
For more information see:
Let’s face it, building a Data Governance program is no overnight task. As one CDO puts it: ”data governance is a marathon, not a sprint”. Why? Because data governance is a complex business function that encompasses technology, people and process, all of which have to work together effectively to ensure the success of the initiative. Because of the scope of the program, Data Governance often calls for participants from different business units within an organization, and it can be disruptive at first.
Why bother then? Given that data governance is complex, disruptive, and could potentially introduce additional cost to a company? Well, the drivers for data governance can vary for different organizations. Let’s take a close look at some of the motivations behind data governance program.
For companies in heavily regulated industries, establishing a formal data governance program is a mandate. When a company is not compliant, consequences can be severe. Penalties could include hefty fines, brand damage, loss in revenue, and even potential jail time for the person who is held accountable for being noncompliance. In order to meet the on-going regulatory requirements, adhere to data security policies and standards, companies need to rely on clean, connected and trusted data to enable transparency, auditability in their reporting to meet mandatory requirements and answer critical questions from auditors. Without a dedicated data governance program in place, the compliance initiative could become an on-going nightmare for companies in the regulated industry.
A data governance program can also be established to support customer centricity initiative. To make effective cross-sells and ups-sells to your customers and grow your business, you need clear visibility into customer purchasing behaviors across multiple shopping channels and touch points. Customer’s shopping behaviors and their attributes are captured by the data, therefore, to gain thorough understanding of your customers and boost your sales, a holistic Data Governance program is essential.
Other reasons for companies to start a data governance program include improving efficiency and reducing operational cost, supporting better analytics and driving more innovations. As long as it’s a business critical area and data is at the core of the process, and the business case is loud and sound, then there is a compelling reason for launching a data governance program.
Now that we have identified the drivers for data governance, how do we start? This rather loaded question really gets into the details of the implementation. A few critical elements come to consideration including: identifying and establishing various task forces such as steering committee, data governance team and business sponsors; identifying roles and responsibilities for the stakeholders involved in the program; defining metrics for tracking the results. And soon you will find that on top of everything, communications, communications and more communications is probably the most important tactic of all for driving the initial success of the program.
A rule of thumb? Start small, take one-step at a time and focus on producing something tangible.
Sounds easy, right? Well, let’s hear what the real-world practitioners have to say. Join us at this Informatica webinar to hear Michael Wodzinski, Director of Information Architecture, Lisa Bemis, Director of Master Data, Fabian Torres, Director of Project Management from Houghton Mifflin Harcourt, global leader in publishing, as well as David Lyle, VP of product strategy from Informatica to discuss how to implement a successful data governance practice that brings business impact to an enterprise organization.
If you are currently kicking the tires on setting up data governance practice in your organization, I’d like to invite you to visit a member-only website dedicated to Data Governance: http://governyourdata.com/. This site currently has over 1,000 members and is designed to foster open communications on everything data governance. There you will find conversations on best practices, methodologies, frame works, tools and metrics. I would also encourage you to take a data governance maturity assessment to see where you currently stand on the data governance maturity curve, and compare the result against industry benchmark. More than 200 members have taken the assessment to gain better understanding of their current data governance program, so why not give it a shot?
Data Governance is a journey, likely a never-ending one. We wish you best of the luck on this effort and a joyful ride! We love to hear your stories.
I have two kids. In school. They generate a remarkable amount of paper. From math worksheets, permission slips, book reports (now called reading responses) to newsletters from the school. That’s a lot of paper. All of it is presented in different forms with different results – the math worksheets tell me how my child is doing in math, the permission slips tell me when my kids will be leaving school property and the book reports tell me what kind of books my child is interested in reading. I need to put the math worksheet information into a storage space so I can figure out how to prop up my kid if needed on the basic geometry constructs. The dates that permission slips are covering need to go into the calendar. The book reports can be used at the library to choose the next book.
We are facing a similar problem (albeit on a MUCH larger scale) in the insurance market. We are getting data from clinicians. Many of you are developing and deploying mobile applications to help patients manage their care, locate providers and improve their health. You may capture licensing data to assist pharmaceutical companies identify patients for inclusion in clinical trials. You have advanced analytics systems for fraud detection and to check the accuracy and consistency of claims. Possibly you are at the point of near real-time claim authorization.
The amount of data generated in our world is expected to increase significantly in the coming years. There are an estimated 50 petabytes of data in the Healthcare realm, which is predicted to grow by a factor of 50 to 25,000 petabytes by 2020. Healthcare payers already store and analyze some of this data. However in order to capture, integrate and interrogate large information sets, the scope of the payer information will have to increase significantly to include provider data, social data, government data, pharmaceutical and medical product manufacturers data, and information aggregator data.
Right now – you probably depend on a traditional data warehouse model and structured data analytics to access some of your data. This has worked adequately for you up to now, but with the amount of data that will be generated in the future, you need the processing capability to load and query multi-terabyte datasets in a timely fashion. You need the ability to manage both semi-structured and unstructured data.
Fortunately, a set of emerging technologies (called “Big Data”) may provide the technical foundation of a solution. Big Data usually includes data sets with sizes beyond the ability of commonly used software tools to capture, curate, manage and process data within a tolerable amount of time. While some existing technology may prove inadequate to future tasks, many of the information management methods of the past will prove to be as valuable as ever. Assembling successful Big Data solutions will require a fusion of new technology and old-school disciplines:
Which of these technologies do you have? Which of these technologies can integrate with on-premise AND cloud based solutions? On which of these technologies does your organization have knowledgeable resources that can utilize the capabilities to take advantage of Big Data?
Just finished reading the McKinsey quarterly on “Putting Big Data and Advanced Analytics to work.” It is an interesting article that hits on the need to have people who understand data and understand the business.
The short-term problem is that if you’ve developed a new model that predicts or optimizes, how do you get your frontline managers to use it? That’s always a combination of simple tools and training and things like that. Then there’s a medium-term challenge, which is “How do I upscale my company to be able to do this on a broader scale?” (more…)