Monthly Archives: November 2007
Your 2008 Data Integration Plans, Part 2:Replacing your data shadow systems or spreadmarts
One of the dirtiest unspoken secrets in business intelligence (BI) and performance management (PM) is that most business people, despite the sizable investment in BI, PM and data warehousing, are not getting the information they need to make decisions. (See this Accenture study.)
And the reason for that, yet another unspoken truth, is that most business people are performing their analysis via data shadow systems or spreadmarts.
A quick recap: a data shadow system is an application built by the business. It gathers data from many sources, and business people often augment that data manually, using Microsoft Office for reporting and analysis. Ask business people how they really get their information and their answer is almost sure to include a data shadow system.
Too often the “problem” of data shadow systems is that people are using Microsoft Excel as the front-end BI tool. That is really not the problem. The real problem causing inconsistent and error-prone data is that people are using both Microsoft Excel and Microsoft Access to perform ETL! A typical data shadow system has:
- between six to three dozen steps (and I have seen systems with hundreds!) of Microsoft Office (Microsoft Access and/or Microsoft Excel) queries or imports gathering data,
- a series of steps to “integrate” the data, and finally,
- a number of worksheets to create the reports.
Building the Business Case for Data Quality
As a new contributor to the data quality blog site, I wanted to start by introducing myself and highlighting the types of topics I plan to discuss on a semi-frequent basis. I am a Principal Consultant with Informatica Professional Services and have spent the past 6 years in the data quality space in a variety of sales and post sales roles. During this time I have seen the data quality market continue to evolve and mature. Thus, I would like to use this column to reflect on the types of use cases I have seen and continue to see when meeting with organization’s faced with data quality problems. I hope these posts can start an active dialogue, regardless if your company is trying to tackle their first data quality initiative or looking to build out a formal center of excellence around data quality.
To start, I wanted to pose a common question I am often asked by clients and prospects – how do I build a business case for data quality? Although an organization may think (or even know) there is a problem, the need to justify the cost around procuring a data quality solution often exists. This justification requirements often comes from the idea that data quality issues aren’t necessarily a core business issue (how wrong this is!) or something that can be handled through manual intervention (this is true – if you have unlimited time and money, but even then your results will be limited). Thus, the following points are meant to help start an organization down the path to building the internal business case through a Data Quality Audit. Note – if you have access to Informatica’s Velocity Methodology, I go into these steps in further detail in the best practice document, “Developing the Data Quality Business Case.”
(more…)
Your 2008 Data Integration Plans, Part 1
Your 2008 strategy and budget are probably all set. Now it is just a matter of tactics to implement your business and IT objectives for next year. Have you examined the top business initiatives and IT projects in the context of integration?
Unfortunately, companies historically design and deploy business initiatives and their supporting IT projects in business and application silos. They justify these projects with a solid business return on investment (ROI) based on very high expectations of what the business will get. The expectation may involve great looking dashboards, terrific visualization, real-time access and widely distributed reports.
But many of these projects will fall into what Gartner Research describes as the “trough of disillusionment.” Too many times these projects will be labeled failures, not because the technology was not terrific, but because the data was not correct or consistent. A beautiful visual display or enabling pervasive data access is only of value to the business if that data is right. It’s all about the data. And that’s all about the integration.
(more…)
Real-time Integration Competency Centers – What are they?
The recent Informatica Release 8.5 launch highlighted Real-time Integration Competency Centers (ICCs) as the optimal model for successful data integration. I’d like to review the concept of the Real-time ICC and why Release 8.5 supports this advanced operational, organizational and technology model.
As data integration moves beyond the realm of data warehousing into operational integration, real-time and data services use cases have exploded in importance to the business and necessitated stronger, unified infrastructure for IT to meet the challenge. Philip Russom, Senior Manager, TDWI Research captures this trend specifically in his quote on Release 8.5.
“The movement toward real-time data access and delivery has been the most influential trend in data integration this decade. The trend has enabled user organizations to initiate a variety of valuable real-time practices, including operational BI, real-time data warehousing, on-demand computing, performance monitoring, just-in-time inventory, and so on. And the trend has led vendors to extend their data integration products, so that many functions operate in real-time, not just batch. Informatica 8.5 is a great example of this trend, because it’s re-architected to support more real-time and on-demand functions for data integration, changed data capture, and data quality.” (more…)
Have an Open Mind While Planning for 2008 (Think Beyond ETL)
I teach data warehousing and business intelligence courses at Northeastern University’s Graduate School of Engineering. It provides me a fresh perspective on what we are doing in the industry. This is because unlike when I am talking with customers, analysts and vendors, my graduate students are always asking “WHY.”
They ask because they do not have our industry experience or background. Yes, I do need to take the time to explain a lot of things that I otherwise would not have to, but that is actually good news. The graduate students also have not been locked into a particular way of doing things because they see everything as a new experience. They have open minds.
It would be a good idea for anyone planning projects for next year to have an open mind, too.
(more…)
Loading ...