Tag Archives: Data Warehousing
The reality in data warehousing is that the primary focus is on delivery. The data warehouse team is tasked with extracting, transforming, integrating, and loading data into the warehouse within increasingly tight timeframes. Twenty years ago, monthly data warehouse loads were common. Ten years ago, weekly loads became the norm. Five years ago, daily loads were called for. Nowadays, near-real-time analytics demands the data warehouse be loaded more frequently than once a day. (more…)
Thousands of Oracle OpenWorld 2012 attendees visited the Informatica booth to learn how to leverage their combined investments in Oracle and Informatica technology. Informatica delivered over 40 presentations on topics that ranged from cloud, to data security to smart partitioning. Key Informatica executives and experts, from product engineering and product management, spoke with hundreds of users on topics and answered questions on how Informatica can help them improve Oracle application performance, lower risk and costs, and reduce project timelines. (more…)
The widespread adoption of electronic health records (EHRs) is a key objective of the Health Information Technology for Economic and Clinical Health (HITECH) Act, enacted as part of the American Recovery and Reinvestment Act of 2009. With the pervasive use of EHRs, an enormous volume of clinical data will be readily accessible that has previously been locked away in paper charts. The potential value of this data to yield insights into what works in healthcare, and what doesn’t work, dwarfs the benefits of simply replacing a paper chart with an electronic system. There’s appropriate enthusiasm that this data is going to be a veritable goldmine for enterprise data warehousing, business intelligence, and comparative effectiveness research. However, there are other, equally valuable, uses for this data to enhance clinical decision-making and improve the value of healthcare spending. Simply having instant access to large volumes of data that span thousands or tens-of-thousands of physicians, hundreds-of-thousands of patients and millions of encounters, offers an unparalleled opportunity to increase the quality and lower the cost of healthcare. (more…)
Most of the big data discussions have been on the technology or the numerously re-played business discoveries used as examples of big data’s power. Many companies are still in the experimental stages of big data, asking for guidance regarding what their benefits would be, how they can re-align themselves to take advantage, and what new processes might be helpful to make them successful with these powerful new capabilities. (more…)
We have all heard of data federation and of late we have also been hearing how simple, traditional data federation often gets passed off as data virtualization. Let’s get back to basics and take a hard look at what the real need is.
Data federation is not a new concept. When it first arrived on the scene many years ago, technologists got excited as it offered a way to quickly access numerous disparate data sources without physically moving data. Years passed and the term kept appearing in research paper after research paper – but what did not happen was the anticipated widespread adoption. TDWI’s Wayne Eckerson does a great job at tracking the evolution of data federation in his recent webinar and blog. Simple, traditional data federation does one thing and only one thing well – it creates a virtual view across heterogeneous data sources, delivering data in real-time, typically to reporting tools and composite applications. In its very simplicity lay its downfall.
The devil, as they say, is in the detail. Your organization might have invested years of effort and millions of dollars in an enterprise data warehouse, but unless the data in it is accurate and free of contradiction, it can lead to misinformed business decisions and wasted IT resources.
We’re seeing an increasing number of organizations confront the issue of data quality in their data warehousing environments in efforts to sharpen business insights in a challenging economic climate. Many are turning to master data management (MDM) to address the devilish data details that can undermine the value of a data warehousing investment.
Consider this: Just 24 percent of data warehouses deliver “high value” to their organizations, according to a survey by The Data Warehousing Institute (TDWI). Twelve percent are low value and 64 percent are moderate value “but could deliver more,” TDWI’s report states. For many organizations, questionable data quality is the reason why data warehouses fall short of their potential. (more…)
The “Business” Needs Critical Data “Now” – We Need The Next Generation Data Federation Technology “Yesterday!”
There is a lot of talk about using data federation, Enterprise Information Integration (EII) or data virtualization to deliver new data to the business, on-demand. However, do existing approaches cut it?
I have been following the data integration space for many years now, and like many of you, I have wondered about the viability of data federation as a data integration approach. Not because it does not hold promise – it does – it has many advantages as a fast, flexible and low cost approach to integrate multiple and diverse data sources in real-time, without the need for physical data movement.
However, according to the numerous architects that I have had the pleasure of meeting with on the Informatica 9 World Tour, simple or traditional data federation has not been able to live up to its immense promise. And why is that I asked – the reasons were many…
- The cost of losing one customer is four times higher than the cost of obtaining that same customer (Return on Behavior Magazine)
- Satisfying and retaining current customers is 3 to 10 times cheaper than acquiring new customers, and a typical company receives around 65 percent of its business from existing customers (McKinsey, 2001)
- A 5% reduction in the customer defection rate can increase profits by 25% to 80% (Return on Behavior Magazine)
- 7 out of 10 customers who switch to a competitor do so because of poor service (McKinsey, 2001) (more…)
In my previous entry, I mentioned how our customers are democratizing innovation in data warehousing and data integration. One thing is clear: there is renewed energy around using data to become competitive in the global market. We also have many options in data warehousing –a centralized enterprise data warehouse (EDW), departmental data warehouses and data marts, including appliance options and cloud computing for data warehousing. And yes, we do have an ongoing argument of EDW versus data marts. What’s different at this time is that, it’s the business that’s fueling this debate. A customer may outlaw a data mart because the maintenance costs and risks managing marts outweigh the potential benefits and thus do not meet the operating requirements. Another may opt for a cyclical data mart to allow for faster time to market and analytic flexibility for departmental business needs with a shorter turnaround.
One of our customers in the entertainment industry recently went live with their real-time data warehousing and master data management project. (more…)