Tag Archives: Architecture
A couple of weeks back I posted on the shortcomings of the application approach to multidomain MDM, so this week let’s take a look at the many reasons why the platform approach is the superior alternative for effective multidomain MDM. The primary technological difference between the two approaches is that MDM “applications” typically employ a predefined data model, business logic, and a dedicated graphical user interface (GUI) tied to solving a single business problem, whereas platform-based MDM allow users to create and use flexible data models, configure it to suite any business logic, and provide visibility across any number of business processes via a single user interface. (more…)
I’m looking forward to doing a Webinar on data virtualization this Thursday, April 22nd. Why? Because this is the single most beneficial concept of architecture, including SOA, and it’s often overlooked by the rank-and-file developers and architects out there. I’m constantly evangelizing the benefits of data virtualization, including integrating data from many and different data sources in real-time, and enabling query-based applications to get data from multiple systems.
The idea is pretty simple, really. Considering that there are many physical database schemas within most enterprises, and typically no common view of the data, data virtualization allows you to map many physical schemas to virtual schemas that are a better representation of the business. For example, a single view of customer data, sales data, and other data that has the same logical meaning, but may be scattered amongst many different physical database systems, using any number of implementation models. (more…)
Loraine Lawson did a great job covering the topic of the integration challenges around the cloud and virtualization. She reports that “…a recent Internet Evolution column [by David Vellante] looks more broadly at the cloud integration question and concludes that insufficient integration is holding up both cloud computing and virtualization.”
In fact, what currently limits the number of cloud deployments is the lack of a clear understanding of data integration in the context of cloud computing. This is a rather easy problem to solve, but it’s often an afterthought.
The core issue is that cloud computing providers, other than Salesforce.com, don’t consider integration. Perhaps they are thinking, “If you use our cloud, then there is no reason to sync your data back to your enterprise. After all, we’re the final destination for your enterprise data, right?” Wrong. (more…)
As a CIO, I am a strong proponent of Enterprise Architecture (EA) and the components of EA articulated by Steven Spewak. Eight years ago, I would sit with the Informatica R&D Chief Architect and describe what I needed to realize our IT architectural vision, as well as the problems I wanted to overcome.
So, what were the problems I wanted addressed?
First, I am a believer of a best of breed strategy. I fundamentally believe the “megavendors” are dictating IT strategy, yet they cannot innovate fast enough – thereby harming IT. To build a best of breed approach, I wanted to build a loosely coupled architecture. In essence, I wanted to abstract the data away from the applications we ran, thereby enabling me to switch vendors if necessary. This would enable me to provide the best solutions to our business as well as maintain negotiating leverage with my vendors. The challenge is that no technology existed to do this cost effectively. (more…)
Are BI managers and professionals sometimes too eager to please the business? Are centralized BI efforts slowing down progress? Should BI teams address requirements before the business even asks for them? These questions may seem counter-intuitive, but Wayne Eckerson, director of research for TDWI, says that the best intentions for BI efforts in many organizations may actually result in sluggish projects, duplication of effort, and misaligned priorities between BI teams and the business. (more…)
There’s no question that integrating analytical and transaction data to deliver “Pervasive Business Intelligence” can be a significant project for many enterprises. However, the good news is that it’s a capability that’s within the reach of many enterprises today. That’s the gist of a Q&A with three industry thought leaders, published in the latest edition of Intelligent Enterprise. (more…)
The recent Informatica Release 8.5 launch highlighted Real-time Integration Competency Centers (ICCs) as the optimal model for successful data integration. I’d like to review the concept of the Real-time ICC and why Release 8.5 supports this advanced operational, organizational and technology model.
As data integration moves beyond the realm of data warehousing into operational integration, real-time and data services use cases have exploded in importance to the business and necessitated stronger, unified infrastructure for IT to meet the challenge. Philip Russom, Senior Manager, TDWI Research captures this trend specifically in his quote on Release 8.5.
“The movement toward real-time data access and delivery has been the most influential trend in data integration this decade. The trend has enabled user organizations to initiate a variety of valuable real-time practices, including operational BI, real-time data warehousing, on-demand computing, performance monitoring, just-in-time inventory, and so on. And the trend has led vendors to extend their data integration products, so that many functions operate in real-time, not just batch. Informatica 8.5 is a great example of this trend, because it’s re-architected to support more real-time and on-demand functions for data integration, changed data capture, and data quality.” (more…)
Recently I moderated a panel at the Boston TDWI chapter (I am a chapter officer) on emerging trends in business intelligence (BI). I framed the discussion by having the panelists position technology in the five stages of the Gartner Hype Cycle.
It was a lot of fun and provided some good insights. The panel agreed that ETL was on the productivity plateau — meaning it was mainstream and commonplace. Everyone assumes everyone is doing it, but I challenged whether it was truly pervasive.
To support my claim I did an informal survey of the audience and asked some questions on their use of ETL. Sure enough, everyone was using it — that’s great news. And everyone was using it to load their data warehouse — again terrific.
But here is where the fun and eye-opening insight begins. When asked if they used their ETL tool to load their data marts it turns out most did not. And how many loaded their OLAP cubes with their ETL tool? Almost nobody.
This is consistent with what I see time and time again at my clients and what I hear from fellow consultants and IT folks. Recent surveys indicate that approximately 45% of ETL work is done by hand-coding.
One technical challenge not often discussed in data integration circles is the impact of real-time data to performance and scalability. I attribute this to a lack of real-world experience in handling real-time data, or a lack of recognition by IT that data integration software can effectively manage real-time data. Many architects and IT developers that I meet lump real-time into the EAI domain. This was a logical assumption 5 years ago, due to the fact that the data integration market was then primarily known for tackling “large batch volume” workloads (or as I like to refer to them “big batch problems”)
Informatica has spent 10 years focused to a good degree on solving that “big batch” problem. The inherent division between design time and run time in the underlying platform architecture enabled the introduction of parallelization/partitioning techniques, 64 bit processing, support for RDBMS vendor supplied batch utilities/APIs and improved data conversion/transformation without impacting the business logic design. This has proven invaluable to our customers in meeting their increasing volume, and in shrinking load window requirements.