Informatica Enterprise Data Management

To paraphrase the Paul Simon song, there must be 50 ways to integrate your enterprise data. In recent years, companies have made all kinds of attempts to integrate both their applications and data - employing techniques from sophisticated enterprise application integration projects all the way down to manual hand coding. However, while most of these approaches work at least some of the time, few, if any, are delivering real agility for their businesses.

Recently, I had the opportunity to moderate a Webcast - sponsored by Informatica and hosted by ebizQ - which explored in detail an emerging approach, called data services, which ties into service oriented architecture (SOA) and creates a data abstraction layer that addresses the complexities seen across enterprise data environments.

Leading the Webcast were Ash Parikh, principal product marketing manager for Informatica and a highly regarded industry speaker and author, and David Ramos, director of business intelligence and analytics for LinkShare Corporation.

In his presentation, Ash urged closer collaboration between the enterprise data management and emerging service oriented architecture (SOA) worlds. (John Schmidt recently provided a nice overview of SOA here at the EDM blogsite.)

Ash observed that current approaches to enterprise data management have worked well from an application point of view, but have been ineffective for enterprise data. [Read more]

It seems in recent years pundits and vendors alike have been applying the 2.0 label to everything and anything emerging across the technology plain. In some cases, the new label has stuck - witness the widespread adoption of the terms 'Web 2.0' and its business sibling, 'Enterprise 2.0.'

In some cases, it’s a case of marketecture, but yet, the 2.0 identifier does convey a certain sense of maturity – that a technology is moving to a new stage of sophistication, of engagement with the business and its end users.

There have been moves afoot to identify the next generation of data warehousing as "Data Warehouse 2.0." However, there are differences of opinion as to what exactly will constitute DW 2.0, and thus no clear standard sense of direction in the market. [Read more]

This Wednesday, June 25, I have the privilege of hosting a Webinar featuring Ash Parikh, Informatica’s Principal Product Marketing Manager and a well-known author and speaker on enterprise data integration issues. Ash will be joined by David J. Ramos, Director of Business Intelligence and Analytics at LinkShare, an Informatica customer that provides online marketing services.

The Webinar, entitled Data Services - Maximizing Business Value Through Right-Time Information, is sponsored by Informatica and will be available live via ebizQ at 12:00 pm Eastern Time.

UPDATE: Archived replays of the Webcast are now available on demand.

Ash Parikh will discuss why many of the current approaches to integration - such as enterprise application integration (EAI), enterprise information integration (EII), and many manual processes still in use – are not giving organizations the agility they need to move to truly real-time, customer-focused enterprises. He will discuss an emerging approach - called data services - that creates a data abstraction layer that opens up all these formerly unreachable data stores across the organization.

David Ramos will explain how LinkShare, which handles 40 GBs of data across 300 million transactions a day, is employing Informatica technology to deliver grid-based data integration and meet the growing real-time data demands of its customers.

This promises to be a very informative and engaging session. Again, the live presentation will take place this Wednesday, June 25, at Noon Eastern Time.

Archived, on demand replay available here.

Are BI managers and professionals sometimes too eager to please the business? Are centralized BI efforts slowing down progress? Should BI teams address requirements before the business even asks for them? These questions may seem counter-intuitive, but Wayne Eckerson, director of research for TDWI, says that the best intentions for BI efforts in many organizations may actually result in sluggish projects, duplication of effort, and misaligned priorities between BI teams and the business. [Read more]

There’s no question that integrating analytical and transaction data to deliver “Pervasive Business Intelligence” can be a significant project for many enterprises. However, the good news is that it’s a capability that’s within the reach of many enterprises today. That’s the gist of a Q&A with three industry thought leaders, published in the latest edition of Intelligent Enterprise. [Read more]

I’m already making my flight arrangements for the 10th Annual Informatica World Conference in Las Vegas this year. [Read more]

The recent Informatica Release 8.5 launch highlighted Real-time Integration Competency Centers (ICCs) as the optimal model for successful data integration. I’d like to review the concept of the Real-time ICC and why Release 8.5 supports this advanced operational, organizational and technology model.

As data integration moves beyond the realm of data warehousing into operational integration, real-time and data services use cases have exploded in importance to the business and necessitated stronger, unified infrastructure for IT to meet the challenge. Philip Russom, Senior Manager, TDWI Research captures this trend specifically in his quote on Release 8.5.

"The movement toward real-time data access and delivery has been the most influential trend in data integration this decade. The trend has enabled user organizations to initiate a variety of valuable real-time practices, including operational BI, real-time data warehousing, on-demand computing, performance monitoring, just-in-time inventory, and so on. And the trend has led vendors to extend their data integration products, so that many functions operate in real-time, not just batch. Informatica 8.5 is a great example of this trend, because it’s re-architected to support more real-time and on-demand functions for data integration, changed data capture, and data quality." [Read more]

One technical challenge not often discussed in data integration circles is the impact of real-time data to performance and scalability. I attribute this to a lack of real-world experience in handling real-time data, or a lack of recognition by IT that data integration software can effectively manage real-time data. Many architects and IT developers that I meet lump real-time into the EAI domain. This was a logical assumption 5 years ago, due to the fact that the data integration market was then primarily known for tackling “large batch volume” workloads (or as I like to refer to them “big batch problems”)

Informatica has spent 10 years focused to a good degree on solving that “big batch” problem. The inherent division between design time and run time in the underlying platform architecture enabled the introduction of parallelization/partitioning techniques, 64 bit processing, support for RDBMS vendor supplied batch utilities/APIs and improved data conversion/transformation without impacting the business logic design. This has proven invaluable to our customers in meeting their increasing volume, and in shrinking load window requirements.
[Read more]

Recently I moderated a panel at the Boston TDWI chapter (I am a chapter officer) on emerging trends in business intelligence (BI). I framed the discussion by having the panelists position technology in the five stages of the Gartner Hype Cycle.

It was a lot of fun and provided some good insights. The panel agreed that ETL was on the productivity plateau — meaning it was mainstream and commonplace. Everyone assumes everyone is doing it, but I challenged whether it was truly pervasive.

To support my claim I did an informal survey of the audience and asked some questions on their use of ETL. Sure enough, everyone was using it — that’s great news. And everyone was using it to load their data warehouse — again terrific.

But here is where the fun and eye-opening insight begins. When asked if they used their ETL tool to load their data marts it turns out most did not. And how many loaded their OLAP cubes with their ETL tool? Almost nobody.

This is consistent with what I see time and time again at my clients and what I hear from fellow consultants and IT folks. Recent surveys indicate that approximately 45% of ETL work is done by hand-coding.
[Read more]

We’ve been discussing the three pillars of an ICC, organization, process and technology, for a while now. In this segment, I’ll focus on a range of technology requirements facing ICC implementations teams, whether they are starting from scratch or morphing a set of disparate solutions into a common infrastructure. It goes without saying, to meet the demand of a broader set of enterprise needs rather than those of a single line of business, the infrastructure powering an ICC needs to evolve and mature.

High Availability
One of the first aspects related to infrastructure is the need for high availability. This pertains to the overall integration infrastructure environment. “Shared Infrastructure” by its very nature increases the need for reliability. An outage of a single point solution is acceptable and explainable but when several organizations are relying on solutions delivered by an ICC, outages can significantly impact revenue and productivity.

[Read more]