Informatica Perspectives

Flipping around traditional ETL (Extract-Transform-Load) on its head is not a new practice.  ELT (Extract-Load-Transform), where processing is handled in the database, instead of the ETL server, has been proven to enhance performance in many types of data warehousing deployments.

For example, Oi, a leading telecom provider in Brazil, implemented an enterprise data warehouse (EDW) consolidating information on 36 million customers, speeding response time to customer requests.  The right-time EDW also enabled Oi to rapidly launch a successful new service offering, which made it easier for customers to recharge their pre-paid accounts for telecom service.

By implementing ELT with Informatica’s pushdown optimization capabilities for this Teradata data warehouse, Oi accelerated its data warehousing loading process two-fold.  This has led to even more timely updates of Oi’s customer information, while lowering costs.
[Read more]

In Chris Cingrani's recent post the question: "Data quality, does anyone care?" was posed. The answer is yes, of course people care about data quality – in fact, there are a lot of good reasons why a lot of people should care very deeply about data quality. Let’s look at the most recent example of where data quality makes a big difference, and that is in the federal election process. [Read more]

Can business intelligence benefit from the current excitement around the rise of Web 2.0 and Enterprise 2.0? Some say the intersection of BI and Web 2.0 will advance us into “Business Intelligence 2.0,” which promises up-to-date information and actionable insights about every aspect of the business. Fellow blogger Rick Sherman recently observed that BI 2.0 isn’t just about tools and technologies, but about “getting more comprehensive, consistent, correct and current data…. We can finally interweave data from the data warehouse with real-time and event-driven data via our data integration efforts.”

Can Web 2.0 make the promise of BI 2.0 more of a reality? [Read more]

If banks and financial institutions had invested in more data integration and business intelligence tools to spot issues arising within their portfolios, could they have avoided the recent credit mess?

Perhaps, to a degree. But it is human beings that are ultimately making the risk judgments, and oftentimes, bad decisions may have looked good at the time they were made.

Still, technology has improved to the point where troubles could have been more effectively flagged. [Read more]

Years, ago, I came across this question in an article in Boardroom Reports:  "What do you call a hamburger that’s 99% meat and 1% garbage?"

The answer was a "garbageburger." In other words, even if a small fraction of the burger is tainted, the whole meal is tainted. The original analogy was being used to illustrate the challenges of time management, but it's an apt analogy for data environments as well. That is, if a portion of the information is bad or unreliable, trust in all the data eventually breaks down. In essence, many implementations of service-oriented architecture (SOA) taking place across companies may be garbageburgers because they are serving up unreliable information – an element that has been out of the control of SOA designers.

Sorry if I ruined anyone’s lunch, but the point had to be made. [Read more]

Predicative analytics sifts though current and historical data to predict future events or behaviors. It incorporates statistical and data mining techniques to determine patterns, highlight risks and identify opportunities.

It doesn’t just extrapolate future performance based on past performance, although that is an input. It also identifies the relationships across many factors to lay out not only what is likely to happen, but also why it may occur along with potentially how to alter the outcomes.

Where there’s data warehousing, there’s predictive analytics. It is increasingly in many industries including retail, telecom, pharmaceutical, insurance and financial services. These industries typically leverage predictive analytics to analyze and predict consumer and business behaviors along with economic predictions. [Read more]

Just as they say success is 10% inspiration and 90% perspiration, it can also be said that the success of a data integration project is 10% technology and 90% chemistry. And when I say chemistry, I'm not talking about hydrocarbons and nitrates, but the chemistry of people.

The success of any complex data integration depends on how the people that make things happen - the teams of administrators, analysts, managers, end-users, and business partners - can collaborate in establishing the business case, setting requirements, selecting technology, and putting all the pieces together.

However, two of the key players in data integration - analysts and administrators - don't necessarily see eye to eye, and this is costing enterprises in terms of staff resources and quality. [Read more]

The United States and world economies are going through tough times. Many financial services firms have suffered severe losses and many other industries are being impacted. Although business spending on IT has yet to contract overall, it has been reduced in the worst hit sectors to date, and is likely to be challenged overall in the coming months.

Although IT spending is not immune from a downturn, there are many facets of this spending that have become invaluable to business such as data warehousing and business intelligence.

At the heart of DW and BI programs is data integration. Whether it is performance management, predictive analytics, Master Data Management (MDM) or other data-centric initiatives, data integration is the cornerstone of these efforts. [Read more]

"Despite certain rumors to the contrary, data warehousing is thriving."

I couldn't agree more with Judy Ko in her recent post, in which she points out that predictions that data warehousing was going to be abstracted away — by service-oriented architecture (SOA) and other new approaches — didn't quite pan out. Instead, if anything, the need for data warehousing solutions only continues to grow. Data volumes are growing, and businesses are demanding ever-more sophisticated business intelligence and analytics to run against that data.

If anything, approaches such as SOA promise to greatly enhance - not replace - data warehousing, [Read more]

In one of my earlier posts I discussed the need for a sophisticated data services-driven technology serving as the foundation for SOA and BPM.

When I was poking around the web recently, I ran into a powerful statement by Michael Blechar from Gartner, covered in the DAMA keynote, titled Survival of the Data Management Fittest:

"Data and processes are intertwined. It will fundamentally change the way organizations think about your roles, and your roles are going to need to evolve".

At this year’s Data Management Association (DAMA) International Symposium,
Michael is quoted saying that:

"In this world there's a very loosely coupled user interface from the assembled services that in turn share access to data. SOA exposes data issues to more people, places and processes, and what I tell companies is that without a focus on information management and meta data management they're going to fail."

It is in speaking to numerous customers, prospects and technologists that I had gathered that without accurate, consistent and timely information, SOA and BPM deployments will face serious information-centric hurdles, affecting the cost-effectiveness and success of the project. As we move towards more agile architectures, I believe that we need to grow typical process-centric approaches to include information centricity as well.

As Michael states:

"Where we are going is beyond the first generation of BPM and SOA [that is process-centric]," he said, "to the next generation of SOA that is information-centric."

Observe that the key word here is "information-centric." Reading such statements from Michael and many others definitely validates the strategy I have been defining for building out an effective IT infrastructure that can benefit from the flexibility of a services and process-driven approach, in the data integration layer. Simply wrapping data access with a web service does not qualify as a sophisticated data service and hence, stringing together such simple services with a BPM tool also does not guarantee agility.

As discussed in Services to Orient your Enterprise Data Layer, Joe McKendrick is of the opinion that neither SOA nor enterprise-application integration alone can effectively handle the enterprise data layer. However, data services delivered within an SOA framework can create a data-abstraction layer to address the complexities seen across enterprise data environments.

I have always said that without serving up good quality, consistent and timely information as a data service or a comprehensive data service built using a sophisticated data integration platform, SOA and BPM deployments will not be able to deliver on their promise of agility.

What are your experiences? What kind of information-centric issues have you run into in your service-oriented deployments? Is inaccurate, stale and inconsistent information passing through your IT infrastructure holding you back?