Informatica Enterprise Data Management

Informatica has launched the Informatica Perspectives blog (RSS) where you can now find the latest Enterprise Data Management discussions among other topics. Please update your RSS subscription to track the following RSS feed for the latest blog posts on Enterprise Data Management.

Thanks,

The Informatica Team

We all know how mandates such as Sarbanes-Oxley place a burden on many businesses, by requiring that they be able to document the reliability and quality of data. Most major mandates, which have now been in place for several years, have given rise to a whole industry dedicated to reporting. In many companies, the equivalents of small departments have been kept busy 52 weeks a year doing little more than generating reports and reviewing data to meet compliance requirements.

Obviously, things can't go on like this. Rather than spending money to just keep simply meeting requirements, many companies are seeking to better integrate compliance into their day-to-day operations in a more automated, systematic form. In doing so, they seek to go far beyond meeting the letter of the law, to take the opportunity to improve and streamline their own processes - which will pay off in battling the challenges of an increasingly competitive marketplace.

By eliminating the silos that have separated data across the enterprise, as well as the silos that have pigeonholed the compliance efforts intended to gather and report this information, organizations can make impressive strides in moving forward with greater agility. In the process, automation can reduce the burden of paperwork and manual processes that drive up the costs of compliance.

Such "sustainable" compliance management can be built on top of three disciplines that already exist within most businesses today. These include governance, or the oversight of corporate activities and processes; risk management, or the identification, assessment and monitoring of risks and controls; and compliance management.  This integrated approach - known as Governance, Risk, and Compliance Management, or GRC, takes its three namesake disciplines and takes a more holistic approach to increasing information visibility and management. [Read more]

To paraphrase the Paul Simon song, there must be 50 ways to integrate your enterprise data. In recent years, companies have made all kinds of attempts to integrate both their applications and data - employing techniques from sophisticated enterprise application integration projects all the way down to manual hand coding. However, while most of these approaches work at least some of the time, few, if any, are delivering real agility for their businesses.

Recently, I had the opportunity to moderate a Webcast - sponsored by Informatica and hosted by ebizQ - which explored in detail an emerging approach, called data services, which ties into service oriented architecture (SOA) and creates a data abstraction layer that addresses the complexities seen across enterprise data environments.

Leading the Webcast were Ash Parikh, principal product marketing manager for Informatica and a highly regarded industry speaker and author, and David Ramos, director of business intelligence and analytics for LinkShare Corporation.

In his presentation, Ash urged closer collaboration between the enterprise data management and emerging service oriented architecture (SOA) worlds. (John Schmidt recently provided a nice overview of SOA here at the EDM blogsite.)

Ash observed that current approaches to enterprise data management have worked well from an application point of view, but have been ineffective for enterprise data. [Read more]

Healthcare is one of the last industries where you hear the term MDM (Master Data Management) mentioned. Most IT industry analysts, software firms and consulting organizations are geared towards your typical company that sells products to people or businesses. MDM examples are always getting a master list of products or cleansing your way to a consistent list of customers, which is not exactly the mindset of healthcare organizations. But lack of MDM is precisely what is adding untold costs on healthcare organizations (and ultimately on all of us) and inhibiting these organizations from improving the quality of health care services at an affordable cost.

Let's divide the healthcare industry (simplistically) into insurers and providers (we will position pharmaceuticals, biotechs and medical device companies as life sciences). Many of the large insurers have invested in data warehousing and data integration, but smaller insurers, i.e. regionally based HMOs (healthcare maintenance organizations) and healthcare providers, such as hospitals and physician groups, have fledgling efforts or have been bogged down in many of the issues below.

Healthcare organizations have significant data consistency issues regarding the following data subjects:

• Patients
• Physicians
• Procedures
• Diagnosis codes
• Service Rates
• Pay for Performance (P4P) measures

Each insurer and each healthcare provider tracks this data differently. The problems are magnified because healthcare is regulated on a state by state basis along with federal and industry regulations. Throw in privacy and security concerns to exacerbate what healthcare groups need to deal with.

Most healthcare organizations, even large ones, are an affiliation of generally small physician groups. These groups may be your local doctor's office, i.e. primary care physicians (PCP), specialists or emergency room (ER) providers. Often these groups do not have a lot of IT resources at their disposal.

Data flow is often flat file transfers between insurers and healthcare provider organizations, as well as from the individual physician groups and the larger provider organization. These flat files are generally not standardized and change each year when contracts are renegotiated between insurers and providers. This is an industry where you typically are not in control of your source data. It is thrown at you and you have to deal with it.

The need for an MDM is significant at healthcare organizations. The benefits from MDM are

• Data consistency
• Productivity
• Enabling more cost effect patient care

It is remarkable when one looks at the amount of resources devoted to manually dealing with inconsistent master data throughout health care. People in this industry do an amazing job of dealing with it, but it is often a time-consuming manual effort with much reconciliation. Having an MDM program would improve overall productivity and enable organizations to process and react more quickly to patients, insurers, employers and physicians.

A hidden jewel of an MDM effort though is enabling health care organizations to provide more proactive care. I have seen healthcare providers develop data solutions oriented to specific populations of patients who have diseases, chronic conditions or are at specific risks. These solutions may be for diabetes or asthma, for instance.

By bringing in historical clinical or demographic data related to patients tied together through consistent master data and taking advantage of predictive analysis, health care providers can proactively help their patients rather than waiting for the next episode when the patient's health has worsened. Many insurers are linking Pay for Performance (P4P) programs with these kinds of efforts because a healthier patient is a great goal by itself but better health also means lower health care costs.

The MDM silver bullet product has not been invented for healthcare industry but these organizations should not despair. There are concrete steps these organizations need to take.

• First, healthcare organizations need to examine where they are spending their resources on handling inconsistent master data and focus their efforts on those areas.
• Second, the efforts need to be in collaboration with business operations, physicians, insurers and IT, and need to involve defining master data and performance metrics.
• Finally, such organizations need to leverage their data warehousing and data integration efforts.

The most recent TDWI Boston Chapter meeting focused on how companies should approach and implement Master Data Management (MDM). Although the meeting had a keynote presenter and panelists with strong industry expertise and experience, the key to the discussions were the questions and insights of the audience attending the meeting.

The Boston chapter, with industries representing financial services, insurance, high tech, medical devices, biotech, retail, professional services and consumer products goods companies offers a diversity of perspectives about the challenges and benefits of addressing MDM.

Two key insights kept being reinforced during the discussions:

1. Leverage, Leverage, Leverage

Any company that will benefit from tackling MDM has most likely already been attacking the problem but not in the focused manner that MDM needs to truly be successful. But the biggest mistake that people saw with their peers and early adopters was the belief that MDM was something different than before.

Too often MDM is pitched as a "green field" opportunity with some solution as the "silver bullet" to one's problems. This approach fails to leverage past efforts from a business and technical perspective, thereby creating the potential for yet another application and data silo.

And, more importantly, failing to realistically assess the shortcomings and successes of existing efforts in making master data consistent is a sure fire way to plan to fail, i.e. repeating the failures of history is inevitable if you fail to learn from them.

Participants suggested looking at existing data warehousing and data integration efforts to leverage data, technical and people resources. Learn from the past.  Turn your joint IT and business efforts to define and manage data into a full-fledged  data governance program. If you have already started that type of program, expand  it and tie it into business successes (below).

2. Business Value

There needs to be specific business value derived from your MDM efforts. Catch phrases like "360 degrees of the customer" or "single version of the truth" are great marketing slogans but are esoteric and will become the brunt of jokes if they don't help you achieve real business value that can be measured. You can use these slogans to rally the troops and to get funding for the MDM efforts, but don't fall into that trap of believing your own sales pitch.

Your MDM will undoubtedly provide business ROI. Participants stressed that you should seek out those business opportunities and target your MDM towards them. Focused MDM efforts are more likely to get business participation, a critical success factor, to help you be on track to building your MDM program. Trying  to boil the ocean, i.e. trying to solve everyone's problems all at once,  generally fails to solve anyone's problems.

There are countless business processes or analytics that can be and are improved  by implementing MDM in your company. Find them, document them and determine  the business ROI that you can quantify or qualify. Get the business people involved  to be your customer references to sell the MDM program.

Success breeds success.

SOA has nothing to do with technology. It has everything to do with defining and managing the business as a collection of service functions and information exchanges. A business may be viewed as an internal organizational unit within a large company that provides services to other other units and consumes services from yet other groups. Or if you view the business as the company overall at the macro level, then you could define and manage the business as a collection of services consumed from its supply chain and provided to its customers. [Read more]

The most recent TDWI Boston Chapter meeting focused on MDM (Master Data Management). The keynote presentation titled “MDM: Data Salvation or the Next Round of Silos?” was followed by a panel discussion. The panelists included representatives from IBM, Oracle (Hyperion), SAP (Business Objects) and Kalido.

In keeping with TDWI principles, the panel discussion concentrated on the how’s and why’s of MDM in customer situations and avoided sales pitches. I moderated the panel discussion and was very impressed with the panelists’ knowledge and insights on their customers’ experiences in approaching and implementing MDM.

The panel discussed the most common characteristics that customers who are experiencing success in their MDM efforts have:

“Educated consumers”

There is a men’s clothing store that has an ad that says an educated consumer is our best customer. That is absolutely true in MDM. Companies that have been involved in data warehousing and enterprise data integration understand the complex and conflicting world of trying to get consistent, comprehensive and current master data or conformed dimensions in DW terminology.

Significant business participation

MDM is not a program that can be undertaken with IT alone. In fact, if you do not have business participation in the beginning there is little chance of success. IT has to sell the MDM program to the business and get real resource and time commitments.

Data governance program

Data governance is more important than what products you use to implement MDM. The old saying about garbage in, garbage out (GIGO) is absolutely true in MDM. You can’t cleanse data nor make it consistent unless you can define the data, business rules/transformations and performance measures you wish to use across your enterprise.

Enterprise Data Integration

Companies that have been implementing enterprise data integration efforts can leverage those efforts in their MDM program. They have gained an understanding of the complexities that a formal MDM effort will entail. For these companies MDM is not a new thing but simply taking their EDI efforts to the next level.

MDM is a journey and companies that have been in the trenches trying to address the problems of inconsistent data are well-positioned to take the next step.

Many thanks to Loraine Lawson who wrote an insightful article on her blog Classifying Integration Initiatives as Tactical or Strategic Her posting got me thinking. 20 years ago we didn’t need Integration Competency Centers – now we do. What changed?

From my perspective, there are three key drivers:

First, organizations have gotten bigger. The Fortune 500 companies today are much larger in terms of revenue, number of employees and global operations than Fortune 500 companies in the 1980’s. This conclusion is based on personal experience and anecdotal evidence. If readers of this column have seen any studies or statistics on the growth of large enterprises, please speak up.

Second, data diversity and volume has been growing at an exponential rate. We now have data in multiple formats such as video, images, audio, and text to name a few – and in multiple protocol formats and multiple languages. On the volume front, I recall an article from a few years ago that stated that manufacturers of storage devices would ship more than 22 exabytes (22 million trillion bytes) of hard disk capacity in 2005 - which is four times the space needed to store every word ever spoken by every human who has ever lived. In a relatively short number of years we’ve seen data storage capacities move from megabytes, to gigabytes and terabytes – now we’re talking in petabytes and exabytes. And coming soon are zettabytes and yottabytes.

Third, information technology has become more complex. One of the first principles of computer science is that re-use and decoupling of components is enabled by adding an abstraction layer. When I started my career (roughly 30 years ago), I was loading programs into computers using paper tape readers and toggle switches (I was pretty good at reading octal and hexadecimal code in my prime). But with increasing layers of abstraction – macro code to structured programming languages to 4^th generation languages to graphical integrated development tools to cloud computing in a web 2.0 environment – we have increasingly distanced the user from the underlying hardware and micro-code. Computers basically still work the same as they did 30 years (binary logic gates), but they are much more powerful, easy to use, and interchangeable which has been enabled by layer upon layer of abstractions.

The common elements across these drivers is constant change and growing incompatibility and complexity. Business processes are constantly changing with broader global scope adding many complications that are not transparent. Data definitions are constantly morphing and volumes are increasing to massive levels as everything becomes digitized and more regulated. Technology waves are constantly adding new variations and complexities without retiring prior generations.

Ergo, we have seen the emergence of integration as a new discipline which didn’t exist 20 years ago because it wasn’t needed then. In the 1980’s, integration was viewed as a one-time project discipline, not as an ongoing capability that an organization needed to sustain efficient end-to-end operations in a complex pattern of incompatible components that are constantly changing.

We could say that projects require micro-integration techniques while the complex systems-of-systems that emerge over time at the enterprise and supply chain scale require macro-integration best practices such as Integration Competency Centers. The ICC disciplines are a powerful capability, but they are non-trivial and hence those organizations that do it better than others will have a competitive advantage.

It seems in recent years pundits and vendors alike have been applying the 2.0 label to everything and anything emerging across the technology plain. In some cases, the new label has stuck - witness the widespread adoption of the terms 'Web 2.0' and its business sibling, 'Enterprise 2.0.'

In some cases, it’s a case of marketecture, but yet, the 2.0 identifier does convey a certain sense of maturity – that a technology is moving to a new stage of sophistication, of engagement with the business and its end users.

There have been moves afoot to identify the next generation of data warehousing as "Data Warehouse 2.0." However, there are differences of opinion as to what exactly will constitute DW 2.0, and thus no clear standard sense of direction in the market. [Read more]

This Wednesday, June 25, I have the privilege of hosting a Webinar featuring Ash Parikh, Informatica’s Principal Product Marketing Manager and a well-known author and speaker on enterprise data integration issues. Ash will be joined by David J. Ramos, Director of Business Intelligence and Analytics at LinkShare, an Informatica customer that provides online marketing services.

The Webinar, entitled Data Services - Maximizing Business Value Through Right-Time Information, is sponsored by Informatica and will be available live via ebizQ at 12:00 pm Eastern Time.

UPDATE: Archived replays of the Webcast are now available on demand.

Ash Parikh will discuss why many of the current approaches to integration - such as enterprise application integration (EAI), enterprise information integration (EII), and many manual processes still in use – are not giving organizations the agility they need to move to truly real-time, customer-focused enterprises. He will discuss an emerging approach - called data services - that creates a data abstraction layer that opens up all these formerly unreachable data stores across the organization.

David Ramos will explain how LinkShare, which handles 40 GBs of data across 300 million transactions a day, is employing Informatica technology to deliver grid-based data integration and meet the growing real-time data demands of its customers.

This promises to be a very informative and engaging session. Again, the live presentation will take place this Wednesday, June 25, at Noon Eastern Time.

Archived, on demand replay available here.