Informatica Data Quality

Just gave a presentation at MIT's Information Quality conference hosted at the Sloan school of management. Data Governance largely deals with softer topics like people, organizational strategies, and processes.  Not necessarily technology.  The irony was not lost on anyone that this presentation given at MIT stressed that technology alone would not solve a company's data quality problems.

It was a real privilege and honor for me to return as a lecturer to some of the same classrooms I attended as a student. MIT's Sloan school is right next to the Media Lab where I did undergraduate research some twenty years ago.  The most profound takeaway from my time as an engineering student was the notion that technology alone could not solve hard problems.  Back in 1986, we were experimenting with sending images and video over the network and the prof's were always stressing that social and organizational considerations factored heavily into technology adoption.  This may sound obvious to grizzled IT veterans, but to the wide-eyed geeks studying at MIT, this came as quite a revelation.  Certainly, this is the underlying driver behind Data Governance - it's a necessary framework so the enterprise can leverage and apply data quality, data integration, and metadata management technology.

The presentation covered several case studies involving successful customer deployments of enterprise-wide data governance programs.  Many of the attendees commented that they found it necessary to gain initial wins on tactical projects so they could gain credibility and navigate the political issues behind an enterprise deployment. There was certainly some really vigorous discussion and debate on this topic.

What experience have you had with implementing a data governance program?  Just like these MIT students, feel free to share your opinions with us.

In my previous posts, I have discussed building the business case for data quality as well as the role that a data quality dashboard plays in supporting this case. As previously noted, these efforts will directly impact your ability to articulate the need to pursue a data quality initiative. The reason for returning to this topic is that I have recently participated in multiple discussions with a variety of companies that were either in the process of forming a data governance council or in the process of building the internal business case to support exploring a data governance initiative. In these discussions two common threads were present – the role of data quality in the data governance initiative and the need to change the culture within the organization if data governance is going to succeed. Although these are only two aspects to consider when pursuing a data governance initiative, they are directly tied to the underlying success or failure of the program. [Read more]

Alice: Would you tell me, please, which way I ought to go from here?
The Cheshire Cat: That depends a good deal on where you want to get to
Alice: I don't much care where.
The Cheshire Cat: Then it doesn't much matter which way you go
– Lewis Carroll, Alice's Adventures in Wonderland

When confronted with the problem of how to address their data quality issues many organisations are faced with a similar dilemma to that which confronted Alice during her travels in Wonderland; “I know that I need to do something, but I don’t know where to start”. Knowing where to start and, equally importantly, the size of the problem as well as where an organisation needs to go are critical factors in ensuring that their data quality journey takes them where they need to be at the price they are prepared to pay.

When planning their “journey” organisations need to address the issue of data quality holistically by considering each of the three DQ pillars in turn; firstly “People”, then “Ideas” and finally “Technology”. Many DQ initiatives have failed as the primary focus has been on delivering a technical solution. However without the right framework in place and operated by the right people this approach will never deliver the results that organisations need. Time and time again within the IT industry it has been proved that the pure application of technology will never solve business issues, as technology in itself will never win the “war”, it is always the right people with the right ideas who use the technology in the right way.
[Read more]

I’ve just been reading a US Department of Education briefing document on improving data quality in education performance data. The report stresses the impact that low quality data can have on measuring the success of education programs. It discusses for example the numerous data quality problems identified in the “No child left behind” program established in 2001. The problems are typical – non-standardized data definitions, inconsistent data from different sources, data entry errors, lack of timeliness.

The briefing document outlines a broad set of data quality guidelines to be implemented right across the education system in the US – at State level, in Local Education Agencies (LEAs) and in schools themselves. The three foundation stones of the data quality framework outlined are:

• suitable technical infrastructure,
• a comprehensive dictionary of data definitions
• staff ownership, organization and training
[Read more]

All he wanted for Christmas was anything but what he got. Jeffrey Skilling, former Enron CEO moved to his new residence at the Federal Correctional Institution in Waseca, Minnesota, where his sentence calls for him to live for the next 24 years for his role in fraud, conspiracy, insider trading and other crimes leading to the collapse of Enron. These crimes led to the loss of thousands of jobs, more than $60 billion in company stock and more than $2 billion in employee pension plans.

But Mr. Skilling will have a new job as well. He will probably work as a food service helper, painter or plumber. While this is not the cush job he had as CEO at Enron where he earned $151.7 million over the three years during the time he perpetuated his fraud, he will get from 14 to 40 cents per hour. At the top pay, Skilling could earn $832 per year. At that rate it would take 74.5 million years to pay back the stock and pension losses he foisted on the stakeholders.

So what is the point here? [Read more]