The next facet of our data governance framework focuses on the three intentionally simplified dependent processes that constitute the data lifecycle. When educating your business sponsors and evangelists on the data lifecycle, I like to categorize it into these three broad areas: upstream processes, stewardship processes, and downstream processes. If you’re an enterprise or data architect, you’ll likely have a much more granular set of steps in a data lifecycle, which is perfectly fine. But when engaging with your business partners, keep it simple and they may actually listen!
Your vision and business case that helps to scope and set the right business priorities for your data governance strategy will most often correctly focus on the downstream operational and analytical business processes, decisions and stakeholder interactions that consume or use trusted, secure data. Therefore much of the focus from those supporting data governance – specifically your executive sponsors, your data governance driver, and your business and IT stewards – target those downstream processes, be it reporting and analysis within your data warehousing or BI environments or data used to support operational processes within targeted transactions systems or applications. But as I’ve discussed often in this blog series, the truly effective data governance organization will aim to eliminate the “garbage in/garbage out” conundrum that is often the leading cause of untrusted, poorly secured data. And much of the data becomes untrusted “garbage” well before the downstream consuming process and application receives it. An effective data governance organization will accept responsibility for assessing and improving all of the process that touch the data and influence its usability.
Overview of the Data Lifecycle Process Stages
Upstream processes.These are the business processes that capture, create, import, purchase, transform, or update data and introduce it into your organization’s information ecosystem. One of the most common, and toughest to solve, data governance challenges centers around the reality that those in the organization responsible for these upstream processes rarely have visibility – or incentive to care – about who is consuming this data downstream, and why. This is where an influential and senior executive sponsor comes in. Your sponsor must enforce and evangelize amongst her peers the recommended data capture and maintenance policies generated by the data governance organization.
Note: The transform processes in this context are the transform rules (either within packaged data integration tools or via custom code) that IT may use to change the format or structure of the data, add necessary technical metadata, or otherwise ensure the data complies with IT modeling standards, but does not address the business-defined quality or security of the data. I instead separately categorize those quality- and security-centric transform rules within the stewardship processes –see next bullet.
Stewardship processes. In an upcoming post about the “Defined Processes” facet of the data governance framework, I will discuss in much greater detail the core processes that make up a data governance function – specifically the major process stages of “Discover, Define, Apply and Measure & Monitor”.
But when referring the lifecycle of the physical data itself (as I am in this post) the stewardship process stage most relevant is “Apply”. Apply refers to the actual ‘application’ from a system- or human-centric workflow of the data policies, business rules, standards and definitions created as part of your data governance program. The automated application of these rules may appear as service-enabled or application-specific rules that archive, cleanse, enrich, mask, match, merge, reconcile, repair, validate, verify or otherwise improve the security and quality of your data. The human-centric workflow refers to the end-to-end stewardship processes you’ve defined to facilitate the identification, notification, escalation and mitigation of any exceptions to your data quality or data privacy rules and policies listed above that require manual intervention to resolve.
Downstream processes. These are the operational and analytical processes that consume, protect, archive, purge and otherwise draw insight and value from data. This is where the rubber meets the road. Your executive sponsors will only agree to support changes to your upstream processes, systems and organizational behaviors – and allow investment to create the organizational and technology improvements needed to enable the stewardship processes – if you can deliver significant business value and ROI against these downstream processes that form the foundation of your vision and business case!