In my last blog for Advanced Metadata, I wrote about how metadata encodes meaning and context, and lets us know what the drawings on the walls of our enterprise caves… sorry: what the status reports and figures in our Enterprise Performance Scorecards mean.
For this post, I’m going to very quickly explore how data management, and data management standards and practices, relate to Regulation. In other words, what is the link between complying with legal obligations and managing data. Just like last time, we’re going to dive into a bit of history to do this, because we have been recording and managing information for millennia at this point. I’ll also look at some specific examples of how data management practices affect, and are affected by, regulatory drivers.
About 7,000 years ago, somebody sat and pressed lines that look like the footprints of chickens into a clay tablet on the banks of the Tigris river. That person was recording the inventory of grain held in a grain store in the city of Babylon. This is among the first examples of written records we have. To record that grain inventory, a number of pieces of data had to be defined, along with certain metadata.
For example, if the inventory said “10 bushels of corn”, someone would need to have defined what a “bushel” was and how much corn a standard bushel would hold. Without that key definition, key context and meaning would be missing, so decisions about how well stocked the granary was wouldn’t be possible without actually going to look.
The “chicken footprints” of the Sumerian language etched on that clay tablet recorded data about things in the real world in a form that allowed the king to make decisions about the adequacy of supplies and the risks to the people in the city without having to go and count bushels himself.
Imagine for a moment that there may have been a legal rule that grain stores had to have a certain volume of grain stored at all times. It should be clear to see how the definition of the metadata about the size of a bushel and the accurate cataloguing, recording and transcription of that data might affect the ability of merchant to comply with that law, and the potential impact on the city if merchants managed their data without reference to some defined standards for data.
Or, if you want a more modern metaphor: The clay tablet was a report from the ERP system used by the Babylonian king to do stock control on their inventories, rather than having to go and do a physical stock take every time he wanted to know how many mouths his kingdom could feed. Heck, he even used a tablet… we’ve come a long way in information management!
Laws and regulatory controls such as standards are usually defined by legislators to ensure standardisation in how data is processed and calculated, or how it is published and presented. Whether imposed by governments or introduced by industry standards bodies, these regulations result in consistency.
When you measure the area of the floor in your living room to buy carpet, you know that the metre length you are measuring is benchmarked against a standard unit of measure, and the mathematical formula for calculating area is defined consistently, so the carpet you buy will be correct size.
Looking at this from an “industry” perspective, we can see an array of laws and regulations that look to address this challenge of consistency and comparability of data, and the trustworthiness of information:
- The Sarbanes-Oxley Act in the United States focused on ensuring proper management and remediation of information-related risks that could contribute to a material error in a balance sheets, so that investors could more reliably compare the performance of businesses. This requires management and governance of the lineage and quality of financial data in the organisation, and clear definition of key business and technical metadata. In effect, it is allowing investors to compare the size of grain stores more objectively, knowing that the inputs into the reporting are accurate and the bushels counted were all the same size and weren’t counted twice or three times.
- Solvency II in the Insurance industry is a regulation that requires organisations to manage the quality and lineage of data in their risk models so that they can ensure they have sufficient capital set aside to cover the policies they have written and underwritten. At a very simplistic level, this is essentially the same problem as the Babylonian king wanting to know if the granaries were stocked well enough to deal with a famine or other disaster, but the regulation is there to allow Regulators (and investor) to make decisions on whether the grain stores are well stocked enough so the kingdom doesn’t collapse when the people come calling for their grain.
- Privacy laws such as the EU’s General Data Protection Regulation, amongst other things, require organisations to communicate what they intend to do with data they record about people, define rules for identification and classification of data that is being managed, and set out rules for the security of the transfer and sharing of data. If our Babylonian clerk had also been recording information about who had provided grain into the granary and who the staff were who were running the granary, and which other kingdoms grain was being shared with, it would have been important for similar standards to be applied to that data so that the king could trust its accuracy, and so that the people who provided the data could trust that the king wouldn’t have them put to death just because their bushel count was off.
Ultimately, data is facts about things in the real world that we have recorded in some format. When we are acting as individuals in isolation from others we can record it and manage it anyway we want.
However, if our organisation grows to more than one, and we are engaged in communication with others, reporting to others, or when others are taking the information we have shared and using it as an input into their own processes, we need to define internal standards and structures for managing and governing that information so that everyone knows what is being talked about. The business and technical metadata that drives context and understanding becomes important internally, and when it is missing we find data quality problems and errors that require correction and rework.
When we begin to interact with entities outside our organisation, the legislative rules or industry peer regulation begin to apply to ensure that the context and meaning of the described facts are commonly and consistently understood and interpreted. These rules act to ensure that our internal management policies, procedures, and controls on metadata and meaning are aligned to deliver trusted and trustworthy information, and also to ensure that people can trust how our organisations are managing data to deliver value, either by improving efficiency, respecting privacy and acting ethically, or by reporting honestly the performance of the organisation.
In that way, the modern kings of Babylon can look down at the summary performance scorecard on their modern tablets and trust that the granaries are indeed full.