There is an old saying that if all you have is a hammer, every problem starts to look a lot like a nail. The humble electronic spreadsheet has been with us since shortly after breakfast on the morning of modern computing, with LANPAR providing spreadsheet capability to users of large mainframe computers as far back as 1969. Visicalc was the software that launched a thousand Macs, and helped a small start-up tech company with a fruit-based name climb out of their garage and become the behemoth they are today.
In most organizations, if you scratch the surface of any business intelligence, governance, or quality management initiative you will find an excel spreadsheet. The humble spreadsheet has achieved such a level of ubiquity in the organization that it has become like the Swiss Army knife for organizational MacGyvers trying to improvise their way around the need to quickly create documentation and records.
It is very easy in such a circumstance to overlook the fact that our modern spreadsheet software is nothing more than an electronic representation of the paper-based accounting ledgers that historically have been used to keep financial records. In that sense, our friendly Babylonian scribe (who we met a few posts ago) would not find them too different to the types of record keeping tools they would have used. And therein lies the problem with using spreadsheets for anything more complex than recording basic tabular data.
They don’t scale in three dimensions. Tracking relationships beyond the basic row and column intersection is painful and cumbersome. Work-arounds to make them scale usually involve complex hyperlinking between cells, sheets, and workbooks. This inevitably breaks.
Version control of iterations of spreadsheets is also a problem. Copies are made and can be saved on individual’s local drives or network drives, emailed copies can be circulated with limited controls, and edits made in one copy may not be reflected back in the “master”. This is actually nothing more than a symptom of a wider problem that Michael Brackett calls “disparate data”. Ironically, the use of spreadsheets to tame data governance and metadata chaos often leads to yet more data governance and metadata chaos.
To put it bluntly: In my experience, long term reliance on spreadsheets to manage metadata or to control data governance activities is contra-indicated with sustainable information management improvement, and mental health.
What should you do?
Spreadsheets are used to solve the immediate problem of having to capture lots of attributes about things in a list form and be able to identify relationships at the intersection of rows and columns. They can serve a valuable purpose in helping you to prototype your requirements for a metadata management or data governance solution.
However, you should be wary of their constraints. Ultimately, to manage and govern information through effective metadata management you need to be able to identify and understand the relationships between data across multiple dimensions. This may result in you trying to hack something out in the mature cousin of the spreadsheet, the Access Database. But again, that has limitations of data synchronization, access, and control.
Take it from someone who has been there… hand crafting your data governance controls in custom Microsoft Access databases is doable but gets complicated very quickly.
These steps, however, are just that… steps. They are steps on the road to a level of maturity where you will eventually:
- Understand your requirements for a technology solution to help manage your metadata
- Understand more about your data
- Not want to have to manually re-key technical metadata into a spreadsheet or figure out another custom import routine into an Access database.
Spreadsheets area primitive information management tool. As Einstein famously said: “We cannot solve our problems with the same thinking we used when we created them.” Spreadsheets represent the same level of thinking that created our disparate data problems and metadata messes.
Rather than simply creating another silo of unconnected disparate data, perhaps you should banish the spreadsheets and step up your level of thinking to see how scalable tools and technologies can help.