Data Quality Index - Is it new?

During a conference a couple weeks ago, I was asked, "What is one thing that is new about data quality?" A wise man once said, "What has been will be again, what has been done will be done again; there is nothing new under the sun." (Solomon in Ecclesiastes 1:9) 

We tend to repeat ourselves without making substantial progress. Instead, we should look to synthesize concepts that have provided consistent results, lower costs, and please stakeholders. Data diagnostics is one example of this approach.A related approach that we have been working with is a data quality index score. Think of it as the S&P 500 or the Dow Jones Industry Average. The data quality index would be a single, specific number for a data domain that expresses in relative terms the quality of the data. In financial services, we understand indexes and they resonate. The next step is to define other data quality indexes that are more specific – Energy services, bio-tech, construction, etc. or in data language, completeness, consistency, duplication, etc. Each index would be comprised of components, which is the third layer, similar to how indexes have individual stocks, bonds, and other financial instruments.

Of course, this approach does not remove the need to have record analysis across data quality dimensions, but like demographic information about a diagnostic, it provides a simple communication method for executives to understand.

Is this a new thought? Was Solomon wrong? No, it is an application of an existing practice that we understand to a new subject area (data). As we move through 2016, we encourage you to test out this index concept, follow our progress with it, and look for other innovative ways to leverage common concepts to communicate in new and interesting ways.