10 Techniques for Data Quality Root-Cause Analysis
Tackling Data Quality Root-Cause Issues
How do you get started with identifying the true causes of Data Quality Root-Cause issues?
This article examines 10 techniques that are proven to help you get to the heart of your data quality issues and move from a reactive to proactive and longer-term defect elimination.
1. Focus on problems that impact customer satisfaction
It costs time and many valuable resources to resolve root-cause issues so targeting improvements that will directly benefit the customer is often a reasonable starting point.
Categorisation and prioritisation is vital here so please read the sections below on how to use Pareto analysis and create a fishbone diagram.
A Cause and Effect (C&E) Matrix is also a useful tool for linking process or data failures to customer impacts.
2. Eliminate the short-term practice of data cleansing
Data cleansing is one of the biggest cost-inducing activities of any business. Cleansing is typically a localised activity that delivers a quick-fix, tactical solution, for a particular system or line-of-business but often does nothing to prevent issues either up or down the information chain.
Data cleansing is not a scalable solution. By ignoring the root-cause of an issue we create a much larger problem as a single fault typically manifests itself exponentially as it interacts with different processes, people and systems across the business.
For these reasons, although it may appear a cheaper solution locally, long-term it is always far more expensive to the business as a whole.
Tip: Identify areas where data cleansing is taking place as an additional starting point for your root-cause analysis efforts.
3. Ensure information chains are accurate and detailed
We’ve covered the basic process of creating information chains before in Data Quality Pro and they are pivotal to root-cause analysis.
Armed with an information chain, you will have a simple yet effective tool when carrying out your tracing activities (see below).
They do need to be well maintained, sufficiently detailed and continuously monitored to help you analyse the data flows however so don’t treat information chain creation as a one-off exercise.
Tip: By monitoring information chains regularly you will create an early-warning system that can identify when potential causes are impacting your business.
4. Include external data sources in your root-cause investigation
We can take all available measures to ensure defect-free data in our organisation but ultimately if data is flowing into our business from 3rd parties this can pose a weak spot.
Ensure you have an effective supplier management policy in place complete with service level agreements that continuously monitor and report defects.
We need to monitor these external flows regularly so we can alert the business when a breach in the SLA has been found.
Tip: Don’t be territorial over these monitoring processes and reports, share the logic and the approach with suppliers so they can implement pre-delivery checks. This creates a win-win scenario as it will help them improve satisfaction ratings and reduce penalty charges not just on your agreement but their other clients.
5. Use the 5 Why’s to discover multiple causes
The 5 Why’s root-cause discovery process is fairly well known but is covered here for clarity.
It is typically used when interviewing those who work along the information chain to investigate potential root-causes.
The interviewer simply asks “Why?” a particular condition arises, in an iterative fashion.
One question people occasionally raise is “When do we stop asking why?”.
This is very subjective but I typically halt the process when I derive at a cause that the organisation is sufficiently equipped and resourced to tackle.
Tip: Don’t just rely on “why?”, also ask the what/when/where/how type questions to, it will give a much richer analysis.
6. Develop a root-cause analysis process that is commonly adopted across the business
If you read 5 different books on root-cause analysis you will almost certainly discover 5 subtly different approaches.
It is important that your business has one process that is clearly defined and documented.
This should become a part of your data governance policy so that when issues are found a systematic process can be implemented.
For a basic process you can use a combination of the following headline activities:
Gather data quality profiling and analysis results
Create or refine the information chain
Gather anecdotal evidence from knowledge workers
Root-cause tracing
Root-cause determination
Define the problem clearly
Brainstorm and workshop the potential causes
Validate the root-cause using data analysis
Prototype a preventative solution(s)
Implement and monitor
7. Learn how to perform Pareto analysis effectively
Pareto analysis is commonly referred to as the 80/20 rule. It is a simple tool for helping us to prioritise the entire root-cause analysis process by focusing work on the most critical areas with higher impact.
A pareto chart is a simple bar chart in which the horizontal axis (x-axis) represents categories as opposed to a metric scale. These categories are typically defects, errors or causes of defects.
The height of each bar (y-axis) can represent a count or percentage of defects or perhaps their impact in terms of cost, delays, rework etc.
By arranging the bars from largest to smallest we quickly see which categories will create the biggest gains if resolved. It also helps us to eliminate the issues that have little impact on the business.
Tip: The most frequent problems may not have the biggest impacts in terms of quality, cost or time. Construct two pareto charts, one using counts (eg. defects, frequency) and one that looks at impact (eg. cost/time). You can now identify the impacts that occur the most frequently and have the biggest impact.
8. Learn how to construct a fishbone or ishikawa diagram
Fishbone diagrams or ishikawa diagrams are one of the most essential tools to be used during the root-cause analysis process and are typically deployed in a brainstorming workshop.
Here are some of the key benefits of a fishbone diagram:
They help teams push beyond the perceived symptoms to uncover potential root-causes.
They provide a much needed structure to any brainstorming activity
Ensure that major causes are not ignored by allowing full participation of the personnel involved
Focuses on the human and environmental side as well as the more technical which often starts out as the prime candidate for investigation
They are extremely useful for implementing cause prevention even when no issues exist, not just cause analysis
Mind-mapping tools have emerged as one of the most effective and simplest tools to use when creating your fishbone diagram. There are now online versions which facilitate remote workshopping plus there are also free products available, just search on google.
To construct your fishbone diagram you must first start with the head which requires you to clearly define the problem.
Don’t make this too expansive or open-ended, clarity is the key so that the brainstorming remains focused.
Next populate the chart with some common categories but don’t be afraid to add your own as the brainstorming process throws out new ideas.
Here are some common category ideas:
Manpower (basically people issues)
Measurements
Machines (think applications, systems etc.)
Materials (documentation, training guides etc.)
Methods (eg. business processes, standard activities, sequences of events etc.)
Mother nature (eg. environmental issues)
Gathering causes. There are two ways to gather ideas for causes. Firstly, go through each category and ask the workshop attendees to brainstorm causes that can be attributed to that category.
Secondly, ask each attendee to brainstorm their own ideas separately, this can even be done anonymously if there is some sensitivity with the exercise.
Tip: Post-it’s are a useful tool for this or have your mind mapping software online during the exercise.
Once you have exhausted all possible root-causes then review the chart to eliminate the causes that don’t apply.
Next, investigate the causes you plan to investigate further and create a plan for confirming that the causes are indeed the root-causes of the issue.
Tip: Pull back from implementing or recommending fixes until we have gathered more data and performed validation.
And here are two additional bonus techniques…
9. Validate your root-cause assumptions with further data analysis and investigation
There is often the tendency to assume that the brainstorming process will define a list of causes that can be resolved.
It is true that you will often know by “gut-feel” that you’re close to the real issues following the workshop session but you still need to validate your causes.
Obtain additional data surrounding the issue if possible and ensure that you are not omitting other causes through incorrect assumptions.
10. Monitor any improvement or preventative measures implemented
The reality is that implementing a solution to eliminate defects may in actual fact “break” other downstream processes and ultimately damage customer satisfaction.
Over time the business and IT communities may have developed specific workarounds or quick-fixes to handle the defective data and by resolving the root-cause of the issue you can in fact cause serious downstream impact.
The key is to prototype and test any improvement solution to find obvious issues and then implement a close monitoring process to assess any impacts that may result elsewhere in the business.
If you have created your information chains correctly you should have a list of downstream data stakeholders who need to be informed of the planned improvement so ensure that all knowledge workers are updated with the planned changes and monitor any customer impacts rigorously until the solution has been bedded in for some time.
If you have an ongoing data quality management and reporting process in place this should just be another data quality rule that is routinely monitored.