Popular: Data Quality | Data Governance | MDM | Modelling | Big Data | Data Analytics | Public Sector | Healthcare | Telecoms | Finance | Technology
For detailed search please visit the Archives
When tackling Data Quality you will invariably need to understand and manage the thousands of business rules that can exist across even medium sized organisations.
To help members understand some of the core methods involved I recently asked Graham Witt, author of "Writing Effective Business Rules", to share some practical tips and techniques.
Read MoreMaster Data Management (MDM) has rapidly become one of the most in-demand skills within the data management industry.
One of the core skills that practitioners require to deliver MDM is the ability to construct and manage a variety of data and functional models.
To help members understand the techniques involved I recently interviewed international modelling expert John Owens of Integrated Modeling Method.
Read MoreHow do you manage Customer and Supplier entities on a logical model in a way that doesn't lead to data quality issues?
John Owens is a long-time contributor to Data Quality Pro and sits on our expert panel. He is an expert in business and systems modelling, creator of the Integrated Modelling Method and currently lives in New Zealand.
Read MoreIn this guest post William Sharp of The Data Quality Chronicle provides some practical advice for leveraging data modelling at the outset of a data quality activity.
In today’s world of increasing feature sets it is easy to become bedazzled by the latest instalment of new functionality. Data quality software is no different than other enterprise applications in this regard. However, perhaps more important than ever, it is critical that users of data quality softwareknowthe data they are analyzing in order to fully leverage the tool and truly deliver good quality analysis.
First things first
With all the advances in data quality software it is easy to become enamoured with the bells and whistles and thus lose sight of the fundamentals. As these rich feature sets enable the less technological oriented business users to participate in data quality exercises, the fundamentals of data analysis become more, well, fundamental.
One of my first questions when I start a new data quality project is, "Does anyone have a data model?”. Sometimes I get lucky and the DBA has one. Sometimes when I am really lucky the DBA also has a data dictionary. I don’t play lotteries due to my lack of being "really lucky”, if you know what I mean.
If I get a data model, I sit and examine it like a CSI agent does blood spatter. It is, after all, my roadmap to solving some mysteries. If there is not a data model available, I dig-in and create one. Most times I start out with pencil and paper (I know, how arcane!). Most data quality projects involve one or two main entities, the customer or a product. As such, I use this as my starting point.
For the sake of simplicity, let’s concentrate on a customer-focused data quality initiative. Commonly referred to as Customer Data Integration, or CDI, these projects involve the most critical person in any business; the customer! Customers are a complex animal, particularly from a data perspective. Organizations often focus on collecting as much data regarding customers as possible and rightfully so. As a result, there is usually a fair amount of data in, or related to, the customer entity.
What works for me is to start out with a big picture and narrow my focus with further analysis. My first picture step is to build what I call my "customer frame”. The customer frame consists of the customer entity and each entity to which it is related. In the figure below you can view some of my basic customer frame based on a typical instance of Microsoft Dynamics CRM (in pencil nonetheless).
What did I learn from the exercise? For starters we can see that:
The list above is just a simple example of how studying the data model translates into practical knowledge which is critical to the success of a data quality initiative.
One of the more subtle points I touch on in this example is the identification of entities in the data model that are not necessarily useful. This is particularly true of Microsoft Dynamics CRM. Microsoft Dynamics CRM provides an "extension” table for just about every table in the database so that organizations can define and store data unique to their enterprise. As such, this is where to look for data that is "near & dear” to the hearts of users.
I came across a prime example of this on my last project when I discovered that my client was storing a unique identifier in one of these extension tables. This helped me identify a code that was akin to a social security number for each unique customer! On a CDI project, data that uniquely identifies a customer is a treasure well worth the time invested to discover it.
While I am definitely one to "geek-out” on new features of my favorite analysis tools, there is simply no replacement for knowing the data. Learning the basics of a data model doesn’t take very long but provides valuable insight into an organizations "data state”. I highly recommend writing up a cheat sheet like the one in the figure above and keeping it close to you as you define your data quality cleansing, standardization and matching routines.
It may come as no surprise to many that when we find defective data we often find poorly designed business functions.
To create long-term data quality health it is imperative that every business understands the functions it should be performing as opposed to the functions and processes it is currently delivering.
Our aim with this tutorial series is to help our readers learn some of the key modelling skills that are typically required for data quality improvement.
Read MoreMaster practitioner John Owens, creator of the Integrated Modelling Method, is our guest expert for this webinar discussing techniques for business modelling and its key role in ensuring long-term data quality.
Read MoreThe ability to correctly identify the business entities and the way we want to model those relationships is pivotal to good information quality. Most organisations opt for physical modeling and create application specific schemas that often lack the high-level vision for how the business really needs to utilise its data.
In this post, senior consultant at EWSolutions, Pete Stiglich, presents an excellent account of the importance of Conceptual Data Modeling on ensuring information quality.
Read MoreI
In this post we speak to John Owens, the creator of the Integrated Modelling Method (IMM), a framework that combines all the modelling techniques that are so often lacking in organisations that are looking to improve data quality.
We recently caught up with John to find out more about his distinctive approach to modelling and how it can provide benefits to our goals of high quality information.
Read MoreThe online community resource serving the Data Quality, Data Governance and MDM community worldwide.
Get free software, tutorials, guides, training, event discounts and regular updates.
Data Quality | Data Governance | MDM | Technology | Data Migration | Big Data | Modelling | Healthcare | Finance | Telecoms | Public Sector | Telecoms |
Arkady Maydanchik | Larry English | David Loshin | Nicola Askham | Sunil Soares | John Morris |
