Data Quality – The Inadvertent Oxymoron


It boggles my mind.

How can a unit/team/group have processes that either flat-out cause data quality problems or enable them by turning a blind eye, yet have budget and resources assigned to fixing up the mess after the fact? How can a phenomenon such as this occur? Is it because the organization is new at Data Quality and their idea of resolving it is to apply re-active fixes because they don’t know any better? I DON’T THINK SO!

I think they know better. I think they know better but they also know that resolving it will require Data Governance which means organizational change. Big Change. And discomfort. Big discomfort.

So here is my question then. How can our leaders, strategists, financial analysts and auditors present their budget figures for the year and get approval for this inadvertent Oxymoron of processes?

Just sayin…


The iaidq Blog Carnival


Each month the IAIDQ (International Association for Information and Data Quality) asks the data quality blogging community to submit their blog posts for the El Festival del IDQ Bloggers.  A different blogger volunteers (or you might be asked like me!)  to host the blog posts along with a brief summary of the submissions on their own blog.  I am very proud to be asked to host the latest event. 


First up is Henrik Liliendahl Sørensen, a Data Quality, Master Data Management and Data Architecture professional who lives in Copenhagen. Henrik is always one of the first (could it have to do with his location do you think 😉 )  to start the #FF’s on twitter and is a great commenter on the blogs of others. One of the things I noticed about Henrik (besides his gentle sense of humor) is that he is always very interested in learning about how things work (differently) in countries around the world. Henrik shares this interest by pointing out the differences in his blogs and comments and he also takes the time to identify the countries of origin when he tweets his #FF’s. I think it’s pretty cool too so thanks Henrik! 

Be sure to read Henrik’s submission called “What are they doing?” , a comment provoking post on the subject of  “assigning values for your customers/prospects industry vertical (or Line-of-Business or market segment or whatever metadata name you like)”.  Don’t forget to have a look at the rest of Henrik’s excellent posts and of course you need to follow @hlsdk on twitter! 


Next is Steve Sarsfield, Data Quality/Data Governance evangelist and author of the highly rated book The Data Governance Imperative . Steve is a popular tweeter, public and webinar speaker, white paper author and youtuber (What? podcasts Steve?.. :P) on the topic of data quality and data governance and has a sizable following due to his experience, insight and passion for all things Governance. Steve submitted an excellent post called Change management and Data Governance on the topic of  the parallels between change management and data governance. Be sure to read Steve’s post and the rest of his blog  Data Governance and The Data Quality Insider and don’t forget to follow @stevesarsfield on twitter. 


Next on the list is one of my new twitter friends Ken O’Connor. Ken is an independent Data Consultant located in Ireland, who specialises in helping organisations satisfy the Data Quality / Data Governance requirements of compliance programmes such as Solvency II, BASEL II, Anti Money Laundering, Anti Fraud, Anti Terrorist Financing, and Single Customer View – classic Master Data Management challenges. Ken submitted his blog post on the “Ryanair Data Entry Model“, a recommended model of data entry used by most low-cost airlines, where customers take care to ensure that each piece of information they enter is correct – because it matters to them! Be sure to check out the rest of  Ken’s Blog: Ken O’Connor Data Consultant, and follow him on Twitter here: @KenOConnorData  



It wouldn’t be a DQ blog carnival without including a post from Daragh O Brien, former IAIDQ publicity director, founder member of the IAIDQ, and  leader of the IAIDQ’s community in Ireland. Author, blogger, and an independent consultant with Castlebridge Associates, Daragh has been an active member of the International information/data quality community since 2004. Daragh is hugely supportive of sharing expertise, and is (obviously ;)) great at encouraging bloggers such as myself to take on a participatory role in the DQ community. Check out Daragh’s great submission: The Who, What, How and Why,  which describes the simple need for organisations to be able to answer the what/why/how and who questions about the information that fuels their business. Check out the rest of Daragh’s The DO Blog and be sure and follow @daraghobrien and @IAIDQ on twitter. 


Initiate, an IBM Company, submitted a couple of blog posts from their Initiate Mastering Data Management corporate blog. 

Larry Dubov,  internationally recognized expert in MDM and CDI and an author of over eighty publications wrote a three part series on data quality metrics: Measuring MDM and Data Governance Success, Defining Data Quality Metrics: Uniqueness, Completeness, Latency & Consistency and More Data Quality Metrics: Standardization, Availability, Adoption and Reference Data.  Dr. Larry Dubov is Senior Director and Partner of Business Management Consulting at Initiate, an IBM Company. You can check out Larry’s complete profile on LinkedIn:

Ian Stahl’s The Business Data Steward: A “Kaizen” Approach to MDM hits on a lot of topic areas around the empowerment and investment of business stakeholders around data management and Ian received some great comments.  Ian is the Director of Product Management for Enterprise Solutions at Initiate, and in that capacity oversees Initiate’s offerings in Financial Services, Manufacturing, Insurance, Retail, Hospitality and other segments of the Master Data Management market. Access Ian’s LinkedIn profile: 

I couldn’t find twitter accounts for either Larry or Ian so let me know if I’ve erred. But never fear, you CAN get all the scoop on these stories and more (including a great sense of humor and some biting insight) by following one of my favorite tweeps, @Initiate.   


The last post is by the infamous Jim Harris, Obsessive Compulsive Data Quality thought leader, blogger, tweeter, philosopher, vicarious reader, lover and quoter of sci-fi, Pixar movies and just about every good line from Battlestar Galactica! Jim, who resides in Iowa,  has over 15 years of professional services and application development experience in data quality (DQ), data integration, data warehousing (DW), business intelligence (BI), customer data integration (CDI), and master data management (MDM). Jim is also an independent consultant, speaker, vlogger and writer, and although I have never met Jim personally, I think of Jim as one of my very best virtual friends :). Jim submitted 6 posts (out of the 21 he wrote in August) and I urge you to read them all, as they provide value and insight into the perceptions, challenges, recommendations, best practices and (of course) philosophy of all things Data Quality related.   

Get Zippy with Selling the business benefits of Data Quality 

What came first, the Data Quality Tool or the Business Need?   

The Road of Collaboration   

The Real Data Value is Business Insight 

The popular (and one of my fav’s) Some is not a number and Soon is not a time   

And The Fifth Law of Data Quality, Jim’s DataFlux Community of Expert’s Post. 

Check out the rest of Jim’s OCDQ Blog and be sure to follow Jim as @ocdqblog on Twitter! 

Thanks to everyone who submitted their blogs, a truly amazing selection of great stories and information! And for the rest of you, don’t be shy! Anyone can submit a data quality blog post and experience the benefits of extra traffic, networking with other bloggers and discovering interesting posts and new ways to tackle data quality issues.


Informal Data Governance?

Can data governance be informal? Is there such a thing? Doesn’t formalization make up a key (and critical) component of Governance? As in; formal roles and responsibilities, formal processes including escalation, formal decision bodies, formal communications, templates, etc etc..
Isn’t informal data governance not data governance at all, but a hugely expensive and time-consuming (and mind-numbing…don’t forget to add mind-numbing…) approach to getting people to buy in?
Just sayin….
What do you think?

Change Management, Communications, and Stirring up Stuff

Yesterday I tweeted that I was going to lay low for a bit because I had stirred some things up. Without going in to too much detail, I commented on an internal blog post that referenced a big corporate change that I didn’t believe any real change was going to occur as I had not seen any modeling of the desired behavior.

Well, did that set off a flurry of activity! And of course that was the whole point of my comment right? And I only said what everyone else was thinking, as many people came to thank me.  What happened next is that my feedback was publicly acknowledged, I was thanked for speaking out, and I was even asked to help with the change effort.

What I want to know is, have you ever seen this happen?  A big change is initiated, there is a robust series of important communications, yet the expected behavior is not modeled for others to follow. I am somewhat surprised that this happens as often as it does. Just do a quick search on google and you will find all kinds of references to instances of this happening. You will also find that in most cases one of the most frequent recommendations for implementing a SUCCESSFUL change, is for the leaders to show others what this change looks like. They need to lead the way by doing what they say they are going to do, and asking others ” how can I help you” ?

This is where my #creative or #insane brain starts to churn. What if our Communications Plan had a section on the modeling of the behaviors? Included would be a description of what the modeled behavior needed to look like, who should do it, how often, what the expected results are and the success measures? Easy peasy.. it’s not rocket science. And then…well who needs a change management plan right? 😉

Just sayin….

It’s not just about the data…

I was sitting in a traffic jam yesterday and there was nothing on the radio so my mind drifted and I started thinking about a couple of things that drive me..that I am passionate about, that I think are HUGELY important. There’s quite a list (ok stop rolling your eyes, it’s not that big) but here are the top three:

  • Communication. It should be in the format of the recipients choice, there should be a feedback mechanism and it should be frequent and comprehensive.
  • Feedback. I only know about 3 people who are really good at giving and receiving it. Everyone else…I never see it. How hard is it to tell someone you really liked their story, presentation, communication, whatever? They will LOVE you for it!
  • Information Sharing. It makes me so grumpy when people don’t share their information. And when they use the excuse because “it’s not yet final”, or “it hasn’t been approved yet” I have a really hard time hiding my body language so it’s not blatantly obvious that I think they are neanderthals…

So I am mulling these things over while sitting in traffic and I’m listening to a news story about a small group of ‘green’ keeners who have started a new thing called ‘Trash parties’. They invite people into their homes for good food and conversation, and then the host brings out the garbage for others to poke through and make suggestions on how they can be better at recycling.  At this point I’m thinking about that moldy three-week old chicken I found in the back of the fridge and tossed in the trash and wondering if they clean out their trash before the company comes. Kind of like those people who clean their house before the cleaning lady comes.

Anyway, I was mulling this over when my mind drifted back to the information sharing peeve of mine and thought what if…what if we did kind of the same thing with our information? We all have tons and tons of information in our personal private folders. You know those 3 versions of documents that are still in draft format? The important emails that house decisions that we have saved…the PDF’s that house industry knowledge? What if we invited our colleagues to poke through our information to see if there was something there that would be of value to them?  I can think of a couple of benefits to this:

  • Like the green keeners, we’d probably do a quick scan first and get rid of the triplicate versions of the same document.
  • We’d also remove some of the industry knowledge white paper stuff that is out of date (I’m sure I’ve got white papers from 2004 on the magic quadrant for CRM solutions).
  • All this pre-sharing information clean-up would help free up some much-needed server space.
  • And our colleagues might find some tidbit that we thought nothing of but could be something really important from their perspective!

The result of all this could be that those of us who don’t like to share might get a little more comfortable sharing information. And maybe, just maybe, we all start to have a better understanding of why all those silos of data and information might not be a good thing.

Ok, so maybe it is all about the data..

The Data Governance Journey – Part 1- Getting Started

Our organization is finally (YAY!!) embarking on a Data Governance program and I’d like to share the saga journey with you. If you are unfamiliar with my story here are the basics:

  • Due to a CRM initiative there was an agreed upon corporate need for Data Quality
  • I was asked to lead the development of the program and establish the team
  • The business functional areas agreed that data quality was needed but were not in a position to sponsor or champion the initiative
  • The program and team had executive IT sponsorship
  • In 4 years the team established a solid program and raised considerable awareness, but due to lack of business sponsorship or a champion much of the work was re-active.

Now, due to a major data related initiative, the corporation has once again agreed that data quality is an issue that needs to be resolved in order for the initiative to be successful. But what’s different this time is that the agreement was formalized.

How we achieved this

A series of workshops were established where those who had agreed that data quality was a critical success factor for the corporate initiative were asked to attend. The workshops were a series of facilitated meetings where different stakeholders were asked to identify goals and objectives, roadblocks and risks and what are the things they felt would enable them to achieve their objectives. The outcomes that resulted were varied but what was obvious were the common themes around their data;  ‘accountability’, ‘establishment of formal policies and compliance processes’, ‘a senior cross functional decision body’, and ‘Training and Communication’, all components of a Data Governance Program. Success measures were also a part of the workshop and agreement was reached on those as well. The final workshop was used to confirm what was agreed upon, what the success measures were and recommended next steps.

Key Strategies

  • Use cross functional workshops to formalize the common themes stakeholders have already informally agreed upon.
  • External consultants help facilitate the process by using subject matter expertise to guide the alignment between business and IT.
  • Always be sure to identify agreed upon success measures.
  • Once agreement is achieved confirm, confirm, confirm the outcomes and next steps.
  • Document the outcomes, agreements, next steps and communicate to the organization.
  • Be sure to include the process to establish funding as a next step.

The Challenges

Most of the stakeholders that were needed to participate are mostly at the senior level and so their availability was a challenge.

What we did

  • Reinforcement of the verbal stakeholder agreements, the corporation’s business objectives and how quality data will help achieve those objectives really helped. Once the first workshop took place everyone was keen to continue and so it got easier to obtain their participation in later sessions.

That’s it so far….I imagine things may get a little more interesting as we get into the development of the program. 

Be sure and tune in for the next episode: Part 2 – Overcoming (the first set of) obstacles.

The role of a Business Analyst in the Management of Information

A new article showed up the other day on our organization’s internal wiki. The topic was ‘The Role of a Business Analyst (BA) in an Agile project’. The content of the article is just starting to be developed, so I took it as an opportunity to add a comment in the form of a question on “the role of a BA in the management of our information as an asset”. My goal was to generate some discussion around this idea, to see what others in the organization are doing/thinking, and to start the process to perhaps develop some formalized responsibilities in this area.

No sooner had I hit the save button when BAM, there was a response that showed interest in the topic and asked me to elaborate! Exactly the response I was hoping for! Muwah ha ha! You probably should know that the person who responded is a very good friend of mine who is quite comfortable with social collaboration and if it wasn’t for her I probably would be having a discussion with myself…ok it’s not really that bad but why is it that some colleagues have no problem updating their facebook status daily but when it comes to collaborating in a wiki they are running for the hills??  Sigh…subject for another day…

So what is the role of a BA in the management of our information as an asset? A  lot may depend on how your organization is organized, what the culture is, what types of resources you have ( maybe you have BA’s solely responsible for information management) and how mature your IM (information management) practices are. But let’s say for the sake of this opinion that you have a somewhat immature organization from an IM perspective, and you are just starting to organize Data Governance and MDM initiatives. This could mean that you are just starting to develop governing principles and operating policies around IM, and you may not yet have them ‘baked in’ to how you manage and deploy projects. Since some resources within an organization may not be familiar with some of the concepts, my approach will be to describe some of the key outcomes that result from not-so-good information management practices, and how a BA might help to resolve them, or ensure they don’t happen to begin with.

Multiple sources, conflicting rules and differing business uses of information

In order to resolve these types of issues you need to link the data to business objectives. This is standard stuff that I don’t often see. I’m not talking “here’s the data I need for this report”. That is not a business objective. The objective should describe the business decisions or triggers or changes that are the intended result.  e.g. This piece of information is used to determine what type of industry our customer is in so that we may ensure that the best person is assigned to them to help them with their needs. When the business objective is clearly linked to the data, it helps IT resources determine how to best architect the data, and business leaders with the information that they really need. It also helps in the resolution of conflicting rules. E.g. while one unit may use the information to identify the appropriate account manager, another business unit may use the information to identify the total amount of risk they have in that industry so that they can ensure they follow regulatory rules. You cannot resolve conflicting rules without having a good understanding of what the information is used for.

Lack of common understanding of what the data means

Ensuring the data has a clear and easy to understand definition and communicating it broadly reduces errors when data is created, transformed and reported on. Eg.  ‘Address’: describes the street location where the company physically resides. Although it looks somewhat self-explanatory, unless it’s clear and understood, what usually happens is you get all kinds of addresses: the mailing address, an email address [I kid you not], a city, etc..  You cannot assume that everyone has the same understanding of what a piece of data means.

There is a problem with the data but no-one seems to care

Identifying who will be responsible for business decisions on the data (and describing what that responsibility is) will ensure you don’t waste time when something needs to be resolved. This need not be a steward of the data as the organization may not be set up that way, but it could be a person or a role, and it should be written into the requirements that there is someone somewhere responsible for making business decision. E.g. The person (or role) responsible for making decisions regarding this data is XXXX. You may find that you are having difficulties getting someone to actually agree to having their name or role written down and being responsible. If this happens, don’t give them the data. Simple as that. No name, no game!

The data is in a mess (still….) and needs to be cleaned up!

Cleaned up to what? Including quality attributes as part of business requirements ensures that everyone understands how good the data needs to be and allows you to better manage  keeping it that way. E.g. The address information must be updated at least one per year to ensure freshness. It cannot contain phrases such as ‘unknown’ and special characters are not allowed. Using the quality requirements allow you to either perform an analysis of the data to understand how bad (or good) it is, and you can also provide the business with a weekly, monthly or yearly report on the quality, so that they can determine if it still meets their needs.

Here’s the summary of my opinion on how a BA can help in the management of our information as an asset

  • Link the data to business objectives
  • Ensure the data has a clear and easy to understand definition
  • Identify who (or what role) is responsible for business decisions regarding the data
  • Identify quality requirements for the data

This is by no means a comprehensive list, but it does go a long way to helping manage information as an asset. And it’s easy peasy stuff! The hard part…ahhh…the hard part.  Try telling someone a report is not really a goal and see what happens. Not really a good approach. The best way to go about it? Keep your rule/definition/goals antenna on at all times. Although it may seem hard to extract this information in a formal setting, most business stakeholders talk about this stuff all the time. In fact, that’s all they talk about! We just need to be really good at hearing it!

The role of a Business Analyst in an Agile Project while taking Information Management best practices into consideration (whew!)? That is a story that I hope we are beginning to work on!