dq:view - Steve Tuck on Data Quality

dq:View has moved...

Hi folks,

Thanks to everyone that has been following dq:View, but I've now taken the decision to move the blog to Datanomic's corporate site. I hope you'll continue to follow my witterings at http://www.datanomic.com/category/resources/blog/ or via an appropriate feed; RSS or Atom.

You can also find me online on LinkedIn as http://www.linkedin.com/in/SteveTuck, on Plaxo as http://stevetuck.myplaxo.com and if you're into Twitter, you can follow my Tweets at http://twitter.com/SteveTuck.

All the best,

Steve

p.s. There are already 2 new entries for you to read on data quality related topics at http://www.datanomic.com/category/resources/blog/.

Trillium Software's Identity Crisis

They say that imitation is the highest form of flattery.  I thought someone was pulling my leg when I first heard about this, but it's true - Trillium Software is currently paying for an advertisement on Google, that uses one word only - Datanomic!  Why would such a well established data quality software vendor make such prominent use of a competitor's name?  And why has Trillium singled Datanomic out for this special treatment?  I'll let you make up your own mind about that.  Meanwhile, here's a screenshot I just grabbed that shows the advert.

---

---

Feel free to Google Datanomic and click on Trillium's link - it takes you to the registration page for a White Paper, but if you want the real Datanomic, simply go to www.datanomic.com.  And Kevin, well spotted but no, this doesn't mean that Datanomic has been acquired by Trillium Software!  LOL

It Can Happen to Even The Best of Us

Did you know that I'm the secretary of the IAIDQ's UK community of practise?  No, I thought not - and you probably don't know what the IAIDQ is either, eh?

The International Association for Information & Data Quality was established in 2004 to "cultivate information excellence" and "help transform organizations and society, improving the quality of life everywhere."  Blimey I think that's the first time I've read the vision statement - big or what?  I really look forward to seeing improvements in the quality of life EVERYWHERE!!!

My main objective in joining the association as a charter member (and then getting involved in some of its organisational challenges) was to facilitate a forum for people struggling to improve poor data quality.  The association has had very limited success in doing that to date (it's less than 2 years old, after all) and I have to admit to finding the name itself amusing (Information and Data are both included and I'll leave the anoraks to debate the difference between the two), but one thing that's clear is that the founders and executive of the association believe passionately that poor data quality costs organisations money (and sometimes worse).

Recently I received this from the association:

It's my new membership card and the thing to note is the dates; the valid through date is the same as the date I renewed my annual membership, i.e. it expired immediately.

The important thing though is that the association recognises that errors can occur and it provides a process to identify and correct them.  Beneath the signatures of founders Larry English and Tom Redman, is a request to "please examine the information below for accuracy" and telling us what to do if we find an error. 

In this case, the mistake was very speedily rectified and I received a new membership card.  Poor quality data cost the association some administration, printing and postal charges, but they acted quickly to deal with the problem and, I've no doubt, resolve the process issues that allowed it to happen.

I should close by saying one thing to the IAIDQ - thank you for your commitment to quality!

The Wrong Guy - identity crisis at the BBC

BBC New 24's coverage of the recent court case between the Beatles' record label, Apple Corps Ltd. and the computer firm, Apple Computer, Inc. was as comprehensive and professional as usual - until they suffered a case of mistaken identify!

The corporation had invited the journalist and author of newswireless.net to express his views on the verdict, but somehow, while the real Guy Kewney (pictured right) waited in the green room, they managed to get completely the "wrong guy" in front of the cameras - the look of terror on his face when he was introduced was priceless!...

But I have to say, the poor chap muddled through the interview in fine style; if you'd like to watch it, here it is!

Perhaps there's a clue to the reason he remained so unflappable in this gentleman's true identity.  At first it was reported that he was a London cabbie, at the BBC perhaps to collect Guy Kewney, but the truth is better than that: it turns out that the man featured in the interview is Guy Goma, a business Studies Graduate from the Congo who was in reception because he was applying for a high-level job at the BBC.  Apparantly Mr. Goma assumed that the whole thing was part of the recruitment process, but was "a little upset" that nobody asked him about his own area of expertise.

And what is Guy Coma's particular area of expertise? I'm pleased to reveal that it's Data Cleansing!  Well Guy, I hope you got the job in the end, I'm sure the BBC could use your experience, but failing that there might be an opening on "Working Lunch"!

dn:Director - a fresh approach to data quality

Why do so many organisations turn a blind-eye to data quality?  One thing for sure is that the legacy data quality software providers have done little to help address this crucial business issue by delivering products that require years of expertise to successfully leverage all of the functionality available (and, just as importantly, to know when to use something else instead).  After a dozen years of working in the field, and having built a highly profitable consultancy business to help clients address this short-fall, I decided a year or so ago to join Datanomic.  I'm delighted to say that, last month, we celebrated the launch of dn:Director, a data quality product that is setting new standards for data quality management in the 21st Century.

I've been privileged to work on data quality projects with many leading, blue-chip companies over the years, but one of the things that struck me was that I was being asked the same questions by clients in 2004 as I was asking myself more than a decade earlier; they were identifying the same old deficiencies in data quality products and having to employ the same workarounds to resolve them.  Sure, the vendors have done something to smarten up the look of their software, but, under the covers sits essentially the same code that was initially developed for mail-room efficiency in the 1980's.

Two more things struck me:

1. All of the software vendors talked about delivering a tool for "business users" but the reality was that just about every project relied on the IT department to develop the business rules.
2. Because of the complexity of using the software to good effect, the cost and duration of projects was prohibitive; the reason I was working with so many blue-chip companies was that they were the only ones that could afford to undertake such major projects!

These were the things that motivated me to create Tranato and subsequently to join Datanomic in 2005 and bring together the two technologies under a shared approach.  Put simply, we feel that a data quality product needs to be much more accessible - you shouldn't need to be a software guru to get value from it.

dn:Director is the result of many years experience in data quality and data management; not just my own, but that of people like Gerry Kelley (Datanomic's VP of Professional Services) and his team, and the shared experiences of our clients and partners.  Taking Datanomic's approach (The Four Cornerstones) and methodology as its foundations, dn:Director has been built from the ground up, using the best-available modern technology.

Developing dn:Director in Java and using standards-based interfaces (such as JDBC, JMS and XML) has enabled us to deliver a technically advanced and extensible data quality product that supports both batch and real-time processes (providing data quality services through SOA).  But the thing that everybody notices first is just how easy it is to use - you should hear what out customers and partners have had to say about it:

"This is great - it's so easy understand and configure business rules"

"I love the way that you can build rules from the data - it's so quick and intuitive"

"This will halve the time it takes to deliver a project"

For more information visit Datanomic's website or call on +44 (0)1223 228400.

Note: I know this is very commercial for a blog entry, but given the amount of personal time, energy (and money) I've committed to making dn:Director a success, I hope you'll forgive me.

A Tesco store, a Drunk and a Credit Card - Data Integrity in action

Because I choose to live near Oxford, whilst Datanomic is based in Cambridge, I frequently drive between the two cities in the wee small hours.  So it came to be that I was doing some grocery shopping in a Tesco store at 1am one day recently when I noticed a kerfuffle at the checkout.

A man in his early twenties, and clearly worse for wear, was remonstrating with the cashier who had refused to take the credit card he was brandishing as payment.  "But I know the PIN" he said, "so why won't you **** take it?"

Nothing the man said, or did, could persuade the cashier to change; she sat there calmly and repeatedly pointed out that the card was issued to a Miss Susan Jones [not the real name] and that in her judgement that meant that it wasn't his.

"It's my girlfriend's" he protested "she gave it to me" and proceeded to call the said girlfriend on his cellphone.  I loitered nearby for a while until the cashier was joined by 2 of her colleagues and I was satisfied that she wasn't about to be physically assaulted.  By making that simple check of data integrity between the name on the card and the person in front of her, the cashier had prevented a fraudulent act - albeit one that had no victim (assuming the card was indeed the girlfriends and the drunk had her permission to use it).

Meanwhile, I sauntered away to the self-service till where, unsupervised, I scanned and packed my own shopping and then paid for it using my American Express card - a card that is not Chip and Pin.  I didn't have to sign a receipt and nobody even looked at my card, let alone checked that it was mine.  As I left the store I heard the cashier telling the drunk that the police had been called, because he had refused to let her confiscate the credit card. 

There's a lesson in this: if you're a drunk, trying to buy more booze with someone else's credit card - use the self-service till and help yourself!

Applying data integrity checks can help to protect your business, but they're of limited value if you only apply them to a subset of data sources.