Posted on Thursday 23rd August 2012 9:42
Yesterday the UK government announced that it might force supermarkets and internet companies to release the personal data they hold on individual customers, in a shake-up of Britain’s data protection laws.
The move is part of a new initiative called Midata which has some pretty ambitious aims. It proposes to make it possible for consumers to request electronic copies of the “historic transaction data” that retailers, or banks hold on them. So far it has been cautiously welcomed by pressure groups. Under the present Data Protection Act, everyone does already have the right to request this information, but acquiring it is often a very slow process and results in the consumer receiving a thick envelope full of incomprehensible computer printouts.
Ministers for the department of Business Innovation and Skills (BIS) evidently hope that mandating companies to hand over this information in an electronic form will usher in a new era of transparency between businesses and consumers. If only it were so simple. BIS’s aims are laudable, but they make two dangerous assumptions:-
1. They assume that businesses habitually aggregate all the data they hold on individual customers. They don’t. Anyone who’s ever rung a call-centre will know that bits of information about your relationship with your bank or energy provider sit in various internal systems which often don’t talk to one another.
2. They assume it’s possible to take records from complex enterprise IT systems and export them in a file format that a consumer can open, much less understand. This is why people who submit Freedom of Information requests often get envelopes full of print-outs back in the post. Most enterprise-level companies run their businesses on proprietary IT systems which store data in arcane, structured and hard-to-read file formats. You might be able to extract individual records, but you can’t open them without the right computer program. The alternative is to convert these records into spreadsheets, PDFs or text files, which takes time and frequently reduces the usefulness of the data.
There is a bright spot on the horizon, however. The same big data informatics and analytics solutions that many companies are starting to use to mine their stores of data for insights about their customers could very well be re-purposed to good effect for solving this problem. Big data solutions are developed to mash up information from multiple sources and which has been stored in various formats and then display them in a comprehensible format, often in online dashboards. This is pretty much exactly what BIS is calling for.
The UK government is intent on driving through changes to Data Protection Act, it might even open up a market for these kinds of solutions which is much more urgent and focused than the Big Data market currently is. As the financial services industry can attest, there’s nothing like the government breathing down your neck with new legislation to drive a new round of IT spending. It’s just ironic that a technology initially developed to solve very big, general problems could almost by accident help solve a much smaller, specific predicament.