More data, more problems?

Posted on Monday 14th May 2012 12:38

Business IT publication, Computing, has just published some very interesting research that shows what a selection of UK IT leaders really think about big data. The results paint a picture wherein many organisations know there are significant benefits in making better use of their data, but lack the skills and systems to analyse it. Moreover, they often have no idea whether the information in their data stores is a goldmine of business intelligence or a dumping ground of old and irrelevant records.

According to Computing’s survey, 72% of IT decision makers recognise that their organisation’s data represents a store of hidden value. The vast majority of this data, said the respondents, is sales-related, with 76% of organisations stating they stored detailed customer records and 74% transactional information. 11% of respondents even admitted that they habitually stored every single scrap of data that went across their network. Organisations are also holding on to this information for longer and longer, with 32% of respondents saying they stored 90% of all collected data for more than three months.

Perhaps the most surprising finding of this survey, however, came when Computing asked IT leaders how their organisation planned to tackle the challenges posed by ballooning volumes of data. Here only a shocking 14% of respondents said their organisations were committed to acquiring the analytical skills required to make sense of this information, compared to 76% which simply planned to buy more storage space. Fewer still (6%) planned to buy more of the analysis tools that could transform huge, inert databases into valuable business assets in their own right.

Given these results it was therefore not much of a surprise to learn that 43% of IT leaders believed that a lack of analytical skills within their organisation was holding them back from realising the value locked in data, while 37% blamed analytical tools.

In all, Computing’s study paints an unfortunate picture of organisations treating potentially business critical data like last year’s Christmas cards. They hoard data on the off-chance that it will be useful in the future, but significantly under-invest in the skills and tools they would need to make it so. As why this could be the case, much of it could come down to cost, with 52% of IT leaders saying they did not have the necessary budget to analyse their data.

This is a shame on two counts. Firstly it’s short-sighted, in that only by learning to analyse data can an organisation decide what to keep, what to discard, and – potentially – determine the trends and market movements contained in its transactional data that could define its next strategic move. Secondly, it starts from the false assumption that the business intelligence tools required to make sense of huge data sets are i. expensive and ii. a cost instead of a potential store of value in themselves.

There has to be a better way of handling big data than buying more hard drives. And business intelligence is at the heart of it.