Thought Leadership


Subscribe to our blog

Your email:

Connect with Ness

Software Engineering Services Blog

Current Articles | RSS Feed RSS Feed

Actually, CIOs Would be Foolish to Ignore Big Data


6914441342 605f947885At the end of last month, Art Langer wrote a guest column in the Wall Street Journal called, CIOs Shouldn’t Let Big Data Rule Their Decision-Making. Wow, I couldn't think of anymore wrong-headed advice than ignoring data.

His point was that sometimes data doesn't tell the whole story. To illustrate this, he cited some experiences from years ago in which relying on data resulted in mistakes. He argued that we should not forget the human factor.

Langer has impressive academic credentials including being the head of the executive masters program in IT management at Columbia University, but telling CIOs to ignore Big Data is just crazy talk -- and not the kind of advice he should be offering future IT managers.

First of all, the examples he cited are really, really old. Today, CIOs have access to data stores that could actually give them much more insight than was possible in the 80s and 90s. And using that data, CIOs can make more informed and better decisions than ever before. To ignore that would be insane.

A recent report commissioned by the U.S. government called Demystifying Big Data (pdf link), described the value of big data this way:

"Hidden in the immense volume, variety and velocity of data that is produced today is new information, facts, relationships, indicators and pointers, that either could not be practically discovered in the past, or simply did not exist before."

And therein lies the flaw in Dr. Langer's argument. Surely bad decisions have been made in the past based on bad data, but ignoring big data because of those past errors doesn't make sense, not when the sheer amount of data available makes it much more likely that data-driven decision making will lead to a more positive outcome.

Of course, there is always going to be the human factor that Langer argued for, and the big data analysis tools available today are not necessarily ready to pull out those serendipitous discoveries the government report alluded to -- but that doesn't mean a man who is helping to shape the minds of tomorrow's CIOs should be so dismissive of it.

In fact, in a presentation at the Enterprise 2.0 Conference last June, author and MIT professor Andrew McAfee argued that if companies want to succeed they need to rely less on human intuition and more on actual data. Interestingly, Langer actually cites one of McAfee's books at the end of his article, but I believe he misses the key points McAfee and fellow author Eric Brynjolfsson were trying to make in their book, Race Against the Machine when he suggests they argued for the human factor.

McAfee said in his June presentation that we had to get past this notion that humans can somehow outthink computers or that it's up to managers "to steer the battleship." As he put it, "People need to get out of the way and have less faith in intuitive decision making and base [their decisions] on data."

McAfee recognized something that Langer missed and that's the fact that people can't possibly keep up with the volume of information out there today, and within that volume lies many insights, ones that it would be impossible for even the smartest humans to find -- at least quickly enough to matter.

Of course, there will always remain a human factor, and machines aren't fool-proof either. They are after all created and programmed by humans, but it's clear that big data, whatever you may think of its place in the current hype lexicon, has a very real role to play in today's CIO decision making, and those that ignore it, put their companies at risk -- and by extension, their own jobs.

Graphic by luckey_sun. Used under Creative Commons License.

blog comments powered by Disqus
Tags: ,