Thought Leadership


Subscribe to our blog

Your email:

Connect with Ness

Software Engineering Services Blog

Current Articles | RSS Feed RSS Feed

The Kirk and Spock Approaches to Big Data


Kirk and Spock

by Ron Miller
Ness Blogger 

There's a pivotal scene in the latest Star Trek movie. Without giving anything away, in a moment of crisis Kirk says he's going with his gut because it's all he has at the moment. In other words, he has no data on which to base a sound decision.

As a manager, you very likely have data -- lots and lots of data, but for whatever reason you might not be willing to make the transition from a Kirk company driven by emotion and your gut to a Spock one driven by information.

MIT professor and author Erik Brynjolfsson speaking this week at the MIT Sloan CIO Symposium in Cambridge, MA called that decision-driven model which itself is being driven by the Big Data phenomenon, a management revolution. And he didn't use the term revolution lightly either.

"Great revolutions in science begin with revolutions in management. New tools make it possible to change the way we think about the world," he said. Brynjolfsson said until Amazon came along, book publishing was a culture of conversations and lunches and gut feelings about which books would hit and which would fail. Then Amazon brought data into the equation and turned the industry upside down.

And typically the management of disrupted companies are the last to know. He says management has to shift from this idea of HiPPO driven decision making, which stands for the highest paid person's opinion drives the decision to one based on logic and data. Think about how Mr. Spock would come to a decision. The data tells a story and you should be listening, but also be aware it's not necessarily the whole story.

He told the audience, it's important to understand that correlation does not necessarily equal causality. MIT Professor Andrew Lo speaking on a Big Data panel yesterday said, there is no better example than this: Data has found that people with lung cancer most often have ashtrays in their homes. Therefore if you take away ashtrays, you'll reduce lung cancer. That logic is obviously flawed because the correlation is not the cause.

That said, there are many powerful examples of using data to get at information or even to affect behavior just because the information exists.

MIT professor Andrew McAfee speaking later in the day at MIT told the story of a restaurant chain where he applied data analysis to Point of Sale data and identified massive stealing going on by employees. Management took the report McAfee wrote and presented it to employees without comment or threats. They got the message. Just by understanding the process was being monitored and data analyzed, the employees cleaned up their act.

That's just one business example of how a business used data to affect employee behavior and improve the bottom line of the restaurants in the chain. 

In another example Brynjolfsson, said without any background or understanding of the market, he studied publically available search data on the housing market and was able to develop a model for predicting housing sales shifts 3-6 months before they happened --whch was more accurate than the more popular ones used by the industry -- and he did this with data.

Brynjolfsson said, "We need to change from hunches and opinions to go with facts and data." And that he says requires cultural shift. Are you ready to make that shift from the Kirk gut-driven approach to the Spock data one? Successful companies are going to be making that shift.

Photo Credit: Star Trek: Into Darkness

blog comments powered by Disqus