IBM makes big data easy for the little guy

Last month, theUniversity of Southern Californias Annenberg Innovation Labunveiled itsFilm Forecaster, a tool that tries to predict upcoming blockbusters by the amount of buzz around a movie on Twitter. Its not the only one of its kind; others like Boxoffice.com have also been tapping Facebook and Twitter data to predict movie success.

But whats interesting to me about the project isnt just the sentiment analysis applied to movie tweets, but how easy it was for Annenbergs Innovation Lab to implement it. The lab, which is sponsored by IBM, usedIBMs new Big Sheets application, which became available publicly last month as part of Big Insights, and has been applied to other projects like tracking the U.S. election and the Egyptian uprising. The film forecaster sounds like a big undertaking for USC, but it really came down to one communications masters student who learned Big Sheets in a day, then pulled in the tweets and analyzed them.

Big Sheets works like a big spread sheet and can be used to gather and analyze petabytes of unstructured data. Because it works with a familiar paradigm, its easy for people to use and can be partnered with programs like ManyEyes to visualize information. Thats what happened in the case of the film forecaster, which started in May and looked at tweets over a 24-hour period. The lab was able to gather between 250,000 and 500,000 tweets for each analysis and break them down into positive and negative messages using a lexicon of some 1,700 words. The student in charge was able to create a visualization a day later based on the data.

Ithink data analytics is incredibly important ! going fo rward, said Jonathan Taplin, director of the Innovation Lab. Tools like BigSheets allow us to do something very important sentiment analysis without using a lot of data-centric people involved in the project. Its incredibly easy to use and very efficient too.

What this shows is that with the rise of big data, were also seeing the emergence of really powerful but simple tools that can democratize data analytics and business intelligence. Big data wont necessarily be handled by just data scientists; it can be wielded by non-technical people. Thats a powerful idea, because it suggests a world in which we can all be data jockeys.

Rod Smith, VP of emerging technology for IBM, said companies are increasingly looking to mine the unstructured data from Twitter and Facebook, where their consumers are. He said with tools like BigSheets, its becoming easier for domain experts and business-side people to delve into data analytics, without the help of IT teams and data gurus. Analytics, he said, will become a core competency for workers as the tools become easier to handle.

It will become almost second nature; youll have ability to get information and do this type of sentiment analysis, Smith said. Someonewho is skilled in other areas and doesnt know the constructs of data will just know where to get it and get insights from it.

IBM wont be the only one offering simple tools for handling and visualizing data. Stacey recently wrote about companies like Tableau, Karmasphere and Microsoft, which are also building simpler tools for data analysis.

Decision-makers and everyday workers will also be on the frontlines of gathering and analyzing the data. And as more of this analysis goes real-time, any lags in drawing insight! s collap se. The promise of big data and these easy-to-use tools is in empowering more people to make fast decisions drawn from real data.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.