Mark Zielinski on Making Use of Big Data Now

Today, I have the pleasure to welcome Mark Zielinski, co-founder and former director at Winning Research in Toronto. He writes about analyzing social network traffic to better understand patterns and derive knowledge from them. Thanks for your contribution Mark.
Since late 2011, the market research industry and market research technology in general has been very focused on the coming rise of “big data”, and what that can mean for professionals in market research.  There has been all sorts of speculation about how the analysis of organic data and passive data “floating around” out there, such as Twitter, LinkedIn, Pinterest, and Facebook traffic could change the way we work very soon.  Companies looking to stay competitive can’t keep doing the same tired old things – they need to keep their ear to the trends, be resourceful, and come up with creative ideas.
The general consensus seems to be that “big data” is never going to replace traditional research – that is, specific research methods like surveys and focus groups that deal with particular topics will always be around.  These specific research methods answer the question of “what” – that is, they are concerned with empirical details.  For example, an online survey may indicate that compared to 2011, this year 5% of soda drinkers no longer drink Coca-Cola on a regular basis.  Where “big data” aims to change research is in the “why” – that is, the broad trends and underlying reasons why certain results have been obtained.  Using our previous example, “big data” may be able to tell us that key nutritional influencers have recently been saying that carbonated sugary drinks reduce life expectancy by an average of 4 years in healthy individuals.  By having both the “what” results and the “why” results, researchers can use this combination of data to have a much clearer picture of a particular situation, and potentially be able to advise their clients on how to act to obtain the results they wish to achieve.
Many research companies, especially the multitude of smaller research agencies, are in a tough situation.  They see these “big data” trends and recognize their importance, but being researchers and not technologists by trade, they wait for an emerging vendor that will fill in this technological gap for them and allow them to stay relevant.  While there are emerging solutions in the market that aim to provide a “one stop shop” for companies to get the big data they need, most are not tailored specifically to market research.  Some of the biggest demand for big data analytics comes (perhaps unsurprisingly) from the finance sector, where trending topics can mean instant influence in the stock market for investors and traders.  In many cases, subscription fees for these services are in the tens of thousands of dollars per user per month, and when you have clients who are still uncertain about the value of the service, it’s a tough expense to justify for many research agencies.
What’s the solution in the current no-man’s-land of big data for market research?  Should research companies just sit back and wait for an affordable, tailored solution to come their way?  Certainly, with enough time, it’s inevitable that somebody will fill that role.  However, at that point, the competitive advantage of being the “first” in that space will have evaporated.  Once there’s a solution that everybody is using, you will become yet another commodity among the other research houses or fieldwork agencies that use that tool.
If you still haven’t invested in a good developer at your company, now may be the time to do so, unless you have a technical, inquisitive mind yourself along with some extra time.  There are fantastic resources out there to begin your own foray into analyzing the constant stream of data in the online world.  Check out two books by O’Reilly publishing that will get you thinking: 21 Recipes for Mining Twitter and Mining the Social Web: Analyzing Data from Facebook, Twitter, LinkedIn, and Other Social Media Sites.  Although they both deal with the Python programming language, and as such require at least some basic background in computer programming, they contain a multitude of ideas and concrete examples that are very much applicable to the market research industry today.  What you start with may be very basic – such as just deriving the most popular trending topics in a given location – but even that will be an example of “big data” that you can use to enhance the results and analyses you deliver to your clients, however minor.
It’s easy to throw up your hands and say, “I don’t know what I’m doing”, and hope for someone to come along and give you what you want.  But the data is out there already, every minute of every day, and even if the value you get from it at the beginning is very minor, you’ll be able to give you clients something more than they’ll get from every other research agency that’s resting on its laurels.
Go out there.  Try something.  If it fails, try it again, in a different way.  Get help when you need it, but don’t let the world gradually pass you by until one day you wake up and realize you’re irrelevant.

Original Article