Big Data vs. Big Insights: What to do with the Data.

While enterprises have all this BIG data in exabytes and zettabytes of storage, what to do with it remains a quandary.

In an era where Big Data is the new big thing, it was refreshing to hear some interesting thoughts during a recent briefing by Gartner and Dell (see link below) by Praveen Asthana VP  Enterprise Strategy and Marketing at Dell, and Cameron Haight VP at Gartner Research.

While enterprises have all this BIG data in exabytes and zettabytes of storage, what to do with it remains a quandary for a lot of customers we have talked to. In the briefing Praveen, VP of Enterprise Strategy at Dell, makes an interesting distinction between big data and big insights. You can get big insights from terrabytes of data so it is not about the size of the data, but the size (or quality) of the insights.  He is referring to solutions aimed at mid-market enterprises, but this viewpoint is a useful way to tackle the Big Data elephant for larger enterprises too.

The issue is an important one for us as a market research firm here at Silicon Valley Research Group where data collection is a key component of what we do. Where does big data fit into the customer insights data collection world? Here are some thoughts.

1. The most impactful big insights from big (and small) data for companies will be those that are customer facing. The implementation of these insights have a direct impact on bottom line scorecard attributes. Other great applications have emerged too- as reported earlier this week in the Wall Street Journal , other applications include predicting and therefore reducing employee turnover by mining data in employee records

http://online.wsj.com/search/term.html?KEYWORDS=big+data&mod=DNH_S

2. Big data collected by enterprises significantly enhances data collected from primary customer research though customer satisfaction surveys, brand audits and similar sources. The perfect data set here is a triangulation between three data sources: primary customer input + big data + social data. The process diagram below illustrates how we have been optimizing customer insights from these sources for our clients.

3. To get the best big insights will require new skill sets-data scientists who can mine and make sense of the data. A mid-market company I recently met,  was struggling with justifying the hire and unsure of exactly what skill sets to focus on, given that they could only hire one data scientist. Market research firms can, and I predict will, add significant resources so that they can service those needs for clients who cannot justify hiring internal staff. This will alleviate the problem for many mid-sized companies like the one mentioned above.  Many of the forward thinking market research firms have already been triangulating social data with primary customer data from surveys (see our Social Sentiment Sleuth™ at www.siliconvalleyrg.com. The addition of big data mining performed by concurrently with data analysis from primary and social data sources will provide much more powerful outcomes for you than big data mined for insights in isolation.

For full Gartner breifing on Big Data, Fabric Computing and other major IT trends, see: http://www.itbriefingcenter.com/programs/gartner_1364_dell.html

Please share your thoughts on data mining and how do you make sense of "Big Data" in your company .