Too Much of a Good Thing?

Just over a year ago I posted my loggest-ever blog entry (OK - a "blob" entry!) about the evolution of customer database analysis, from market-driven segmentation analysis 20 years ago to dossier-building individual targeting today. But in the last year or so, a tidal wave of data, driven largely by social media, has crashed onto the beach and washed away those dossier sand castles. "Big Data" has arrived, and it seems to be upsetting everyone's apple cart.

In the January issue of Oracle's Prophit magazine, Oracle VP Venki Rajah, head of Oracle's Industry Strategy and Insight team focused on business technology, writes: "The growth of high-volume, low-density data has outstripped the capacity to provision and operate the required infrastructure in an efficient manner. Enterprises are unable to handle unstructured data, limited by existing infrastructure and data architectures. A lack of enterprise-ready statistical analysis tools prevents the kind of analysis necessary to spot trends. Finally, executives are unable to make real-time analytical decisions because they don’t have a user interface that provides actionable information."

Rajah's prescriptions: better engines to capture unstructured data, better analytical tools, trend spotting via statistical analysis, and interfaces that help turn insight into action.

But however you slice it, it seems to me that the tide is sending us back full circle to the point where actionable market segments become more important than individual dossiers. Tracking any one customer's behavior across all channels and media is going to be less important than seeing the forest for the trees -- trend and pattern analysis -- so that the "outliers" (gifts one buys for others) don't become the significant pointers in cross-sells/upsells but rather the overall activity of the customer, compared to the overall activity (buying and commenting and reviewing and rating and ranting and returning) of all other customers that a company tracks, grouped into actionable "segments," becomes the critical dataset.

At that point, the privacy issue will go away, not like 20 years ago when the data itself was a proprietary competitive advantage, but because the data becomes a commodity, and the analytical tools and actionable insights build true value.

Like always, everything old is new again!