The Big Data Bamboozle

A recent survey by CompTIA determined that a majority of companies are "not getting enough value for their investments in big data because it's difficult to analyze all the information in real time, much less convert findings into actionable intelligence."

There are numerous avenues of comment to pursue on this subject, but the one I keep coming back to on this blog is that "Big Data" is nothing new for direct marketers/merchants, who traditionally have taken a very different approach to it than the ones currently in vogue. That different approach, which I have detailed in a previous post, is essentially to use the data related to customer and prospect activity to create customer segments with designated RFM scores and segmentation psychodemographics, rather than trying to locate where each individual should be placed in some kind of multidimensional grid of massive relevant variables.

The reason for the practical approach used by "traditional" direct marketers is that they want "actionable" data (whether it's big or not), and you can't really take action when you have more than, let's say, five to ten rankings. I'll stick with five - A through E. The A listers are your best customers, the E category is the least profitable group. When you test five offers against these five groups, you are dealing with a grid of 25 (5x5) scores, which is more than enough to identify which offers work best with which demographics and purchase histories.

The Tangled Web of Data
A lot of the so-called Big Data is tracking data based on a person's online browsing activity. You can generate a ton of this stuff in a hurry, and in the mistaken belief that it will yield value, subject it to all kinds of analysis. But in the long run, all you really need to know is whether elements viewed eventually led to a purchase, or not. Of course, if "intrusive" promotional activity based on the interim data is effective in conversion to a sale, then the real-time analysis is theoretically worth it. and the number-crunching can thus be cost-justified. This is what is most often assumed when referring to Big Data, but I think it needs to be specified better. The Big Data monster lives on the Web, and not in the entire world of marketing as a whole.

Of course, any real-time interaction, such as behavior in a retail store, could also get entangled. But to explain why much of this is probably "overkill" I will use an analogy with biofeedback. A person can, if they wish to, monitor all kinds of physical functions about themselves, from pulse rate to oxygen level, using wearable gadgets. If you permit yourself to be hospitalized, you can even monitor a lot more with bedside hook-ups, or you can have very frequent blood tests with a slew of panels for chemical analysis. And you can get your genome mapped, and do frequent breathalyzer tests, and have alerts for lead poisoning and UV exposure ... and the list goes on and on.

What for? If you are reasonably healthy, there is absolutely no point to doing any of this! Even when you are exercising, how much feedback do you really need? For aerobic exercise, there is basically one simple test: are you perspiring, or not? If not, you're probably not working hard enough.

As a side note, a lot of the profiling data that marketers use comes from ZIP Code or Census Tract data that is derived from data maintained by the U.S. Census Bureau. This is a form of Big Data, in breadth and scope, if not in depth (and for a percentage of the data, the depth is definitely there, too). But here again, marketers simplify the data by aggregating it into a range of ZIP Code types (a dozen is fine), and this gets factored into the customer or prospect's demographic "score."

So the KISS principle (Keep It Simple, Stupid!) applies. And so it should. If I assign a customer to one of five rankings in a segmentation spectrum, and test that assignment with a few offers, the customer will either move up or down on the scale, or stay where they are. There is nothing very difficult about that. And that's all you need to know. It's as John Keats wrote in his Ode on a Grecian Urn: "Beauty is truth, truth beauty,—that is all ye know on earth, and all ye need to know."