Monday, February 4, 2013

Big Data: Big Deal or Big Detriment

 tags: Big Data, direct marketing, data analysis

Perhaps I am by nature a skeptic or maybe it's the statistics courses and the years of direct marketing that make me onebut I am not yet convinced that big data will provide the benefit for direct marketing needed to justify its costs.  As a former database manager and data analyst, I enjoy the discussions of big data – how to store it, how to handle it, how to sort and analyze it more quickly and efficiently.  As a marketer, I am intrigued by the possibility of new analytics and more precise targeting.  

Big data refers to large amounts of structured and unstructured data combined into a final set that requires computer systems to be analyzed it.  What distinguishes Big Data from ordinary data is the extremely large amount of information in a variety of types and the speed at which it can (or cannot) be created, collected, and collated and analyzed.  (There is a cost for computer storage that is not discussed here.  Although this is very low for small amounts of data, it can become significant when discussing big data.) 

Every item of data has associated costs for collection, storage and updating.  Each item of data that is added increases the overall cost.  The acquisition cost might be low and updating minimal.  (A customer's gender might be obvious from the given name and gender changes are rare.)  Or the acquisition cost might be low but updating more costly, such as a customer's initial address and any subsequent address changes.  Or the acquisition cost might be high but the cost of updating low and so on through all the permutations.  Data needs to be updated or it becomes useless or even misleading.  People move, grow older, their tastes and interests change.  Obviously, this is particularly important if you are using the data to market and predict likely buyers of various products or services.

Each item of data should add relevant information.  For marketers, there is little or no point in collecting data that will not allow better marketing or customer service efforts.  The question then is how to determine what is relevant to better predicting buying behavior or to delivering the product or service.  The prediction of buying behavior can be done through analysis and testing.  Each additional data item can be ranked by what it contributes analytically.  As one moves from the items that provide the best indications to the items that are less predictive, each item contributes any increasing small marginal value.  Thus, an item low on the scale that adds little to the selection of likely buyers of a product can be relatively expensive in what it adds to the analysis even if its cost is low.  Obviously with big data, there might be a great many such items, each adding cost. 

Of course, there might be something out there that predicts buying behavior very well or affects it strongly but remains unidentified.  In assembling and analyzing big date, we might find that better predictor than the ones we now use or improves our selections in ways that justify the increased costs. 

One of the things that separates direct marketing from advertising and other forms of marketing is that it is results-driven, not just data-driven.   In relation to customer or prospect data, it's how you use what you have to produce better results.  More data may mean better selections or it might mean more costs with no added benefit.  Some data-driven disciplines have found that too much data does not improve, and can even decrease, the accuracy of predictions.  I think the results aren't in yet for direct marketing and big data.