To Byte or Not to Bite: The Myths, Realities, and Trends behind the Science of Big Data Analytics

Without data, a company would never survive in today’s global environment. With some data, it might have a fighting chance, depending upon the quality and timing of the information.  But what happens when a company has access to too much data, sometimes referred to as ‘Big Data’? Ironically, it too could go out of business even with the best technology and staff to manage it.  Why? …partly because the data’s ultimate value depends upon who interprets and communicates the recommendations to the rest of the company, a task often left to an internal employee or ‘Data Scientist’ who may be no more than a recent university graduate armed with theories and little industry practice.  

According to Dr. Jesse Harriot, the Chief Analytics Officer at Constant Contact and author of “Win with Advanced Business Analytics”, “setting up a data analytics initiative within a corporation is not a trivial endeavor”.  It requires a lot of sponsorship at the corporate level and can take a year or two before achieving a meaningful balance between the influx of web data and its collective value to the company. Harriot shared his wisdom at a recent conference in Boston titled, “The Science of Marketing: Using Data & Analytics for Winning”. This power event organized by MITX, a Boston-based, non-profit trade association for the digital marketing and Internet business industry – (mitx.org), served up an impressive venue of expert panelists who shared their best practices and experiences.

Among them was a star performer, care.com, the largest online directory that connects those in need of care with care providers. Their co-founder and Chief Technology Officer, Dave Krupinski, discussed how the company uses analytics to drive all aspects of their marketing function including, attribution analyses, customer segmentation, user experience, and predictive analyses. As Krupinski explained to a packed room of 300+ professionals, “most CEOs blindly jump into ‘big data’ analytics expecting immediate returns, only to discover (and after great expense) the many intricacies required to get it right.”

Is ‘big data’ analytics really worth the trouble?

If economic times were healthier then maybe not, but with a slowing economy, companies are forced to either come up with the next differentiating product/service that will give them an extra edge over their competition or figure out better ways to surgically target likely buyers based on real-time data. But, increasingly, fickle-minded consumers whose loyalties remain largely unpredictable have made the task exceptionally challenging. …and yet, no one can blame consumers for their lack of brand loyalty when on average they are bombarded with over 500 ad messages per day.

A Typical Corporate Scenario
In a mocked up example for discussion purposes, a typical CEO hires a ‘Data Scientist’ or promotes someone from IT, after reading positive reports from companies that have boosted their sales using ‘big data’ analytics. Once budgets are allocated and a team is in place, software with funny names such as Hadoop, MapReduce, and HAWQ appear. These packages digest massive data sets (mostly unstructured data from the web) and respond quickly to complex SQL queries entered by a team operator or analyst. The output is then parsed into a more visual friendly format perhaps using expensive Business Intelligence (BI) software and when ready, shared at weekly management meetings. For this example, the meeting is adjourned without much warning. Management felt that the results from the Big Data Analytics Team were not aligned with corporate priorities, a common problem that points part of the blame on the Data Scientist’s poor understanding of managements business needs and on the CEO for not creating a comprehensive, formal data governance.

Disappointed CEOs tend to view ‘big data’ analytics as a ‘think-tank’ style department that delivers flawless dictates to the rest of the company, when in fact, ‘big data’ analytics should be a collaborative data-sharing effort among all departments.The secret of getting ‘big data’ analytics to work is less about massaging structured and unstructured data quickly behind closed doors and more about the timely reintegration of field data from every department to continually tweak predictions and outcomes.

What should a CEO do to encourage data sharing among departments?

Most department heads do not share their data with their cohorts either by choice or due to incompatibility issues.  To address this reluctance, a CEO should first explore a standardized database structure and data exchange format that would allow departments to share their data seamlessly. Next he or she should develop an incentive plan to encourage staff members to not only share their data but request data from others. The fewer restrictions imposed on inter-departmental data exchanges, the more likely, new ideas will blossom. Moreover, the positive behavioral changes in the workforce will help the data analytics team stay focused on corporate priorities. Keeping internal operations lubricated with both internal and external data analytics will boost a company’s revenues by default. This approach can lead to a passive revenue strategy that focuses more on balancing an operation guided by ‘big data’ analytics than relying on traditional consulting advice or CEO hunches.

A Five Stage Journey
I turned to a visiting professor at the Harvard Business School, Tom Davenport, to categorize the ‘big data’ analytics journey a CEO can expect to take. Davenport listed five progressive stages needed to achieve ‘big data’ competence in today’s business environment. First, there are the ‘Analytically Impaired Companies‘. These are companies that have some customer data but lack a centralized strategy to leverage its use.  Next up are the ‘Localized Analytics’. These entities outsource their data needs to companies that follow traditional marketing practices. Then come the ‘Analytical Aspiration’ types who centralize their data sources, enjoy C-level support, and operate an in-house data analytics team. At this level companies are just beginning to grapple with their ‘big data’ analytics issues. A fourth phase has been designated to ‘Analytical Companies’ who are showing some success in using data to drive their business. Finally, and at the top of the heap are the ‘Analytical Competitors’. These companies have fully integrated proven algorithms that combine unstructured web data, with reintegrate field data to seamlessly predict a specific customers expected wants and desires based on their personal past history with the company and elsewhere including the same for their closest peer group.

Most daunting to any CEO is the notion that companies ranked at Davenport’s ‘Analytical Competitors’ level can rely almost entirely on their algorithms to run their business. The indisputable outcomes dictate their level of ad spend per quarter, allocation of ads across multi-platforms, inventory levels per SKU, quality of maintenance support, head count, and so much more. At some point one might even ask what the role of management should be for a company ranked ‘Analytical Competitor’ and the talent/expertise needed to be an effective CEO in this soon-to-be, new normal.

© 2013 Tom Kadala