What Is the Impact of Big Data?

Big data has had a long buildup. The term first popped up in an Institute of Electrical and Electronics Engineers publication in 1997 discussing the challenge of working with blocks of information as large as 100 gigabytes. By the late 2000s, breakthroughs in storage capacity and computing power were leading to euphoric proclamations about the transformational potential of data and analytics in almost every aspect of business and society.

Yet, in 2016, the Economist Intelligence Unit is still pointing to the big data revolution as imminent: “Companies in all industries will strive to collect, interpret, and capitalize on vast amounts of new data. ‘Big data’ will be harnessed to understand consumers, improve healthcare, and cut firms’ energy bills.”

Perhaps, the reason for the sluggishness with which companies are taking full advantage of big data is that doing so requires much more than surmounting massive technical challenges. A report by Accenture noted that the task goes beyond integrating internal and external data into usable analytics; firms also have to restructure internally, even reshaping corporate culture, to use those insights effectively. Teams must have both data science and creative capacity, and fit into an organizational structure that can respond to analytics nimbly.

In a conversation with Yale Insights, Yale SOM’s K. Sudhir said that big data’s time has come, especially in marketing. “Almost every aspect of our daily lives gets recorded and leaves some kind of a digital trace,” he pointed out. Nevertheless, there is still “a lot of gut-based decision making,” said Sudhir, the James L. Frank Professor of Marketing, Private Enterprise and Management and director of the Yale China India Insights Program. “We now have the opportunity to do evidence-based management.”

Smartphones give marketers an unprecedented window into consumer behavior with the ability to track search, browsing, and purchasing. Sudhir sees the offline and online distinction blurring, if not disappearing. “Marketers are able to deliver their message, the right kind of targeting, the right kind of offer almost any time,” he said. “It makes marketing much more efficient.”