The Digital Transformation Playbook: Rethink Your Business for the Digital Age

New Tools to Wrestle Unstructured Data
The second trend shaping big data is the rise of new technological capabili-
ties for handling and making sense of all this unstructured data. If not for 
this, big data would be simply a giant haystack in which the needle of busi-
ness insight might well be invisible. Fortunately, a range of technological 
developments is expanding our abilities to use the unstructured data that 
technology is producing.
The continuing exponential growth of computer processing power is a 
big factor in our improved ability to use data. Moore’s law, coined by Intel 
cofounder Gordon Moore in 1965, predicts a doubling in the performance 
of computer chips roughly every eighteen months as transistors become 
faster and smaller. For fifty years, the prediction has held, and the results 
have transformed the world. ENIAC, the first modern computer, was built 

---

100
T U R N D A T A I N T O A S S E T S
in 1946 and filled a room the size of a small gymnasium. But by 1983, when 
I first studied computing, my student-grade Texas Instruments pocket cal-
culator had more processing power than ENIAC. Moore’s law tells us that 
this decade’s supercomputer is the next decade’s pocket device.
Recent technologies have further enabled data processing on a large 
scale with acceptable costs. In-memory computing can accelerate analytics 
to the kind of real-time computing that allows digital advertising to select 
the ad seen by each visitor to a webpage, based on the weather where they 
are, the sites they have visited recently, or any other critical determinants 
that can be mined through data. Hadoop is an open-source software frame-
work that enables distributed parallel processing of huge amounts of data 
across multiple servers in different locations. With Hadoop, even the big-
gest data sets can be managed affordably.
Other tools focus less on increasing power and more on making sense 
out of the chaos of unstructured data. New data-mining tools allow pro-
grams to sift through the raw stuff of social media and pick out patterns 
that human managers then can examine to recognize trends and key words.
Perhaps the biggest advances in managing unstructured data have 
come from new developments in “cognitive” computing. Natural language 
processing, for example, can interpret normal human language, whether 
from spoken commands, social media conversations, or books or articles, 
without adaptation. It is critical to the development of systems that can 
identify patterns in big-data sets of human language, such as recordings of 
customer phone calls to call centers. Another key development is machine 
learning—resulting in computing systems that can recognize patterns and 
improve their own capability over time, based on experience and feedback. 
As computers are modeled around neural networks, they go beyond just 
spotting patterns in unstructured data: they receive feedback from their 
environment or human trainers (indicating which conclusions were wrong 
and which were correct) and reprogram themselves over time.
Natural language processing and machine learning are combined in a 
system like IBM’s Watson, which can read vast amounts of written language 
and develop ever more accurate inferences by using feedback and coach-
ing from human experts. Watson famously debuted on the world stage by 
playing the quiz show Jeopardy!—where it bested the top human champi-
ons by combining encyclopedic recall with a human-like ability to have 
educated “hunches” (e.g., estimating that its best guess to a question had 
a 42 percent likelihood of being correct). Since then, Watson has moved 
to the real world. Physicians have trained Watson, using a library of mil-
lions of patient case histories, to the point where Watson is more accurate 

---

T U R N D A T A I N T O A S S E T S

101
than many doctors in making an initial diagnosis of a new cancer patient. 
Watson and similar technologies will be at the forefront of the next wave of 
big-data analytics—informing everything from customer service, to fraud 
detection, to advertising media planning.