The Digital Transformation Playbook: Rethink Your Business for the Digital Age

Traditional innovation cycle
Innovation by rapid experimentation
Observe
Observe
$
$
$
$
$
$
$
$
$
$
$
$
$
$
$
$
$
Generate
ideas
Generate
ideas
Analyze
and discuss
Design
prototype
and test
Customer
response
Team
decision
Cost of learning
Cost of learning
Design
product
Build
product
Launch
and market
Customer
response
Figure 5.1
Financial Impact of Rapid Experimentation.

---

136
I N N O V A T E B Y R A P I D E X P E R I M E N T A T I O N
This distinction is important. In any innovation effort, you are dealing 
with uncertainty and inevitably will face a significant failure rate among 
your new ideas. (If you don’t, then the ideas you are testing are not genu-
inely new, and the potential gains will be limited.) With experimentation, 
the ideas that don’t work should fail early in the development process, 
long before your product gets to the public and while the cost of chang-
ing course is much lower. Waiting too late in your innovation process to 
show your idea to customers has the inverse effect: it increases the costs of 
error, reduces the likelihood that you can muster the organizational will to 
change course, and discourages you from testing other options.
Many firms measure the costs of running experiments (which in some 
industries can still be expensive), but very few attempt to measure their cost 
savings when learning from experiments—whether from early cancellation 
of what would have been an expensive flop or from course correction that 
turns a struggling project into a successful one.
Be Fast and Iterate
The second key principle of experimentation is speed. John Hayes, Ameri-
can Express’s global chief marketing officer, spoke to me about his compa-
ny’s focus on learning through experimentation. He explained that one of 
his primary goals as a leader is to get his teams to learn faster—in iterative 
cycles of days rather than weeks or months.
12
For a nimble organization like 
American Express, institutionalizing that kind of faster learning can be a 
real source of competitive advantage.
Hayes’s insight echoes that of an earlier famous experimenter, Thomas 
Edison, who proclaimed that “the real measure of success is the number of 
experiments that can be crowded into 24 hours.”
13
When John Mayo-Smith was chief technology officer for R/GA, he 
worked on numerous innovation projects with brands like Nike, includ-
ing Nike+, FuelBand, and other early wearable technology successes. “Our 
goal at R/GA was always about building something quick. If you were 
our client, we didn’t spend four months scoping out a project. We aimed 
to have something built in two weeks, to start showing to real athletes, and 
getting their feedback.”
14
Mayo-Smith’s approach to building technology as 
successive stages of workable iterations has been adopted by teams from 
Caltech to NASA.
15

---

I N N O V A T E B Y R A P I D E X P E R I M E N T A T I O N

137
Increasing the speed of experimentation may require infrastructure, 
too. When Edison built his lab in West Orange, New Jersey, the physi-
cal layout was designed to facilitate speed in moving from any insight or 
hypothesis to a quick working test of it. Supplies of all kinds—tools, chemi-
cals, ores, minerals, filaments—were stored in stockpiles in close proximity 
to every experimental lab so that delays in procuring equipment would not 
slow down the exploration of any new idea.
16
To speed up its own innovation experiments, global snack maker Mon-
delez (formerly Kraft) uses a “garage” that is designed to get any new idea 
from concept to prototype and into the hands of visiting customers within 
two days’ time.
17
Design firm IDEO places its prototyping shops in close 
proximity to its development teams so that physical product ideas can be 
fabricated in days or even hours.