The Digital Transformation Playbook: Rethink Your Business for the Digital Age

Fail Smart
Failure is inevitable. We can define failure as trying something that doesn’t 
work. Obviously, that is not the ultimate goal of innovation, but it is an inev-
itable part of the process of innovation. Intuit’s cofounder Scott Cook has 
said that in their entry to the Indian market they ran thirteen early experi-
ments; two of their ideas proved successful, one had to pivot (undergo a 
dramatic shift in the business model), and the other ten failed.
26
What if 
Intuit had been unwilling to tolerate failure in new innovations? If you try 
to avoid any failures, you will retreat into whatever seems most safe and 
never innovate.

---

142
I N N O V A T E B Y R A P I D E X P E R I M E N T A T I O N
The challenge of failure is to fail smart. We can think of a smart failure 
as one that passes these four tests:
r
 Did you learn from the failed test?
r
 Did you apply that learning to change your strategy?
r
 Did you fail as early and as cheaply as possible? (For example, you 
didn’t waste a lot of resources developing a very advanced prototype 
before you discovered that the customer doesn’t want the product.)
r
 Did you share your learning (so that others in your organization won’t 
make the same mistake)?
Defined this way, smart failure is actually an essential part of experi-
mentation. It is needed to eliminate bad options quickly and to build on 
the learning that testing generates (like Hyman and Fleiss’s early lesson that 
they would be shunned by fashion designers and needed to market directly 
to consumers). Smart failure is simply a series of cheap, effective tests that 
show you the gaps between where you are and where you need to get. As 
baseball legend Babe Ruth said, “Every strike brings me closer to the next 
home run.”
Stefan Thomke makes a distinction between what he calls a “failure” 
and a “mistake.” For him, a mistake involves not learning from a failed test, 
repeating the error, and spending more resources without generating new 
learning.
27
We could also call that failing dumb.
Now that we’ve seen the seven overarching principles of good experi-
mentation, let’s take a look at the process for each type of experiment. We 
will do this with two step-by-step planning tools: the Convergent Experi-
mental Method and the Divergent Experimental Method.
Tool: The Convergent Experimental Method
This experimental method is particularly useful for innovating on existing 
products, services, and processes; for optimizing and continually improv-
ing them; and for comparing versions in the later stages of an innovation 
process. Convergent experiments can sometimes be run very quickly—in 
a matter of hours or even minutes (e.g., testing e-mails or Web designs). 
Others (e.g., testing a retail concept) will take longer. You can see the 
seven-step Convergent Experimental Method in figure 5.2.

---

I N N O V A T E B Y R A P I D E X P E R I M E N T A T I O N

143
Step 1: Define the Question and Its Variables
The first step of any convergent experiment is to define the question you are 
seeking to answer. This could be “How will our new service offering affect 
customer retention?” or “Which of these two pricing tiers will yield the 
highest total revenue for our new product line?” or “How will the planned 
redesign of our customer service portal affect customer satisfaction?”
In a convergent experiment, the question needs to be as specific as pos-
sible. It should also be framed, if possible, as a causal question: “If we do X, 
then what will happen to Y?”
Once you have stated the question, you need to translate it into two 
kinds of variables:
r
 Independent variable (or cause): This is the factor that you will be test-
ing in your experiment. Typically, it is a variation on current business 
practice. The aim of the experiment is to understand the effect of intro-
ducing this innovation.