The Digital Transformation Playbook: Rethink Your Business for the Digital Age

Measure What Matters Now
It is important to take measurements in any experiment. But what do you 
measure? As interactions become more digitized, the number of things that 
can be measured is growing, and it is easy to get distracted by all the num-
bers you could be tracking—particularly in a real-world experiment with a 
large customer sample.
One solution is to try to identify the most important single metric for 
the success of your innovation. Alistair Croll and Ben Yoskovitz call this the 
“One Metric That Matters.”
20
They stress how that one metric that matters 
most will change over time as a start-up moves from the early stages of cus-
tomer definition to solution testing and eventually to revenue and scaling 
a business. The same is true when innovating within an existing enterprise: 
the one metric that matters most will change over time.
In the case of Intuit’s Fasal, the ultimate goal was 10 percent more rev-
enue for farmers. And, eventually, that would become a key metric to mea-
sure (as well as metrics like advertising revenue, once that became part of 
the business model). But at earlier stages of the product design, the com-
pany may have wanted to focus on different measures, such as “How many 
of our initial test farmers are able to receive and utilize the pricing informa-
tion?” and later “How many new subscribers are we getting each week as we 
begin to roll out the public product?”
Although it is important to know the most critical metric for the cur-
rent stage of your innovation, you should gather data on other metrics as 
well. This data may help explain the changes you see in your key metric. 
When Wawa introduced a flatbread sandwich to its menu in a number of 
test stores, the chain measured customer adoption and found the product 
was a big popular success. But it also measured the change in overall prof-
itability at the stores. It turned out that customers were spending less on 
other, higher-margin items, so Wawa was actually losing money thanks to 
the popular new sandwich. Rather than rolling it out to more stores, the 
chain pulled it from the menu entirely.
21

---

140
I N N O V A T E B Y R A P I D E X P E R I M E N T A T I O N
Test Your Assumptions
Another key principle of experimentation is to test your assumptions. 
Although this is essential to eliminating risk in any new venture, it is espe-
cially important for innovations that take your business into unknown 
territory.
When Jenn Hyman was still an MBA student, she developed an idea for 
a new company. Seeing her sister agonizing over whether to spend $1,500 
on a Marchesa dress to wear to a friend’s wedding, she saw a great business 
opportunity: Why not offer to rent designer dresses for special occasions? 
Joined by classmate Jenny Fleiss, Hyman decided to try to launch a new 
business: Rent The Runway. But rather than spending time writing up a 
business plan with detailed projections on pricing, costs, market size, and 
revenue, the two decided to start running experiments to see if their basic 
idea would even work.
22
The business seemed promising to Hyman and Fleiss, but they realized 
that their idea was based on assumptions about customers, their interests, 
and their willingness to pay for such a service, not to mention product 
selection, the durability of dresses during repeated rentals, and the right 
channel to market their service. So they made a plan and methodically 
tested their assumptions in a series of experiments. Their first two market 
tests were run on college campuses (Harvard and Yale); at each, they sent 
out invitations to students, rented a room, and brought a large selection 
of designer dresses for rent. They quickly validated the assumption that 
middle- and upper-income women would pay one-tenth the price of a 
designer dress to be able to rent it for one occasion. They also tested what 
the impact of selection size was (increasing the number of styles raised 
the rate of rentals) and whether the dresses would be returned in good 
shape (only 4 percent came back with stains, which were easily removed). 
In their third experiment, they tested whether customers would still rent 
the dresses if they could not try them on in person (their plan for the 
business was to offer rentals online). Rather than hiring a Web designer 
to build a website, they sent e-mails with photos of rental dresses to 1,000 
women in New York City. Although the rental rate dropped from 35 to 
5 percent of those invited, it was still high enough to proceed with a plan 
for an online business. In their fourth experiment, they reached out to 
the fashion designers themselves. Their hope was to convince designers to 
promote the rental service on their own websites, so that visitors looking 

---

I N N O V A T E B Y R A P I D E X P E R I M E N T A T I O N

141
at a dress on Diane von Furstenberg’s website would see that they could 
rent it rather than having to go to Neiman Marcus to buy it. They met with 
twenty designers, and the response from most was quite negative. Fearing 
cannibalization of dress sales, most replied that they would help the new 
business “over my dead body.” Hyman and Fleiss knew they had to revise 
their marketing plan. Rather than focusing on order fulfillment and letting 
the designers lead the marketing of their service, they would purchase an 
ample inventory of dresses, build an e-commerce website, and drive traffic 
there themselves.
23
When Hyman and Fleiss went looking for investors, Bain Capital was 
impressed with the speed with which they had tested the parameters of 
their new business model and signed on board with the first round of 
financing. Rent The Runway launched less than a year after Hyman’s first 
flash of insight while watching her sister’s dress dilemma. Two years later, 
Rent The Runway provided dresses for 85 percent of the women attending 
the 2013 U.S. presidential inauguration.
24
Rent The Runway was a new start-up, and it is sometimes easier to rec-
ognize all the things you don’t know about your business when you are just 
starting. For an established company, used to operating in its known terri-
tory, it is easy to overlook the step of testing your assumptions when you 
are planning an innovation. In their book Discovery-Driven Growth, Rita 
McGrath and Ian MacMillan explain how successful firms take on undue 
risk by not identifying the underlying assumptions of their new ventures. 
The authors suggest methods to identify such assumptions and test them, 
and they tie this process to development milestones on any new project.
25
This mindset is essential to good experiment-driven innovation.