The Physics of Wall Street: a brief History of Predicting the Unpredictable

From Coastlines to Cotton Prices 
•
67
remarkably, the same pattern emerges. If you look at just the wealthi-
est people in a country, 80% of their wealth is controlled by just 20% of 
them. the superrich tend to have the same disproportionate amount 
of wealth as the plain old rich. And indeed, the pattern continues. 
eighty percent of the resources controlled by the superrich are con-
solidated in the hands of the ultra-superrich. And so on.
this kind of pattern should be familiar by now. Wealth distribution 
across a population displays a kind of self-similarity, or a fractal pat-
tern. In fact, the distributions that Pareto discovered, called Paretian 
distributions, are a type of fat-tailed distribution — revealing a kind 
of wild randomness in income distribution, though not quite as wild 
as the drunken firing squad’s shots. When Mandelbrot was looking at 
the data for IBM, he had not yet invented fractals. His seminal work 
on the coastline paradox was almost a decade away. But similarly to 
Pareto half a century before him, something about the pattern struck 
Mandelbrot. It reminded him of his doctoral work on Zipf, who also 
had discovered an odd self-similarity in how word frequencies were 
distributed.
Although Mandelbrot had largely left academia, his work for IBM 
on wealth distribution was of some interest to mainstream econo-
mists, and so he was occasionally invited to give scholarly talks. It was 
in 1961, immediately before one of these lectures, that he made his 
second serendipitous discovery.
the talk was to be delivered to Harvard’s economics department. 
Shortly before it was scheduled to begin, Mandelbrot met with one 
of the faculty members, an economist named Hendrik Houthakker. 
As soon as he walked into Houthakker’s office, Mandelbrot noticed 
a drawing on Houthakker’s chalkboard. It was nearly identical to the 
graph that Mandelbrot was planning to use in his talk, as part of his 
discussion of income distribution and Pareto’s principle. Mandelbrot 
guessed that Houthakker must have been working on a similar prob-
lem and made some comment about their shared interests. Houthak-
ker responded with a blank stare.
After another awkward attempt or two, Mandelbrot realized that 
something was wrong. He backed up and pointed to the graph on the 

---

board. “Isn’t that a wealth distribution plot?” Puzzled, Houthakker ex-
plained that the drawing on his board had been from a meeting with a 
graduate student earlier in the day, during which Houthakker and the 
student were discussing historical data on cotton prices. the picture 
was a graph of daily returns from cotton markets.
Houthakker went on to explain that he had been working on cotton 
markets for a while now, but the data weren’t cooperating with theory. 
By this time, Bachelier’s work had been rediscovered and economists 
had begun to accept that markets undergo a random walk, as Bachelier 
and osborne had argued. Houthakker was interested in verifying this 
hypothesis by looking at historical data. If the random walk thesis was 
correct, you should see many small price changes over the course of 
a day or a week or a month, but very few large ones. What Houthak-
ker’s data showed, however, was not what the theory predicted: he was 
seeing too many very small changes, but also far too many very large 
ones. Worse, he was struggling to come up with a value for the average 
price change, as Bachelier’s theory predicted must exist. every time 
Houthakker looked at a new set of data, the average would change, 
often dramatically. In other words, cotton prices seemed to behave 
more like a drunken firing squad than a drunken vacationer.
Mandelbrot was intrigued. He asked Houthakker if he could look 
more closely at the data, and Houthakker agreed; in fact, Houthakker 
told Mandelbrot that he could have it all, since he was ready to aban-
don the project.
Back at IBM, Mandelbrot had a small team of programmers tear 
through boxes of Houthakker’s cotton data, analyzing everything in 
detail. He quickly confirmed Houthakker’s most troubling findings: it 
appeared that there was no “average” rate of return. the prices looked 
random, but they weren’t explained by the standard statistical tools or 
Bachelier’s and osborne’s theories. Something weird was going on.
Mandelbrot had seen unusual distributions before. In addition to 
studying Zipf’s and Pareto’s work, he was familiar with a third kind of 
distribution, discovered by one of his professors in Paris, Paul Lévy. It 
was Lévy who, upon reading a small section of one of Bachelier’s pa-
pers, concluded that Bachelier’s work was plagued with errors. Much 
68 
•
t h e p h y s i c s o f wa l l s t r e e t

---