The Physics of Wall Street: a brief History of Predicting the Unpredictable

The Prediction Company 
•
143
technology sent signals via a vibrating magnet attached to the bettor’s 
torso, hidden by clothes. one night, the wires on farmer’s magnet kept 
coming undone, burning his skin whenever the signal arrived. every 
ten minutes he had to jump up from the table and announce some-
thing like “Boy, have I got the runs today!” on his way to the men’s 
room to fix the equipment (this continued until the pit boss followed 
him in and sat in the next stall until farmer decided to call it quits for 
the night). But by the summer of 1978 the computers were running 
well enough that the team took them to vegas — and started to profit.
Meanwhile, as the team at eudaemonic enterprises continued work 
on building a better bettor, farmer, Packard, and some of the others 
in the group began thinking more about the physics at the heart of the 
project. they had derived the equations they needed to predict rou-
lette. But thinking about roulette had piqued their interest in a more 
general problem. roulette is an example of a dynamical system that 
exhibits some pretty funky behavior. Most importantly, where the ball 
lands is sensitive to the initial conditions — much like the weather sys-
tem Lorenz discovered. Working out how to use computers to solve 
the differential equations necessary to predict roulette had unwittingly 
put farmer and Packard at the cutting edge of the newest research in 
chaos theory. farmer’s advisor was right that there was a dissertation 
in the roulette calculations. What he didn’t know was that the disserta-
tion would be part of a rising tide of ideas that would usher in a new 
age of physics.
In 1977, some of the physicists working on eudaemonic enter-
prises (farmer and Packard, along with an undergraduate named 
James crutchfield and an older graduate student named robert Shaw) 
started an informal research group called by turns the dynamical Sys-
tems collective and the chaos cabal. Shaw threw out a nearly finished 
dissertation to start working on chaos theory full-time; farmer offi-
cially switched away from astrophysics. By the late 1970s, a great deal 
had been done on chaos theory. Lorenz had discovered many of the 
basic principles and had then come up with simple examples of cha-
otic systems and described how they behaved. He was the first person 
to recognize that there is a kind of order in chaotic systems: if you 
draw pictures of the paths traced by objects obeying differential equa-

---

tions, they tend to settle down into regular patterns. these patterns are 
called attractors, because they tend to attract the paths of the objects. 
In roulette, for instance, the attractors correspond to the pockets of 
the wheel: whatever trajectory the ball takes, in the long run it will 
settle down into one of these states. But for other systems, the attrac-
tors can be much more complicated. A major contribution to the study 
of chaos theory was the realization that if a system is chaotic, these 
attractors have a highly intricate fractal structure.
But despite these foundations, the subject was still young. Work had 
been done in fits and starts, without any real research center. nor-
mally, graduate work in physics is a collaboration among graduate 
students, young postdoctoral researchers, and a professor. But chaos 
theory was still so new that these kinds of research groups didn’t yet 
exist. You couldn’t go to graduate school to study chaos theory. the 
dynamical Systems collective was an attempt to fix this, by pulling 
its members through graduate school by their bootstraps. Some of the 
faculty at Santa cruz were skeptical about this divergence from the 
traditional academic curriculum. But the department was new and 
open to novel ideas, and enough professors were supportive that the 
four initial members were permitted to guide themselves, collectively, 
to Phds in chaos theory.
from the very start, prompted perhaps by the roulette experience, 
the dynamical Systems collective was interested in prediction. It was 
a novel way of thinking about chaotic systems, which most people 
were interested in precisely because they seemed so unpredictable. the 
collective’s most important paper, published in 1980, showed how you 
could use a stream of data from, say, a sensor placed in the middle of 
a pipe with water flowing through it to reconstruct what the attractor 
for the system would have to be. And once you had the attractor, an 
essential part of trying to understand how a chaotic system would be-
have over time, you could begin to make some predictions. Previously, 
attractors were understood as a theoretical tool, something you could 
get only by solving equations. Packard, farmer, Shaw, and crutchfield 
showed that, in fact, you could figure out this important feature em-
pirically, by looking at how the system actually behaved.
the dynamical Systems collective lasted for four years, during 
144 
•
t h e p h y s i c s o f wa l l s t r e e t

---

The Prediction Company 
•
145
which time it made seminal advances in chaos theory and managed to 
turn years of thinking about roulette into respectable science. But the 
eudaemons couldn’t stay in graduate school forever. farmer gradu-
ated in 1981 and immediately went to Los Alamos. Packard left the 
following year, to take a postdoctoral position in france. Both men 
were on the verge of turning thirty when they left school. eudaemonic 
enterprises was making money from roulette, but it was ultimately a 
state of mind, not a way to earn a living.
It was a miracle that either farmer or Packard got academic jobs, 
with degrees in chaos in the early 1980s, when few physicists knew 
what the new theory of dynamical systems was all about, and even 
fewer recognized it as something worth pursuing. Los Alamos, like 
Santa cruz, was far ahead of its time, and farmer was fortunate to find 
himself at the center of research in the new field. (Packard had similar 
luck. After his postdoctoral year in france, he landed positions at the 
Institute for Advanced Study, in Princeton, new Jersey, and the center 
for complex Systems research, at the University of Illinois, the other 
two hotbeds of complex systems research.) things got even better for 
farmer in 1984, when a group of senior scientists at the lab launched a 
new research center devoted to the study of complex systems, includ-
ing chaos. the center was called the Santa fe Institute. Physics would 
play a central part in the Santa fe Institute’s research, but the center 
was designed to be essentially interdisciplinary. complex systems and 
chaos arose in physics, in meteorology, in biology, in computer science
— and also, the Santa fe researchers soon realized, in economics.
one theme that characterized much of the research in complexity and 
chaos during the early 1980s was the idea that simple large-scale struc-
tures can emerge from underlying processes that don’t seem to have 
that structure. to take an example from atmospheric physics, consider 
that the atmosphere, at the smallest scale, consists of a bunch of gas