Way of the turtle

Monte Carlo RAR%
140%
RAR %
2000 Iterations
130%
120%
110%
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90% Confidence
100%
Figure 12-3
Monte Carlo Distribution of RAR%
Copyright 2006 Trading Blox, LLC. All rights reserved worldwide.
•
205
•

---

Some Like It Rough
As the exercises above have demonstrated, a backtest is at best a
rough approximation of what one may expect in the future. Robust
measures are better predictors of future performance than are their
more sensitive counterparts, but the process is still imprecise. Any-
one who tells you that you can expect to see a particular level of
performance is lying or does not know what he is talking about. If
he is trying to sell you something, I strongly suspect the former.
Chapter 13 covers some of the methods you can use to make
your trading more robust—that is, less likely to suffer from wild
swings in performance.
206
•
Way of the Turtle

---

•
207
•
thirteen
BULLETPROOF SYSTEMS
Trading is not a sprint; it’s boxing. The market will beat you up, 
screw with your head, and do anything it can to defeat you. 
But when the bell sounds at the end of the twelfth round, 
you must be standing in the ring in order to win. 
N
ew traders who build trading systems are looking for a single
supercharged trading system that demonstrates the best possi-
ble results in historical testing. They believe a system that shows supe-
rior performance via historical data will indicate similar performance
in future trading. They look at tests showing a system (call it Omega)
that has a 10 percent better CAGR% and a 0.2 better MAR than
another system (call it Alpha) and conclude that they would be fool-
ish to trade with Alpha when Omega seems so much better.
Later, with more experience, one realizes that there is no such
thing as a perfect system. The Omega system might perform better
in certain types of market conditions, and because of the preva-
lence of the most favorable conditions in the past the Omega sys-
tem might have outperformed the Alpha system significantly in
testing. Unfortunately, there is no guarantee that those conditions
will occur with the same frequency in the future as they did in the
Copyright © 2007 by Curtis M. Faith. Click here for terms of use. 

---

past. In other words, the distribution of the types of markets may
be different in the future from what it was in the past. So if the per-
formance differences that were shown in testing between Omega
and Alpha are a result of the particular distribution of types of mar-
kets, those differences could disappear if the distribution is differ-
ent in the future.
Consider this example. Suppose Omega works much better than
Alpha when markets are trending and quiet but Alpha works bet-
ter when markets are trending and volatile. Now suppose that in
the 20-year test that was done there were 13 years in which the
trends that occurred were predominantly quiet and 7 years in
which the trends were mostly volatile. If the same distribution
occurs in the future, Omega will have better performance.
But what if 5 of the 7 years of volatile trends occurred during the
last 10 years of testing? What if there were changes in market
behavior as a result of trader effects that would result in trends
being more volatile in the future? This might indicate that the
Alpha system would be more likely to have better performance in
the future since it performs better when there are volatile trends.
Conversely, what if the market seemed to indicate a likely cyclical
shift from quiet to volatile markets and back again? Would this not
make it more likely that the Omega system would perform better
in the future as the markets shifted back to more quiet trends from
the period of recent volatile trends?