Way of the turtle

174
•
Way of the Turtle

---

MAR Ratio as Drawdown Threshold Varies
0.0
50%
Drawdown Threshold
49%
48%
47%
46%
45%
44%
43%
42%
41%
40%
39%
38%
37%
36%
35%
34%
33%
32%
31%
30%
29%
28%
27%
26%
25%
24%
23%
22%
21%
20%
–0.2
–0.4
0.2
0.4
0.6
0.8
1.0
1.2
1.4
Figure 11-6
Change in MAR Ratio as the Number of Days in the Moving Average Varies
Copyright 2006 Trading Blox, LLC. All rights reserved worldwide.
•
175
•

---

which you achieved in testing. It is also another reason why opti-
mizing parameters is good: You can see cliffs and fix the source of
the problem before you start trading.
The Importance of Sample Size
As was noted briefly in Chapter 2, people tend to place too much
importance on a small number of instances of a particular phe-
nomenon despite the fact that from a statistical perspective very lit-
tle information can be drawn from a few instances of any event.
This is the primary cause of overfitting. Rules that do not come into
play very often can cause inadvertent overfitting, which leads to a
divergence in performance between backtests and real trading.
This can happen inadvertently over the course of many instances
because most people are not used to thinking in these terms. A
good example is seasonality. If one tests for seasonal changes in 10
years of data, there will be at most 10 instances of a particular sea-
sonal phenomenon since there are only 10 years of data. There is
very little statistical value in a sample size of 10, and so any tests
using those data will not be good predictors of future performance.
Let’s consider a rule that ignores this concept and uses the com-
puter to help us find the perfect way to overfit to our data. You
might notice that September was a bad month for a few years; then
you might test a rule that lightens positions in September by a cer-
tain percentage. Since you have a computer, you might decide to
have it search for any sort of seasonally bad periods in which you
should lighten up.
I did this for the system in this chapter. Then I ran 4,000 tests
that tested reducing positions at the beginning of each month and
then lightening up by a certain percentage for a certain number of