which you achieved in testing. It is also another reason why opti-
mizing parameters is good: You can see cliffs and fix the source of
the problem before you start trading.
The Importance of Sample Size
As was noted briefly in Chapter 2, people tend to place too much
importance on a small number of instances of a particular phe-
nomenon despite the fact that from a statistical
perspective very lit-
tle information can be drawn from a few instances of any event.
This is the primary cause of overfitting. Rules that do not come into
play very often can cause inadvertent overfitting, which leads to a
divergence in performance between backtests and real trading.
This can happen inadvertently over
the course of many instances
because most people are not used to thinking in these terms. A
good example is seasonality. If one tests for seasonal changes in 10
years of data, there will be at most 10 instances of a particular sea-
sonal phenomenon since there are only 10 years of data. There is
very little statistical value in a sample size of 10,
and so any tests
using those data will not be good predictors of future performance.
Let’s consider a rule that ignores this concept and uses the com-
puter to help us find the perfect way to overfit to our data. You
might notice that September was a bad month for a few years; then
you might test a rule that lightens positions
in September by a cer-
tain percentage. Since you have a computer, you might decide to
have it search for any sort of seasonally
bad periods in which you
should lighten up.
I did this for the system in this chapter. Then I ran 4,000 tests
that tested reducing positions at the beginning of each month and
then lightening up by a certain percentage for a certain number of
Do'stlaringiz bilan baham: 
