Way of the turtle

170
•
Way of the Turtle
Example Parameters A and B
B
A
Figure 11-4
Example Parameters A and B

---

future actual results are just as likely to be better than or worse
than those indicated by the historical tests that use value A for the
parameter. 
Why is this? To make it clearer, let’s assume that the future will
vary in such a way that it is likely to alter the graph slightly to the
left or the right, but we do not know which. The graph in Figure
11-5 shows A and B with a band of values to the left and right that
represent the possible shifts that result from the future being dif-
ferent from the past that we’ll call margins of error.
In the case of value A, any shifts of the optimal parameter value
to the left of A on the graph will result in worse performance than
point A, and any shifts of the optimal parameter value right of A
will result in better performance. Thus, the test result with a param-
eter value of A represents a decent predictor regardless of how the
future changes since it is just as likely to be underpredicting as over-
predicting the future.
This is not the case with value B. In all cases, any shift at all,
either to the left or to the right, will result in worse performance.
Lies, Damn Lies, and Backtests
•
171
Parameters A and B with Margins of Error
B
A
Figure 11-5
Parameters A and B with Margins of Error

---

This means that a test run with a value of B is very likely to over-
predict the future results. When this effect is compounded across
many different parameters, the effect of a drift in the future also
will be compounded. This means that with many optimized param-
eters it becomes more and more unlikely that the future will be as
bright as the predictions of the testing done with those optimized
parameters.
This does not mean that we should use value A in our trading.
Even in the event of a sizable shift, the values around the B point
are still higher than those around the A point. Thus, even though
optimization reduces the predictive value, you still want to trade
using values that are likely to give good results in the event of
drift.
The optimization paradox has been the source of much decep-
tion and scamming. Many unscrupulous system vendors have used
the very high returns and incredible results made possible through
optimization, especially over shorter periods, by using market-spe-
cific optimization to show historical results that they know cannot
be achieved in actual trading. However, the fact that optimization
can result in tests that overstate likely future results does not mean
that optimization should not be performed. Indeed, optimization
is critical to the building of robust trading systems.