Way of the turtle

Robust Performance Measures
Earlier chapters in this book used the MAR ratio, CAGR%, and
the Sharpe ratio as comparative performance measures. These
measures are not robust, since they are very sensitive to the start
and end dates for a test. This is especially true for tests of less
than 10 years. Consider what happens when we adjust the start
and end dates for a test by a few months. To illustrate this effect,
let’s run a test that starts on February 1, 1996, instead of January
1 and that ends on April 30 instead of June 30, 2006, removing
just one month from the beginning of the test and two months
from the end.
A test of the Triple Moving Average system with the original test
dates returns 43.2 percent with a MAR ratio of 1.39 and a Sharpe
ratio of 1.25. With the revised start and stop dates, the return jumps
to 46.2 percent, with the MAR ratio increasing to 1.61 and the
Sharpe ratio increasing to 1.37. A test of the ATR Channel Break-
out system with the original dates shows returns of 51.7 percent, a
MAR ratio of 1.31, and a Sharpe ratio of 1.39. With the revised
dates, the return climbs to 54.9 percent, the MAR ratio increases
to 1.49, and the Sharpe ratio increases to 1.47.
184
•
Way of the Turtle

---

The reason we see this sensitivity across all three measures is that
the MAR ratio and the Sharpe ratio have return as a component of
their numerators and return, whether expressed by CAGR% used
for MAR or monthly average return used for the Sharpe ratio, is
sensitive to start and stop dates. The maximum drawdown can also
be sensitive to start and stop dates when that drawdown occurs near
the beginning or end of a test. This has the effect of making the
MAR ratio especially sensitive since it is composed of two compo-
nents, both of which are sensitive to start and end dates; therefore,
the effect of a change gets multiplied during the computation of
this ratio.
The reason CAGR% is sensitive to changes in start and stop
dates is that it represents the slope of the smooth line that goes from
the start of the test to the end of the test on a logarithmic graph;
changing the start and stop dates can change the slope of that line
significantly. Figure 12-1 shows this effect.
Note how the slope of the line labeled “Revised Test Dates” is
higher than that of the line labeled “Original Test Dates.” In the
example above there was a drawdown at the beginning of the test
On Solid Ground
•