Beware of the Threshold Distortion

There is one trap that I have seen much too often. It is using thresholds in the presence of randomness (noise). It is used in arguing against the existence of investor skill. I say this very bluntly: this method of presentation propagates a lie.

Here is the method:

A person observes the initial price of a stock or mutual fund to see how well it does over a fixed number of years. This can be as short as one year. It is seldom longer than five years. He looks at prices (or total return) once more after a second interval. Then he compares what has happened during the first interval to what happens during the second.

There are three relevant times: the start, the middle time and the end.

For purposes of discussion, let us examine what happens when prices are completely random (after subtracting out the average). Prices at each of the three times can be positive or negative (relative to the average). The typical price is zero (relative to the average).

Let us now introduce a threshold at the middle time and use it for selecting stocks (or mutual funds). We will set the threshold high. Only a few stocks (or mutual funds) will exceed the threshold.

Consider how these stocks (or mutual funds) behave. Typically, they start close to zero, rise high enough to pass the threshold. Typically, they should fall back to a value close to zero. In a few instances, they would have started or ended up significantly higher or lower than zero. But typically, they would have been close to zero. We should see a sharp price increase in the first interval followed by a comparable price decrease in the second.

When stock (or mutual fund) prices are truly random, selecting the best performers during the first interval is the same as selecting which stocks (or mutual funds) are most likely to do the worst in the second interval.

If there were no randomness whatsoever, selecting the best performers during the first interval should be the same as selecting the best performers during the second interval.

What we see is critically dependent upon the degree of randomness when compared to the level of skill (i.e., the noise level as compared to a signal).

What Does This Mean?

In the stock market, the single-year randomness of the S&P500 index is plus and minus 10% (i.e., plus and minus one standard deviation). A very high level of skill corresponds to a 2% improvement in the long-term annualized total return. Even a 1% improvement corresponds to a high level of skill.

The randomness in the market is much more than the level of skill that we should expect. You tell a lie if you use thresholds as if there were no randomness. When you take randomness into account, your baseline for comparison should be for the selected stocks or mutual funds to underperform during the second interval by as much as they outperformed during the first. Unless they give back everything, skill exists.

[Actual comparisons need to include confidence intervals and to report the power of any statistical test.]

What Should You Do?

Mark Hulbert, who reports newsletter performance in The Hulbert Financial Digest, identifies the sweet spot for selecting superior newsletter writers as using (close to) nine years of past performance. It can take another nine years before the skills of superior newsletter writers return to dominance.

Hulbert’s time periods are consistent with what is needed to show skill. There needs to be enough time to reduce the effects of the 10% year-to-year randomness (standard deviation) of the S&P500 index to levels comparable to those of exceptional skill (exceeding 2%).

If you are inclined to invest in mutual funds or to follow a newsletter, focus on the latest nine years. Use caution. [Some of that information can be misleading.]

Failure Mechanisms and Success Mechanisms

I have already identified the strongest failure mechanism in Bull Street Bull. There were very few mutual funds before 1980. For all intents and purposes, index funds have only been around since 1976 (starting with the Vanguard 500 index fund). The Hulbert Financial Digest began in 1980. The bulk of what we know based on what happens during a powerful, secular (long lasting) bull market.

We have some early indications of what to expect during today’s secular bear market. The mantra for investing in all-stocks, all of the time, will no longer hold. Low cost actively managed mutual funds are likely to perform better than passively managed index funds. I expect some of our better newsletter writers to lose their advantage in this new environment. I expect the performance of some other newsletter writers to shine.

I expect the most important success mechanism to be varying stock allocations according to overall stock market valuations.

We should be prepared just in case a new bubble emerges. I do not expect it to happen.

Additional Remarks

Value investors will recognize the threshold effect. Value investors buy when prices are below normal. They set negative thresholds (compared to average prices).

Value investors recognize that stocks are (almost) always mispriced. The large single-year standard deviation of stocks makes this obvious. Benjamin Graham referred to the strange, moody behavior of Mr. Market. Professor Robert Shiller wrote about the causes of Market Volatility.

Some readers may be aware of Robert Arnott’s research into the use of capitalization weighting in index funds. [The S&P500 index is an example.] He has shown that there is a distortion caused by the weighting that has reduced index fund performance by 2% per year. The underlying mechanism is very similar to that of the threshold distortion. Capitalization weighting puts extra money on those stocks that happen to be at their highs simply because of randomness.

Readers may be surprised at my enthusiasm for an advantage of 1% or 2% (annualized, long-term). Compounded over a long period of time, a 1% or 2% advantage builds up to a large amount. A 10% return compounded over 30 years multiplies an initial investment by 17.4 times. An 11% return compounds to 22.3 times the initial investment. A 12% return compounds to 30.0 times the initial investment. Over 40 years, the multiples are 45.3, 65.0 and 93.1, respectively.

These comparisons extend to mutual funds as well. Mutual fund investors need to be careful about costs. Many actively managed mutual funds charge 1% to 2% more than passively managed index funds. What is worse, many fund managers engage in excessive trading and other behaviors that drag down performance. The effect of a 1% to 2% loss through fund costs is just as important as a gain of 1% or 2% through skill. [John Bogle refers to his Cost Matters Hypothesis. Fees and other costs compound to make a tremendous difference. Yet, they are one of the few things that an investor can control. Costs are certain. Returns are random.]

Have fun.

John Walter Russell
June 29, 2005.