Pigeonhole Principle and the Trap of Higher Win Rate

Most of the time, we see traders focus heavily on higher winning rate and at all cost keeping it that way by various measures. System traders and rookie system creators are also obsessed with higher win rate. Is higher win rate really that important?

Pigeonhole Principle

In mathematics, pigeonhole principle tells us that given a small number of classifications available, with a big collection of data, it is bounded to produce some unexpected results at times.

Normally, it is not a problem. For statisticians and mathematicians who have dealt with all kinds of raw data, they know very well what pigeonhole principle is and will do the necessary steps to further analyze their results to remove the "noise" from their statistics. (Of course there are researchers paid to paint the results they’ve obtained but that is something outside of the scope of this article.)

However, when the study is conducted by casual researchers, like traders / programmers trying to find statistical bias from the price data, many would fall straight into one of the two traps. Believing in results that are too good to be true is one of them. Curve-fitting their models to produce higher win rate at the cost of stability is the other.

The Problem Lies In The Borderline Cases

A nudge of a trading model’s parameters giving it a boost of 10% more winning trades is not necessarily a good thing if the winning amount of most of the winning trades has been reduced. In another words, a shift of the median winning amount closer to zero. It is the first sign that the change is not helping the model.

On the other hand, changing the parameters to get a drop of of winning trades to near 50% winning rate is not necessary a bad thing. It is especially true if the median winning amount has moved higher and improvement in the overall performance is observed.

As you can see from the 2 scenarios I mentioned above, the problem is that nudging the parameters of a trading model can easily move borderline trades switching from winners to losers, and vice versa. Depending on the sensitivity of your model to these changes in the parameters, it is bounded to show sudden boost in win rate at times. In many of these situations, the change means nothing to the core driver of the model.

There Are Winners, Losers And Noises

To solve this problem, you need to classify the trades in a trading model into three categories instead of just two. There are the winners and losers that are not affected by mild nudging of the parameters. There are also the noise that are sensitive to the parameter nudging.

Optimization and parameter nudging should focus on the improvement of the real winners and losers, not the noise. The trades in the noise category, at times, may tilt towards mild winning thus boosting overall historical performance. But due to the fact that there are many more winning trades with low winning amount, the median and average winner will be reduced. Now you know a way to identify the good optimization results from the bad ones.

Do Not Try To Remove Noises By Curve-Fit

There is absolutely no point trying to remove the noise trades from a system by just parameter nudging or optimization. If your model does not have independent filters for the purpose of filtering noise trades, curve-fitting your core driver to avoid noise trades usually damages the performance of the real winners. So don’t even try.

The good winners and the noise trades are likely results from two distinctly different pre-conditions. By changing your original parameters to nudge the noises into winners, you are also changing the conditions for which you get those good winners in the first place. Hence, the model is no longer the one you have in the first place after you modified the parameters to squeeze out more winners.

For example, many people would nudge their breakout models to delay their breakout entries with stop-in price further away from the reference price. That, in turn, cut deep into the real winners for the purpose of getting more lame winners. Such changes is not a solution to obtain better trading models.

Reduce Impact of Noises By Filters Independent of Original Drivers

This is the part that most rookie system designers hate the most. It is also what separate great traders from good ones. By paying attention to the minute details of the pre-conditions and the initial development phase of the trades, there can be subtle signs that a trade may not workout. Great traders are good at what they do because they observe and dissect the information dispassionately while average traders having their emotional roller coaster ride.

The discovery of pre-conditions that are independent from the original drivers will give you filters to remove, reduce or early terminate the noise trades.

Filters can come in all forms. Some can be time based. Some can be chart patterns (from higher timeframe). It is important to think in terms of what the oppose side would do when looking for signs to jump in. These signs are usually the best warning signal that a trade has gone bad.

Share