The Lawrence Chan Blog

Another update to the site is now in beta.

The new forum and file download area replaces the old discussion area with many new features:

– more robust forum engine

– members can upload files of various kinds

– private messaging among members

The look and feel of the forum will still undergo some fine-tuning.

If you discover problems using the new forum area please let me know.

No comment yet

Most of the time, we see traders focus heavily on higher winning rate and at all cost keeping it that way by various measures. System traders and rookie system creators are also obsessed with higher win rate. Is higher win rate really that important?

Pigeonhole Principle

In mathematics, pigeonhole principle tells us that given a small number of classifications available, with a big collection of data, it is bounded to produce some unexpected results at times.

Normally, it is not a problem. For statisticians and mathematicians who have dealt with all kinds of raw data, they know very well what pigeonhole principle is and will do the necessary steps to further analyze their results to remove the "noise" from their statistics. (Of course there are researchers paid to paint the results they’ve obtained but that is something outside of the scope of this article.)

However, when the study is conducted by casual researchers, like traders / programmers trying to find statistical bias from the price data, many would fall straight into one of the two traps. Believing in results that are too good to be true is one of them. Curve-fitting their models to produce higher win rate at the cost of stability is the other.

The Problem Lies In The Borderline Cases

A nudge of a trading model’s parameters giving it a boost of 10% more winning trades is not necessarily a good thing if the winning amount of most of the winning trades has been reduced. In another words, a shift of the median winning amount closer to zero. It is the first sign that the change is not helping the model.

On the other hand, changing the parameters to get a drop of of winning trades to near 50% winning rate is not necessary a bad thing. It is especially true if the median winning amount has moved higher and improvement in the overall performance is observed.

As you can see from the 2 scenarios I mentioned above, the problem is that nudging the parameters of a trading model can easily move borderline trades switching from winners to losers, and vice versa. Depending on the sensitivity of your model to these changes in the parameters, it is bounded to show sudden boost in win rate at times. In many of these situations, the change means nothing to the core driver of the model.

There Are Winners, Losers And Noises

To solve this problem, you need to classify the trades in a trading model into three categories instead of just two. There are the winners and losers that are not affected by mild nudging of the parameters. There are also the noise that are sensitive to the parameter nudging.

Optimization and parameter nudging should focus on the improvement of the real winners and losers, not the noise. The trades in the noise category, at times, may tilt towards mild winning thus boosting overall historical performance. But due to the fact that there are many more winning trades with low winning amount, the median and average winner will be reduced. Now you know a way to identify the good optimization results from the bad ones.

Do Not Try To Remove Noises By Curve-Fit

There is absolutely no point trying to remove the noise trades from a system by just parameter nudging or optimization. If your model does not have independent filters for the purpose of filtering noise trades, curve-fitting your core driver to avoid noise trades usually damages the performance of the real winners. So don’t even try.

The good winners and the noise trades are likely results from two distinctly different pre-conditions. By changing your original parameters to nudge the noises into winners, you are also changing the conditions for which you get those good winners in the first place. Hence, the model is no longer the one you have in the first place after you modified the parameters to squeeze out more winners.

For example, many people would nudge their breakout models to delay their breakout entries with stop-in price further away from the reference price. That, in turn, cut deep into the real winners for the purpose of getting more lame winners. Such changes is not a solution to obtain better trading models.

Reduce Impact of Noises By Filters Independent of Original Drivers

This is the part that most rookie system designers hate the most. It is also what separate great traders from good ones. By paying attention to the minute details of the pre-conditions and the initial development phase of the trades, there can be subtle signs that a trade may not workout. Great traders are good at what they do because they observe and dissect the information dispassionately while average traders having their emotional roller coaster ride.

The discovery of pre-conditions that are independent from the original drivers will give you filters to remove, reduce or early terminate the noise trades.

Filters can come in all forms. Some can be time based. Some can be chart patterns (from higher timeframe). It is important to think in terms of what the oppose side would do when looking for signs to jump in. These signs are usually the best warning signal that a trade has gone bad.

2 comments

No comment yet

The forex main page is revised with more details included for each trading model.

The signals are now updated automatically right after the end of each trading day.

The premium strategies can now be purchased individually.

3 comments

When A Day Starts At 5 pm

Forex markets open continuously 5 days a week, 24 hours a day. The usual time the retail forex markets open is on Sunday at 5 pm, North America Eastern time, until Friday at 5 pm in the same week. Some brokerages offer trading outside of this schedule, but majority of the trading activities happen within the normal trading hours. So one trading day with the forex markets is normally 24 hours from 5 pm to 5 pm Eastern time.

But Sometimes It Starts At 6 pm

Since 2007 US introduced the new Daylight Saving Time (DST) schedule, there is a one week period at the start of DST and another one week period at the end of DST that the open time for the week becomes 6 pm on Sunday and ends on Friday 6 pm. It happens due to the fact that other countries, especially European countries, do not follow the US time to operate their businesses.

International banks cannot just move their settlement time earlier on foreign exchange transactions. Once the European countries and North American countries (US and Canada) having their time realigned after that 2 special weeks, the 5 pm Eastern time kicks in again as the normal time for marking the start (and end) of a trading day.

Not All Daily Data Are The Same

The 24 hour timeframe used within DaytradingBias.com on forex data uses the definition above for a trading day. The reason why it has to be so specific because people often use daily data from their brokerage trading platform without questioning how the trading day is recognized. Very often, the data is pre-generated using fixed rules of which data is collected off the time their computer servers are set to. Hence the historical daily data you have may not be what you think they are.

Data inconsistencies can lead to disastrous results. When you use daily data in combination with intraday data for historical strategy testing, they have to be representing the exact same period of time for the strategy test to be valid. For example, the daily data may contain information leaked from future intraday data because of the way the daily bar was constructed. It can happen easily as there is no standard rules for the daily data collection to follow.

Unless you have verified the intraday data against the daily data yourself to confirm the accuracy of the data, do not assume the daily data is collected the way you think it is supposed to be.

A Trap

In case you are still confused of what I am talking about, let’s take a look at the following situation.

Brokerage A offers its own trading platform with both daily and intraday historical data for charting and strategy testing. Their server time is set to UTC standard time to collect daily data. Sounds good so far isn’t it? At least they are collecting daily data using the universal standard time, right?

The catch is that most trading/charting platforms chart the intraday data using the client’s local computer time. It is the preferred method, by the way, to use the local time so that retail clients, who in general has no idea what international time zones mean, will not call to complain that the time is not correctly displayed on their charts.

If you are not aware of the potential problem, you would assume the daily charts are showing daily bars containing data from 12 am to 12 am, your local time. That means, you are likely having the wrong high/low/close information on the daily chart because there are a total of 12 major time zones. A mismatch is very likely.

For those who use indicators, that turn in your daily bar chart with your favourite oscillator could be using data from today as oppose to the close of previous trading day you are thinking of.

We are talking about trading signals generated from incorrect data. Your money is on the line. It is a big problem.

Workarounds

To align your intraday charts with your daily charts, there are several workarounds you can try.

First, you need to know how the daily data is collected by your brokerage. This can be done by a little reverse engineering. All you have to do is to load up a week of hourly bars and compare that to the daily bars. By checking the day close values on the daily bars to match against the hourly, you should be able to tell the cut-off time quite easily.

Now, you have several choices to proceed from here.

You can set your computer to the time zone matching the one the daily data is collected by your brokerage. What I mean is that you can change your computer time zone so that its midnight time matches the time they have the cut-off time for a day. That way the daily data would match the intraday data properly at once.

This method solves the problem of the daily and intraday data not aligned properly, which is good enough if you just need a way to deploy trading models without false signals triggered, or that you are daytrading and need the properly higher timeframe readings.

Another way to deal with the issue depends on your trading / charting platform. If your platform can produce 24 hours chart from hourly or minute data, you can generate the correct daily data with respect to any time zone you are using. All you need to do is to use 24 hour bars in place of your daily bars in the charts to get the correct information. They will be aligned correctly with your intraday data automatically as they are all set to the same time zone.

Now, if you want to research on forex data the way I do, you need to set your computer (or your trading/charting platform if it can do that) to use Eastern European Time including the standard European Daylight Saving Time schedule. Then, use 24 hour charts in place of the regular daily charts. It will match the way I am using my historical data and you will be able to see the daily patterns I mentioned in the site.

No comment yet