Trading System 101 – Part 3 Practical Considerations

Many traders spent weeks, even months to create their perfect trading models. Happy with the results they see in their backtesting, and tracked the real-time performance for certain period of time, they are ready to take the leap of faith to trade their models live.

Could that be just a bit too early? Well, that depends on how prepared the person is. Unluckily, I have to say that many of these traders (rookie system designers) are getting their worst nightmares and rudest awakening.

Historical Testing (And Real-Time Tracking) Can Be Misleading

The title to this section says it all. Most beginners thought that they have discovered the perfect system very early in their career of system development. Well, it could be true, for a very small percentage of them. For the other ones, it could simply means yet another lesson to learn.

There are a lot of traps in trading system backtesting. Depending on the platform you are using, it may have one or more of the weaknesses that I am going to talk about. If the platforms you are using suffer a particular weakness, make sure you do not use the system testing results as-is. You can post-process the trades recorded by the system and remove the ones that are not realistic. That way you can get a better picture on how your trading system may actually perform in real-life conditions.

I am going to outline a few common issues that make your backtesting less reliable than you think. If your system suffers any one of these problems, take the time to either fix these issues within the system first, or, as a minimum adjust the system results manually. Remember – you need an accurate account of your system before you can make intelligent choices whether you want to trade it live or not.

1. Trades Made At The High Or Low Of A Bar May Not Get Fill In Real-Life

This one is not new. Ever since the first historical backtesting program was invented, this issue has haunted system testers all along.

In short, when your trading system made a trade at the high or low of the price bar it is placed on, it may not get filled at all in reality. Why? The order you placed at that price could be just one of many orders placed at that price, thus, there is no guarantee that your order will be filled. The historical data simply declared the fact that some trades took place at the high or low price of the bars, but it never states that you are guaranteed to get filled at those prices.

So, if your system uses limit orders extensively, and it got filled a lot at the high or low prices of the price bars, then you should be aware that your system may not be performing like your historical test at all. At best, the real-time trades will be filled frequently at the bars after the ones that are supposed to get filled. In the worst case, when your system truly picked a local high or low, the those trades may never happen.

Another situation that this may happen is that the system is trading at a very high resolution (i.e. 5-second bars are higher resolution than 1-minute bars). That alone can create many data bars with only 1 to 2 ticks range, making it very hard to get fill at the high or low of those data bars in real life situation.

There are two possible ways to correct this problem.

First, you can modify the system to act a bit more aggressively, by changing your system logic to place orders 1 tick better than the price you originally designed. If that does not take care of most of the orders that touch the highs and lows of the price bars, then you have to make your system even more aggressive. Eventually your system will have 95% of its trades not touching the extremes of the price bars. That way, you will be confident that your system will not be affected by this problem.

Second, if your platform supports some sort of worst case scenario analysis, then you can enable such option so that trades that happened on the extremes of the price bars are simply rejected. In NeoTicker, you can set the system to fill on Worst Case or Limited Worst Case scenarios. That way you can tell if the worst case situation actually happened, and how your system will fair in such adverse situation.

2. Systems That Fire Orders Too Frequently Will Have Inconsistent Trading Results

In between you, your broker, and your trading system, there are many unknown factors. If you trade with online order placement, your ISP, Internet itself, and simply the hardware of each involved parties could fail you and hold-up your orders. When an order is placed during the break down of one of these components, it is fair to say that the affected orders will be hanging in a pretty undetermined state. Even if the order went through perfectly from you to the broker, the time it will take to confirm the order is still an unknown.

Thus in many situations, a system that fire orders at or close to the average reporting speed of your brokerage will not be able to obtain the necessary order status in a timing fashion. That translate to difference from the historical testing that order results are obtained instantaneously.

To resolve this problem, your system has to be designed to compensate for the time lag involved. For most trading system platforms, orders are generated on a bar by bar basis and not really following a real brokerage account model. The advantage of such design is that historical data testing can be done easily. The disadvantage of such system is that if the orders are routed to your brokerage, you will have so many problems that your system will simply break down.

The key solution here is the ability to manage the orders directly within the system.

In short, you would want to send orders that does not survive just one bar, but have their own proper time in force setting so that you can control what to do with the orders when they are filled, or, not filled, which you can then have a choice to properly cancel them, or start to manage the position by placing new orders.

3. Your Order May Never Get Filled Due to Liquidity Issues

For thinly traded instruments, this is the most common problem affecting trading systems that are not properly designed to handle such situation.

Historical data does not really tell you how many shares or contracts you can trade at a particular price even if the historical price bar traded through that particular price. You cannot assume that you can simply throw an order with huge number of shares or contracts at a particular price and expect that being filled completely, even if the price you set does not suffer from problem number 1.

This particular problem does not really affect you unless you are very careless and throw money at the system without even monitoring at the price actions of the target instruments for a few days.

If you are careful, then you probably would have guessed that your best bet for your orders is to not exceed the average trade volume for the current data bars. That way, even if your order is a large one, it will still get a chance to get filled without being noticed by the other traders.

Thus if you can incorporate average volume into your system to determine the size of your orders, you may want to do that so that you can see the effect of the constraints in order size for the particular instrument you trade.

For instruments with high liquidity, such problem still persists, but only during price extremes (like melt up and melt down in price), which you can manually invent the system to not placing orders until after such incident is over.

4. Your Order May Modify The Market Behaviour

This particular problem does not happen very often. It only happens when you are luck enough to discover models that work pretty well. Usually that results in quick success in the markets you trade, with bigger and larger bets in a short period of time.

You may probably feel invincible for a short period of time, but your peers who trade in the same market will adapt to your existences and trading styles. After all, you are taking money from them trade after trade. Thus if you rule a market for a period of time using a single model, it is likely your opponents will catch up and that alone will modify the market behaviour enough that makes your model no longer useful, or, as a minimum, without the edge it was originally designed with.

This is called the discount process of your model. The greater success your model has, the faster the discount process matures.

To avoid this problem completely is impossible. As I suffered that early in my career, I found recovering from the damage of this problem could take a long time. To avoid going down the same path I suffered, the most important thing to do, is to monitor the statistical behaviour of the data that your system is depending on – when that changed, you have to put your system in the watch list to stay high alert and pull the plug if necessary.

Summary

There are many ways to make your trading system more realistic. For example, the fill mechanism I mentioned above based on Worst Case scenario, or, using something like NeoTicker’s simulation server to conduct tick by tick replay to reconstruct the complete testing situation. All these efforts you spent are useful if and only if you are willing to accept the fact that you have to spent enough time to properly shape your model for real life usage.

If you are listening and understand the importance of properly avoiding the issues I have mentioned here, the systems you design will enjoy a much longer survival time.

Share