Mean Deviation vs. Standard Deviation
By Lawrence
A great article presented by Mr. Nassim Nicholas Taleb on the abusive use of Standard Deviation is a must read by anyone who is involved with trading. The article explained why standard deviation is not really a standard and how it is less than desirable as a yardstick to measure volatility in data series. What I am going to do here is to show how standard deviation measure up to mean deviation in actual market data.
When Not To Apply Standard Deviation
Standard deviation is not applicable in general in all time series analysis on financial data. The reason is very simple. When you have data that is expected to be distributed like a normal distribution (i.e. bell curve), the standard deviation formula gives you something meaningful. The moment you cannot be sure if there exists sure a distribution, yet you are still using standard deviation to estimate the behaviour of the data set, you are abusing the formula and making wrong inference about the data.
Since all data from financial markets demonstrate fractal-like behaviour with very fat tail, the use of standard deviation in any analysis is inappropriate. Unluckily we’ve observed such abuse in financial forecast everywhere. It is a shame to the human race that people call themselves educated yet plotting all these regression lines and reporting all kinds of boundary studies based on standard deviation figures on data that should not be handled that way.
Mean Deviation Is The Better Alternative
Mr. Taleb suggested in his article that Mean Absolute Deviation (MAD) is a better measure of distribution behaviour. In his article, Mr. Taleb already provided compelling evidence that MAD is superior to STD for all real-life data. I am not going to repeat what he says in the article. It is a good idea to at least browse through his article when you get the chance.
In my experience, standard deviation often reacts incorrectly while mean deviation does not suffer the same problem. In other words, if you choose to use standard deviation as your volatility measure, you will often encounter false positive signals because of sudden surge in standard deviation readings. If you substitute standard deviation by mean deviation, the problem will likely go away, giving you better results overall.
A Visual Explanation
Following is a Emini S&P 5-minute chart with Standard Deviation band and Mean Deviation band overlay onto the price series.
Two things stand out from the chart immediately.
First, the mean deviation lines (green lines) almost always bounding a tighter range in comparison with the range coverage by the standard deviation band.
Second, standard deviation band often continue to expand while mean deviation band already collapse in range.
It is a simple example showing why it is better to avoid standard deviation in your indicator toolbox.
The Mathematical Explanation
For those who wonder why standard deviation behaves like a bad boy, here is the technical explanation without using mathematical equations.
Standard deviation formula on a moving time window can be expressed by an induction formula. What that means is that as long as you know enough about the state of the standard deviation value up to the previous instance, you can derive the new standard deviation value from the state information and the input of the latest data point. This property implies standard derivation does not really give you completely new analytical information based on all the data points at this point. It is mostly derived from the past, the out-dated data.
Mean deviation is completely different from standard deviation. Every instance where you have to evaluate an answer, you need to completely recalculate the result based on all the data points again. No cheating like standard deviation, so to speak. Hence, every results you get from mean deviation is actually a completely new look at the data, giving you fresh information and new perspective.
Reference
2014: What Scientific Idea Is Ready For Retirement – Response from Nassim Nicholas Taleb
