In the last decades, finance professors told the world that financial markets are random and the only model that works is the random walk model. If the financial markets are random, it is then very difficult to predict them using technical analysis. For the technical analysis to work, financial markets need to be non random for sometimes if not all the time. Stock market random walk became a dogma. But recently the tables have been turned on this random walk dogma and a new subject of behavioral finance has proven using quantitative methods that financial markets can be non random and predictable. Did you read the post how behavior finance can help investors?
There are now a number of behavioral finance theories that explain in detail how financial markets can be non random to some extent and predictable. If this is true then technical analysis can provide us with some relief. When you backtest a technical analysis based trading strategy and the backtest results are positive that it gives some credence to the non randomness of the financial markets.
We need a theory that can provide a sound theoretical framework that explains most of the things that we observe in the financial markets. Commodity markets are well known to trend well. Futures market has been developed to transfer risk from hedgers to speculators. Behavioral finance stipulates that the profitability of commodity futures trend following systems is a compensation to the speculators for risk transference from commercial hedgers. So our theory of behavior finance stipulates that commodity futures market should show enough profitable trends to entice speculators to take on the price risk that the commercial hedgers like the multinational firms want to shed. Watch this interview with a billionaire hedge fund manager.
For behavior finance to succeed as a good theory of financial markets, it should not only explain a wide variety of phenomenon observed in the financial markets but it should also be able make quantitative predictions that can tested practically. Is behavior finance better than technical analysis? Technical analysis never explains why its methods works. Technical analysis lacks the ability to explain the financial market behavior even though sometimes its methods work. Technicians brush this off by saying that they never try to explain why the market behaved the way it did.
Technical Analysis And Efficient Market Hypothesis (EMH)
The fundamental premise of technical analysis is that anything that can affect the price fundamentally, psychologically, politically, economically gets reflected in the price immediately. Now this is a very vague statement that cannot be tested quantitatively. But the premise on which technical analysis is based is fundamentally flawed. Just think over the statement that price reflects all the possible information. If this were true and price immediately reflected all the available information then it would not contain any future prediction in it. Can hedge funds bring a global financial meltdown?
Suppose, stock ABC is at $50 per share. You observe a double bottom pattern that predicts that stock ABC price can easily go above $60. But if the basic premise of technical analysis is true than the pattern is already reflected in the stock ABC price and there should be no forecasting ability in it. Now Efficient Market Hypothesis (EMH) also based on this premise that price reflects all available information. So how come EMH contradicts technical analysis when both are based on the same fundamental premise. We need to behavior finance that can explain many shortcoming of technical analysis and the EMH.
Prices need to be nonrandom to some extant for the financial markets to work efficiently. This is the new thinking on financial markets. There are many skeptics of EMH now who question its validity and it is no longer a dogma that it was in the past. Let’s discuss Efficient Market Hypothesis (EMH) and see what is says and then discuss the implications of those sayings. EMH founders lime Eugene Fame stipulated that random walk was a necessary condition for an efficient market.
In an efficient market prices reflect all the known and knowable information.So the current price is the best estimate of a security value. An efficient market cannot be beaten no matter what you do. No fundamental or technical trading strategy can produce risk adjusted returns that can beat the market index returns. An efficient market is comprised of millions of rational traders, investors, hedgers and speculators. These rational market players are constantly observing the price and adjusting their buy and sell decision that on a collective level ensure that price is at its rational level.
In the world of efficient markets, prices can only change when new information is released. When new information is released price immediately jump to the new rational level. When there is no news, price fluctuates randomly around the rational level. So in an efficient market, price should follow a step function and there is no chance for a trend to develop. So trend trading has no place in an efficient market. But practically we observe that price does trends at times and trend trading is a profitable trading strategy. So the EMH stipulates no trends are possible in the market while practically we find markets developing trends.
Prices in an efficient market are highly unpredictable so no form of technical analysis should work. EMH rules out profitable trading systems. But the notion of market efficiency is false. Nassem Taleb shows that even if we assume everything is highly random and the probability of beating the market is only 50%. If we have a large number of hedge fund managers say something like 10K, there will be a number of them with winning streaks. After five years, 312 hedge fund managers will have beaten the market five years in a row.
When EMH advocates were confronted with the above evidence they came up with a new version of EMH known as the Strong form of EMH, Semi Strong form of EMH and Weak form of EMH. According to this new EMH version, strong form efficient market means that all public and private information is immediately reflected in price. So it is impossible to beat a strong form efficient market. In a semi strong form efficient market all public information gets immediately reflected in price. Now for a market to be semi strong efficient, news should get reflected into price immediately. This means that price should not under react. This also means that price should also not over react.
We can test this. If the market is semi strong efficient, price should not over react and it should not under react rather it should immediately get close to the new rational level. Now if the price over reacts and overshoots the new rational level, it will try to correct itself later on by climbing down to the rational level. So if the price over reacts, then it means price movements can be predicted. In the same manner if price under reacts, it will climb up to the rational level later on. So if price over reacts or under reacts to new information, then it means its movement is predictable.
Eugene Fame the founder of Efficient Market Hypothesis studied the news events like corporate dividends and earnings announcements, mergers, acquisitions and takeover and found that an initial instantaneously price movement was triggered by the economic news event and after that there was no movement. This confirms the EMH that price quickly and accurately discounts news events. So price moves when there is a news event. What this means is that price should not move when there is no news.
What about the stale information? Information that is already in the public knowledge. We technicians are mostly trading based on stale information that is already on the charts.According to EMH, stale information has no value as it is already reflected in price. So EMH implies that technical analysis is useless. This is what EMH implies, trading systems based on stale information cannot beat the market and generate risk adjusted profits superior to the market index returns. So in order to refute EMH, we need a trading strategy that can beat the market index on a risk adjusted basis. This is important for you to understand. If our trading strategy is three times more risky than investing in the market index, it should make at least 3 times more return than the market index in order to beat it.
Now there is also a weak form of EMH that stipulates that public information gets reflected in price immediately and non public or private information may take time to get reflected in price. According the weak form of EMH, technical analysis indicators such as momentum indicators are useless as they are based on stale information. Auto correlation studies have shown that returns are almost independent of past returns. What this means is that we cannot use the past returns to predict the future returns.
However there is a caveat in this findings. Auto correlation measures the linear dependence. Variance ratio studies have shown that current price returns have non linear relationship with past returns.In an efficient market, smart investors and dumb investors have same opportunities meaning smart investors have no edge over dumb investors. This is due to the EMH premise that whatever information is known or knowable has already been reflected in the market price. This premise contradicts another EMH that stipulates the arbitrageurs act as a sort of police force and are responsible for bringing prices to rational level.
This is what’s happening in the efficient market. Dumb and irrational investors are trying to move the price away from rational level while the rational investors acts as arbitrageurs and force the price to return to rational level.For this to succeed arbitrageurs should have more trading capital than the dumb and irrational investors otherwise they wouldn’t be able to enforce price discipline in the market. Having more trading capital means that smart investors must have made higher market beating returns in the past otherwise it would be difficult to have more trading capital. These facts contradicts EMH earlier stipulation that price is a random walk and it is difficult to beat the market index on a risk adjusted basis.
There are many contradictions in the Efficient Market Hypothesis. EMH says that information has no value as it is instantly reflected in market price. So if I work hard to gather information and develop a trading strategy based on that information, I will never be able to beat the market. So how much I work hard and how much research I do, whatever I discover and develop a trading strategy based on it will have no value. In short, EMH says that there is no payoff to information gathering and processing it into investment strategies.
If information has no value then no one will incur the heavy cost in terms of time and money of doing market research.This means no one will try to make the markets informational efficient. These are contradictions in EMH theory. Investors need excess returns to gather information but at the same time says that whatever information they gather has no value. Recent research has proved that market inefficiencies are necessary to motivate smart and rational investors for gathering information and processing it. The excess market returns these smart investors earn as a result of doing market research is an incentive and compensation for their efforts on working hard on doing market research. This activity of doing market research in the long run eliminates market inefficiencies and moves the prices to their rational level.
Markets are basically zero sum games. If you buy, someone is selling so there is always a counter party who is taking the other side of the trade. Noise traders who trade the noise in the belief that it is a trading signal are the ones who finance the excess returns that the smart investors make as a result of doing market research and building trading strategies based on it. Smart investors are infact profiting from acting early on information that has not yet being incorporated in the market price. EMH theory denies all this. According to EMH, all public information gets reflected in market price and there is no delay in it.
When you take trade based on your market research information, you pay a trading cost in the form of spreads, slippage, swaps etc. There is no point in trading if you don’t get compensated for taking on these trading costs. Anything that limits trading activity, information gathering, market research, rules that stop short selling infact stop the market from becoming efficient. As said above, these activities like trading, market research, information gathering, short selling are not feasible if there is no proper compensation for doing these things. EMH simply can’t explain how these activities will be undertaken if there is no compensation for doing them.
EMH theory is based on the assumption that investors are rational, investing errors made by these rational investors are random so they cancel out collectively and keep the prices at rational levels and pricing errors immediately invite rational arbitrageurs. What this means is that prices are any time reflect the rational level. If the information is positive, rational investors immediately bid prices up. If the information is not good and negative, rational investors bid the prices down. These price changes are instantaneous and price immediately reach the new rational level.
The errors made by rational investors are uncorrelated. So if one rational investor is foolishly buying, another rational investor is foolishly selling and on the market level, these random pricing errors when summed together are zero. If the pricing errors on the market level don’t add to zero and result in biased value, rational arbitrageurs are watching.the market according to EMH theory. These rational arbitrageurs immediately act and correct the pricing error and bring the prices to their rational level.In the long run, these arbitrageurs become wealthy and irrational investors are driven out of the market when they cannot sustain more losses.
In reality investors are not rational as envisioned by the EMH theory. Investors are prone to make irrational decisions by holding onto their losing investment too long in the hope of market taking a turn. In the same manner, investors most of the time fail to diversify and increasing their tax liabilities by selling stocks too early. So investors can make wrong risk assessments, they can also make wrong probability judgements as well irrational decision framing. Investors make systematic errors when assessing risks and making their investment choices.
Behavior Finance and EMH
Behavioral Finance has developed a framework known as Prospect Theory that tries to explain these market realities which cannot be explained by EMH theory. Decision making always involves uncertainty. Prospect theory uses psychology to explain why investors hold onto losing stocks for too long and sell their winning stocks too early. The simple explanation offered by Prospect theory is that investors try to avoid the pain of lose when they hold to losing stocks in the hope that market will turn and the losses will go away.
Investors hastily make investment choices based on scanty data also know as small data error. Based on a few good earning reports, investors can conclude that the company is a growth company and invest heavily in it which will make its stock overpriced. Investors are also prone to take wrong decisions based on how the choices are framed. For example, investors will try to go for a sure profit even if it is not significant. In the same manner, investors will try to avoid a sure loss even if it is not significant as compared to a big loss. This error is know as the Disposition Effect and it explain why investors sell their winners quickly and hold onto losses for too long.
EMH premise that investor errors are uncorrelated is also borne out observation. Most of the time, investors display a herd mentality and almost similar decisions are taken by the crowd. When selling starts everyone starts to sell and in the same manner when buying starts everyone starts buying. This sort of explains how trends develop in the market. EMH argument that arbitrage brings price down to rational level is also faulty. First there is no way to ascertain what is the true rational level for market price at any moment. So it is difficult to spot price mispricing.
Arbitrageurs also don’t have unlimited capital. So if price moves too far away from the rational level for too long, arbitrageurs lack the wealth to move prices back to rational level. This is also known as Noise Trader Risk. Noise traders can keep the price irrational too long and make rational investors insolvent in the process. So an arbitrage trade that might look risk free in theory can be highly risky in reality. So EMH argument that arbitrageurs enforce rational price levels may not hold most of the time in the market.
This was precisely what happened in the case of Long Term Capital Management (LTCM) hedge. LTCM has spotted a mispricing opportunity and heavily invested in it. When the market moved against its positions for a short time, LTCM simply lacked the capital to stay in the market. So arbitrageurs like LTCM can get wiped out if they choose too much leverage and market moves against them for a short time. Arbitrage also means finding securities with identical risk profiles which is not possible in reality. So EMH assumption that arbitrage opportunities immediately get removed by arbitrageurs does not hold in practice. So in practice arbitrage has very limited power when it comes to moving prices to their rational level.
Another major contradiction is the price volatility. If prices follow fundamental news closely then they should not show volatility that is shown by the financial markets. EMH stipulates that prices move based on information. But there are instances when price moved big time without any news causing it. Especially in the stock market crash of 1987.happened without any significant move price moved 20%. A big failure of the EMH theory is the finding that price can be predicted with stale information which is already available in the public domain.
This is important for you to understand. Information does not move the price. It is the interpretation of information that moves prices. There are millions of buyers and sellers interpreting new information. Some take the new information serious others take it non serious. Some take information positive and some take it negative. So each one is taking his own decisions which then collectively move the market price. Trading is a zero sum game. If someone is winning, there is someone who is losing. Just like a tennis match, the best player wins in the end. Today hedge funds employ PhDs in mathematics and physics who use advanced mathematics in predicting market prices. Competing against these people is not easy using trendlines and fibonacci levels. This is one of the main reasons why trendlines and fibonacci levels and other technical indicators no longer work.
It has been statistically proven using cross sectional studies of portfolios of stocks the prices are predictable using stale information that is already known publicly. Momentum strategies are a proof of this fact. Price momentum is positively correlated with the recent lags and negatively correlated with distant lags. This explains why momentum persists in the short term and reverses in the long term. EMH theory is unable to explain why momentum persists. It is time to talk about behavior finance theory that has overtaken EMH theory as the new standard.
Behavior Finance uses economics, cognitive psychology and sociology to explain most of the things the EMH theory cannot explain like irrational investor behavior. According to behavior finance, markets comprise mixture of behaviors ranging from fully rational to fully irrational with all the variations in between. Behavior finance is based on the premise that arbitrage has limited power to move prices in the market and investors are not all fully rational. Some are rational, some are irrational, Arbitrage is not risk free as there are no perfect correlations between two securities. So we don’t have a risk free trade when we short one security and long another security.
Then there are limits to investor rationality. Under conditions of uncertainty, cognitive psychological studies have shown humans tend to make systematic errors. So under a given set of circumstances, behavior finance can predict what type of human decision errors will most predominantly be made. According to behavior finance, investors are always making biased decisions that ensure that market prices always deviating from rational levels. Price does eventually reach rational levels but it can take a while unlike EMH theory that says price reaches rationality pretty soon.
Let’s discuss what types of biases investors can show. The most important bias shown by investors is known as Conservatism Bias. When new information is released, most investors tend to give little weight to it which means investors tend to ignore new information and under react to it. Over time, prices do eventually reach their rational levels as investors catch up with new information. This explains how trends are created in the market.
Conservatism Bias And Confirmation Bias
This conservatism bias is reinforced by confirmation bias. Humans tend to give more weight to information that supports an existing belief and less weight to information that contradicts an existing belief. So when information supports an existing investment decision, investors overreact by giving it more weight. Similarly when information contradicts an existing investment decision, investors tend to down weight it and under react to the new information.So confirmation bias forces investors to cling to their existing investment decisions unless they see a streak of contradictory evidence that contradicts their prior decisions. Couple this with error of small numbers that can force investors to make erroneous decisions.
Under complex uncertain situations, investors tend to simplify. This simplification is done through a process known as Anchoring. This is how anchoring is done. Investors made an initial estimate of their investment decision based on preliminary information known as Anchor.Later on upward and downward adjustments as new information comes in. The problem is most of the time investors choose a completely irrelevant anchor in making the estimate and when the estimate is relevant future adjustments can be too small.
Anchoring
Anchoring is responsible for investor under reaction to new information. Investors under react to bearish and bullish news.In case of bullish news, under reaction means price is cheap and need to rise to catch up with the true level. In the same manner, in case of bearish news when investor under reacts, price is dear and over time, it will need to fall to catch with the new true level. This explains how trends get developed in the market. Anchoring also explains why momentum strategies work. Traders anchor on a certain high like 52 weeks high and then under react to bullish or bearish information. Investors always tend to be overconfident about their investing skills and interpretation of public information. Overconfidence and overoptimism results in investors pushing prices too much which eventually became a cause of reversals. I have developed a course- Behavior Finance for Investors in which I explain how knowing behavior finance can make you a better trader and investors.