Simple computer models invariably trump expert human judgment, whether predicting wine prices, making medical diagnoses, analysing job performance or even assessing a couple’s marital stability. Is investment any different?
Can informed investors outperform models by combining their own assessments with quantitative screening techniques? And if the quant approach is better, why must most investors rely on the subjective judgment of fund managers?
A number of colourful anecdotes illustrate the superiority of the algorithmic approach. In 1990, Princeton economist Orley Ashenfelter was excoriated after producing a simple weather-based formula for predicting the quality of wines. The 1989 Bordeaux – not yet tasted by critics at the time – would be the "wine of the century", he said, and 1990 would be even better. Although derided as "ludicrous and absurd" by oenophile Robert Parker, Ashenfelter's predictions were spot on.
Parker still dismisses Ashenfelter's "Neanderthal" model, which he compares to a movie critic "who never goes to see the movie but tells you how good it is based on the actors and the director". Psychologist and Nobel economics winner Daniel Kahneman disagrees: Ashenfelter's predictions have an accuracy rate higher than 90 per cent.
Algorithms need not be complicated. Marital stability is well predicted by a simple formula: frequency of lovemaking minus frequency of quarrels (“You don’t want your result to be a negative number”, adds Kahneman, who notes even back-of- the-envelope algos are “certainly good enough to outdo expert judgment”).
These are not isolated examples. One famous study, Clinical Versus Mechanical Prediction: a Meta-Analysis , examined 136 published studies comparing the accuracy of simple quant models to expert human judgment. The studies ranged from diagnosis of heart attacks to psychiatric assessment, from the risks of criminals reoffending to college performance. The models were better 46 per cent of the time, and beat or equalled experts 94 per cent of the time. On only eight of the 136 occasions did experts outperform, and always in cases where they had more information.
The superiority of mechanical prediction “holds in general medicine, in mental health, in personality and in education and training settings”, the authors concluded. The mechanical models were consistently better, “regardless of the judgment task, type of judges, judges’ amounts of experience, or the types of data being combined”.
Investment implications
The investment implications are obvious. What about a "quantemental" approach – a blending of fundamental and quantitative analysis – whereby skilled investors use quant screening techniques but also utilise their own investment experience? Behavioural finance expert and market strategist James Montier is not hopeful.
He once developed an asset allocation tool based on a combination of valuation and momentum. The model worked well initially, generating signals confirming his own bearish disposition, but then its indicators turned bullish.
“I chose to override the model, assuming that I knew much better than it did (despite the fact that I had both designed it and back-tested it to prove it worked) . . . I spent about 18 months being thrashed in performance terms by my own model.”
Again, this is not a cherry-picked example. In many of the studies examined in the aforementioned meta-analysis, notes Montier, experts were given access to the quant findings and still managed to underperform the model. “The evidence is clear”, he says. “Quant models usually provide a ceiling (from which we detract performance) rather than a floor (on which we can build performance).”
Take the case of renowned hedge fund manager and author Joel Greenblatt, whose value-based quant strategy trounced the market over two decades. Greenblatt's firm, Formula Investing, later offered investors a choice: invest systematically according to the algorithm or receive the model's output but use discretion as to the stocks chosen.
Between May 2009 and April 2011 the model returned 84 per cent compared with 63 per cent for the S&P 500. The self-managed accounts, however, returned just 59 per cent. As Greenblatt noted, they "took a winning system and used their judgment to unintentionally eliminate all the outperformance and then some".
Overvaluing information
In a recent paper, quant investor Wesley Gray noted a study in which participants took part in a live trading game, with some given privileged information. Those given perfect insider information beat the market. However, those given partial insider information did not. In fact, they were outperformed by the uninformed participants. Why? "They suffer from overconfidence and overvalue their own information set, and therefore can't use it effectively," Gray said.
Not only are investors undermined by overconfidence, they are led astray by irrelevant information. In one revealing experiment, people were asked if they would approve a mortgage application to a well-paid graduate who, it is discovered, has not paid a recent $5,000 credit card debt. Just 29 per cent approve the application. A second group were then asked the same question, with one twist – it is unclear if the unpaid debt is $5,000 or $25,000, and the correct answer will not be revealed until the next day. Most people opted to wait to hear the correct sum before deciding. The next day, they are told the sum owed is $5,000, not $25,000. A majority in the second group – 54 per cent – approve the application.
Both groups have been given the same information, writes Dr Gray, but the second group perceives it has more information as it has been meted out over time.
“Humans are cognitively inclined to overvalue information that requires effort or time to obtain,” he concludes – even if it is utterly useless. Countless studies confirm the same points: we are overconfident, we overweight the value of our opinions relative to others and we are misled by what academics politely label “non-essential” information.
Obvious conclusion
The late Paul Meehl, a psychologist and advocate of mechanical models, once said no other controversy in social science "shows such a large body of qualitatively diverse studies coming out so uniformly in the same direction", adding: "It is time to draw a practical conclusion."
That was in 1986. Quant investing is now more mainstream but remains a fringe field. Why? Well, it would help if there was a move towards "making the news nerdier", as US stats guru and bestselling author Nate Silver puts it. There is also an obvious self-serving bias – analysts and fund managers are unlikely to welcome the prospect of being replaced by computers, as James Montier has pointed out.
Whatever about the options, investors should consider the inherent problems of a discretionary approach. Former Davy analyst Neil Osborne, now chief investment officer at Dublin-based Covestone, has argued there is "no reason to think investing should be different to, say, medicine or the social sciences in terms of the effectiveness of statistics versus intuition and judgment".
At a minimum, ordinary investors should devise some simple rules, he recommends, such as “I will buy a stock only if it’s cheap and not because I like the story” or “the price I bought a stock is an irrelevant anchor, I must sell it when its price exceeds what I deem to be its value”.
Whatever one’s rules, it’s best to stick to them rather than allowing personal opinion to override them. As Dr Gray warns, the “secret sauce of human judgment ruins the beautiful simplicity of a calculation”.
