The aim of actively-managed funds is to beat the stock market, but precious few manage to do so. This simple fact of life is perplexing to ordinary investors. Such funds, after all, employ scores of well-educated professionals who scour the markets for stock-picking opportunities. Why can’t they deliver? Why should it be so difficult to outperform the overall market?

In truth, there are a host of reasons why the vast majority of funds are destined to continue their underperformance in the following decades.

1 Do the maths

The vast majority of stock market activity is driven by fund managers and market professionals. You can’t expect most of them to outperform the market – they are the market. If they outperformed, noted renowned investor and author

Benjamin Graham

, “that would mean that the stock market experts as a whole could beat themselves – a logical contradiction”.

Even if a fund manager does beat the market, fund investors are likely to be suffer after fees are deducted (actively managed funds are invariably more costly than passive funds that blindly track indices, due to the extra research and trading fees involved). "The laws of addition, subtraction, multiplication and division" mean that most active funds will underperform, said Nobel laureate William Sharpe. "Nothing else is required."

2 Career risk

It is hard to argue that markets are perfectly efficient, given that they can lurch from extremes of overvaluation (dotcom mania in 1999) to obvious undervaluation (like in early 2009, in the aftermath of the global financial crisis). Surely, one might ask, any half-decent fund manager should be able to easily profit from such extremes?

That ignores the obvious problem of career risk. Savvy managers who chose to pass on technology stocks in the late 1990s were pilloried for doing so, with investors yanking money from their underperforming funds.

"We were told we were complete idiots and several clients banned us from their buildings," admitted James Montier of GMO, the renowned money-management firm headed by legendary value investor Jeremy Grantham.

That is echoed by another value investor, Jean-Marie Eveillard, who had earned an enviable reputation after hugely outperforming global markets over a two-decade period. That didn’t stop clients deserting him after his fund began to badly trail markets, fuelled by insane technology stock valuations, in the late 1990s.

“After one bad year, investors were upset,” said Eveillard. “After two they were mad and after three, they were gone.” By early 2000 – just before the tech bubble burst bubble – impatient investors had withdrawn two-thirds of his fund’s assets.

Everybody saw the dotcom bubble, Jeremy Grantham has said, but fund managers risked being fired if they skipped the party. Investment behaviour is driven by career risk, he says. The prime directive “is first and last to keep your job”.

3 Competition

In an inefficient market, it is easier to exploit mispricings. Conversely, it is more difficult to outperform in a highly efficient market, where opportunities are quickly snapped up by competitive and eagle-eyed professionals.

Active investors often point to the success enjoyed by renowned value investor and author Benjamin Graham, who profited in the decades after the 1930s Depression. By the 1970s, however, Graham acknowledged that the game had become more difficult.

“In the light of the enormous amount of research now being carried on,” he said, it was unlikely that extensive stock-picking research efforts “will generate sufficiently superior selections to justify their cost”.

Today’s markets are more competitive than even Graham could ever have imagined. Back in the 1990s, for example, hedge funds could justify their high fees by pointing to their stellar outperformance. However, competition has since increased enormously – the number of hedge funds globally has ballooned from fewer than 600 in 1990 to over 10,000 today.

Consequently, the easy money is no longer to be found, and hedge funds as a whole have underperformed conventional 60-40 portolios (60 per cent equities, 40 per cent bonds) every year since 2002.

4 Closet indexing

There is another obvious reason why many managers fail to beat their benchmark – they are not even trying to. Roughly a third of US fund assets are invested in so-called closet

index funds

, according to a 2013 study by former Yale professor

Antti Petajisto

.

These funds purport to be actively managed, but in reality are made up of many of the same stocks that comprise the benchmark index (Petajisto defines a closet index fund as one where less than 60 per cent of its investments differ from the benchmark).

Petajisto refers to “active share”, the fraction of a fund’s holdings that differs from the benchmark index. A fund with a share of 0 per cent, for example, would be a pure index fund, while a 100 per cent active share would have no overlap whatsoever with the benchmark.

He gives the example of Fidelity Magellan, the fund managed by iconic investor Peter Lynch between 1979 and 1990. Lynch's enormous success led to a flood of funds to Magellan. However, Fidelity made a "conscious decision to become a closet indexer", the study found, with its active share falling from almost 90 per cent in Lynch's time, to just 33 per cent in 2000.

Closet indexing has become an increasing problem in recent years, according to the study. Essentially, investors are being sold little more than expensive index funds that cannot possibly beat their benchmark.

5 Cash holdings

Fund managers must maintain an allocation to cash in their fund, to allow both for stock purchases and investor redemptions. Unfortunately, fund redemptions tend to be greatest in the aftermath of a crash or bear market, due to an outbreak of investor panic. Fund managers may need to sell stocks at such moments to meet investor redemption requests. In other words, instead of profiting from market bargains, they are forced to sell at the worst possible time.

6 They’re human

Cash holdings are always very high at market lows, but it’s not just because of investor redemption requests – fund managers are as prone to panic as anyone and most are too spooked to dive in during times of panic.

Take the case of Nobel economist Harry Markowitz. On Black Monday in 1987, when stocks fell by over 20 per cent, Markowitz knew what adjustments he needed to make to his retirement account. However, his emotions got the better of him. "I visualised my grief if the stock market went way up and I wasn't in it – or if it went way down and I was completely in it. So I split my contributions 50/50 between stocks and bonds".

Markowitz helped to found modern portfolio theory and sees man as a rational economic actor. If he couldn’t overcome his emotions, there’s little reason to think that petrified fund managers – under pressure from their clients and bosses and fearful of career-ending losses – can do so.