For a number of years, I have mused over the equity premium puzzle, and have generally written that the premium return that equities earn over stocks is less than most asset allocators assume. In January 2006, wrote an article on this topic at RealMoney: Kiss the Equity Premium Goodbye.? A few quotes:

This article won’t win me a lot of friends in the money management industry. Here’s the skinny: Stocks are unlikely to return much more than bonds over the next 10 to 20 years.Most investment consultants tell people to invest in equities because, in the long run, stocks beat bonds and cash. I agree, but how big is this advantage? Many studies suggest that the equity premium is somewhere in the vicinity of 6%; i.e., stocks beat cash by 6 percentage points a year. Against bonds, the advantage is said to be 4% or so.

However, there are persuasive arguments that the value of the equity premium will be much lower going forward. In the book Triumph of the Optimists, Elroy Dimson, Paul Marsh and Mike Staunton argue that the future equity premium in the U.S. is likely to be closer to 4% over cash for two main reasons:

• Corporate cash flows have grown faster in the last 50 years than in the prior 50, and investors have bid up stocks as a result. However, the authors believe that such high rates of growth will not continue. If corporate cash flow growth reverts to the rate of the first half of the 20th century, future returns based on current equity values will be poor indeed.

• Perceived risk in stock investing has diminished. Investors have bid prices up in anticipation that the equity premium is higher than it should be, and on the belief that it is not risky to try to capture it.

The researchers Peter Bernstein and Robert Arnott draw similar conclusions, but they get there in different ways. They point out that over the years, the size of the future equity premium has varied with the level of belief in its existence. When market players deny its existence, equity valuations are low, past equity performance has probably been poor, and the future equity premium is large — think of 1931, 1937, 1974, 1982, November 1987 and 2002. When everyone believes in the inevitability of stocks, ? la “Dow 36,000” (we’ll get there by 2025 or so), equity valuations are high, past equity performance has probably been great, and the future equity premium is small — think 1929, 1972, August 1987 and February 2000.

I believe stocks have been bid up because of the benefit needs for the retirement of the baby boomers. Though the savings rate is low, investment vehicles such as pension plans have made large commitments to equities, partially because plan sponsors can justify lower contributions to benefit plans by assuming a high rate of return, which stems from assuming that the equity premium will persist.

and this:

This doesn’t directly generalize to the market as a whole, because all stocks are owned by someone at the end of each day. Isn’t there always a buyer for every seller, and vice versa? Yes, but they aren’t always public-market buyers and sellers. Cash comes into the market via IPOs (both primary and secondary), rights offerings and any other way that new shares get created for the payment of cash. Cash comes out of the market through dividends, buybacks and any other way that companies disburse cash to shareholders, whether directly or in exchange for shares.

Companies tend to sell stock when it is advantageous; IPOs happen more frequently when valuations are high, and buybacks happen more frequently when valuations are low. This suggests a project for future study: Calculate the dollar-weighted return for the public equity market as a whole, and compare it with the time-weighted return figures.

It’s a difficult but not impossible project, but I don’t have the time or resources to do it. If it hasn’t been done already, it might make a rare practical Ph.D. thesis for someone. Returns for the market as a whole would equal the change in market value, plus the cash cost of shares taken out of the market (buybacks, LBOs, etc.) and dividends, less the cash added to the market through IPOs and other forms of share issuance for cash (i.e., employee stock option exercise, rights offerings, etc.). I don’t have firm numbers, but my guess is that dollar-weighted returns are less than the time-weighted returns by 1 percentage point, give or take 50 basis points.

Though the composition of an index fund changes by period to reflect additional equity issuance/buyback by companies in the index, it misses the effect on returns from having to allocate more capital when valuations are high, and having to receive capital back when valuations are low. In a whipsaw period like that which we have had from 1998 to the present, it makes a lot of difference, because many investments during the bubble era put fresh capital into the market at a time of high valuations, with buybacks predominating as valuations troughed.

In short, though the academic studies rely on time-weighted rates of return for their conclusions regarding the equity premium, which represents buy-and-hold investors, dollar-weighted returns, which is what most investors actually receive on their investments, are lower. The difference occurs because corporations issue stock when valuations are high, and retire it when valuations are low.

Okay, here’s the punchline — with not just a hat tip, but a full bow to Eric Falkenstein at Falkenblog, Ilia D. Dichev has done the research that I wanted to see done.? The difference between time-weighted and dollar-weighted returns is around 1.3% for NYSE stocks (1926-2002), around 5.3% for NASDAQ stocks (1973-2002), and 1.5% for developed market stocks generally? (1973-2004).

Doing a very rough average, and considering that the NASDAQ was in a boom period for most of the study period, I am comfortable with a reduction in the US equity risk premium over bonds down to 1-2% on average, and over cash to 3-4% on average.

At that level, being in stocks works for long term investing, but it would almost never pay to be 100% in stocks.? The old 60/40 stocks/bonds allocation begins to look really intelligent over the long haul, but maybe not today because high quality bond yields are so low.

So, where does this leave me on the equity premium puzzle?? It is no longer a puzzle.? One can gain moderately over the very long haul in stocks versus bonds, but with significant volatility.? Don’t risk what you can’t afford to lose in the stock market, and other risky investment vehicles.

PS — this makes the old dictum on the cost of equity valid again — the cost of equity capital for a firm should be 2-3% above their longest bond yield.? Bye, bye, CAPM.