On Benchmarking
Sorry for not posting yesterday, there were a number of personal and business issues that I had to deal with.
Sometimes I write a post like my recent one on Warren Buffett, and when I click the “publish” button, I wonder whether it will come back to bite me. Other times, I click the publish button, and I think, “No one will think that much about that one.” That’s kind of what I felt about, “If This Is Failure, I Like It.” So it attracts a lot of comments, and what I thought was a more controversial post on Buffett attracts zero.
As a retailer might say, “The customer is always right.” Ergo, the commenters are always right, at least in terms of what they want to read about. So, tonight I write about benchmarking. (Note this timely article on the topic from Abnormal Returns.)
I’m not a big fan of benchmarking. The idea behind a benchmark is one of three things:
- A description of the non-controllable aspects of what a manager does. It reflects the universe of securities that a manager might choose from, and the manager’s job is to choose the best securities in that universe.
- A description of the non-controllable aspects of what an investor wants for a single asset class or style. It reflects the universe of securities that describe expected performance if bought as an index, and the manager’s job is to choose the best securities that can beat that index.
- A description of what an investor wants, in a total asset allocation framework. It reflects the risk-return tradeoff of the investor. The manager must find the best way to meet that need, using asset allocation and security selection.
When I was at Provident Mutual, we chose managers for our multiple manager products, and we would evaluate them against the benchmarks that we mutually felt comfortable with. The trouble was when a manager would see a security that he found attractive that did not correlate well with the benchmark index. Should he buy it? Often they would not, for fear of “mistracking” versus the index.
Though many managers will say that the benchmark reflects their circle of competence, and they do well within those bounds, my view is that it is better to loosen the constraints on managers with good investment processes, and simply tell them that you are looking for good returns over a full cycle. Good returns would be what the market as a whole delivers, plus a margin, over a longer period of time; that might be as much as 5-7 years. (Pity Bill Miller, whose 5-year track record is now behind the S&P 500. Watch the assets leave Legg Mason.)
My approach to choosing a manager relies more on analyzing qualitative processes, and then looking at returns to see that the reasons that they cited would lead to good performance actually did so in practice.
Benchmarking is kind of like Heisenberg’s Uncertainty Principle, in that the act of measurement changes the behavior of what is measured. The greater the frequency of measurement, the more index-like performance becomes. The less tolerance for underperformance, the more index-like performance becomes.
To the extent that a manager has genuine skill, you don’t want to constrain them. Who would want to constrain Warren Buffett, Kenneth Heebner, Marty Whitman, Michael Price, John Templeton, John Neff, or Ron Muhlenkamp? I wouldn’t. Give them the money, and check back in five years. (The list is illustrative, I can think of more…)
What does that mean for me, though? The first thing is that I am not for everybody. I will underperform the broad market, whether measured by the S&P 500 or the Wilshire 5000, in many periods. Over a long period of time, I believe that I will beat those benchmarks. Since they are common benchmarks, and a lot of money is run against them, that is a good place to be if one is a manager. I think I will beat those broad benchmarks for several reasons:
- Value tends to win in the long haul.
- By not limiting picks to a given size range, there is a better likelihood of finding cheap stocks.
- By not limiting picks to the US, I can find chedaper stocks that might outperform.
- By rebalancing, I pick up incremental returns.
- Industry analysis aids in finding companies that can outperform.
- Avoiding companies with accounting issues allows for fewer big losses.
- Disciplined buying and selling enhances the economic value of the portfolio, which will be realized over time.
- I think I can pick good companies as well.
I view the structural parts of my deviation versus the broad market as being factors that will help me over the long haul. In the short-term, I live with underperformance. Tactically, stock picking should help me do better in all environments.
That’s why I measure myself versus broad market benchmarks, even though I invest more like a midcap value manager. Midcap value should beat the market over time, and clients that use me should be prepared for periods of adverse deviation, en route to better returns over the long haul.
Tickers mentioned: LM
January 4th, 20083:01 am at
“and simply tell them that you are looking for good returns over a ***full cycle***. Good returns would be what the market as a whole delivers, plus a margin, over a longer period of time; that might be as much as ***5-7*** years. (Pity Bill Miller, whose 5-year track record is now behind the S&P 500. Watch the assets leave Legg Mason.)”
David,
Interesting that you mention “full cycle” and 5-7 years as a measurement period. I’m assuming that by full cycle you mean both a bull and bear market as understood by the definition of a bear market constituting a 20% drop in the broad market. What’s interesting is that if you use the past 5 years, you’ve got just a bull market, and if you use 7 years you have both a bull and a bear. This seems to be a hotly debated and controversial topic.
I can think of one manager in particular that I have a small position in, that if you look at 5 years he looks really bad relative to the S&P 500, and if you look at 7 years he looks great relative to the S&P 500, and his performance/views seem to be a popular target for many bloggers.
I’m not a fan of Miller. I think he is a good fund manager, but arguably one of the most overrated of all time. His cumulative track record is nothing compared to Buffett, Lynch, or Heebner, but he seems to get a ton of acclaim for mostly the statistical fluke of consecutive years outperforming (many by a negligible margin). He hasn’t done well during the 5-year bull market, but what is worse IMO is he did very poorly during the bear market of 00-02 during a time frame when many value managers did incredibly well. If he can’t defend capital in a bear market, or outperform in a bull market when does value get added?
My view is the market from 1990-2000 has been radically different from 2000-2007. What worked in the 90s hasn’t worked this decade, and I think many stars of the 90s haven’t adapted their process to a new reality.
January 4th, 20082:10 pm at
Re yesterday’s comments on value versus growth, one of the best paper’s I’ve read is the Tweedy Browne “What has worked in Investing” report.
http://tweedy.com/library_docs/papers/what_has_worked_all.pdf
January 5th, 20083:40 pm at
In earlier comments, I was using benchmark in a different sense than those listed. Perhaps others were as well.
For the purpose of gauging the performance of the individual investor, perhaps the most important aspect of benchmarking is the one Bill Rempel alluded to the other day: how does the return (and risk taken) compare to the return/risk she would have using an interesting alternative – e.g. a passive or fund strategy that didn’t require her to spend much or any time on it.
Another goal of comparison with the most relevant benchmark/index for a given portfolio is to help one understand the factors that affected the portfolio over shorter time frames. That’s useful, since very long term returns are dominated by individual company performance while very short term returns are dominated by market and industry wide moves. The time frame of a year is shorter than what most investors call long term.
While my use of “benchmark” is different, I’ll amplify David’s criticisms of the way the mutual fund industry is setup. The typical mandate is for a fund is to be close to 100% invested in an asset class and to be very highly diversified within that class. There’s at least two drawbacks of that setup: 1) if I am paying someone to manage money and be in tune with the market and the world economy, it makes sense to use their expertise to make some judgments about asset allocation (otherwise I have to do a big portion of the work by myself and affect the greatest portion of the returns with my asset allocation decisions) and 2) staying on top of individual company investments takes a lot of time and not all ideas are equally good, so there are inefficiencies in asking for a portfolio of stocks with no more than 5% in any one position; adopting such restrictions makes medium term returns likely to be quite similar to passive strategies (before costs reduce them). I believe this, rather than the premise of market efficiency, explains why so few active mutual funds beat their passive comparison benchmarks. This article about Bill Miller consistently lagging in his more restricted portfolio is also consistent with that idea – http://www.thestreet.com/_rms/funds/mutualfundinvesting/10358908.html – not sure how it played out this year (when seeking more liquid stocks helped performance).
January 5th, 20085:10 pm at
I don’t care a whole lot for benchmarking either. As you say in this post, loosening the constraints should give better results if the manager of the money is a professional in all areas. I don’t at all like the idea of limiting yourself in the stock market.
January 6th, 200810:39 am at
FWIW I was referring to the passive index buy as the proper *relative* benchmark for those individuals who were trying to determine if their activity was adding value to their investments. I’m not a fan of relative benchmarking except in special situations, like that one, and a couple of others. Fund managers with tight constraints, perhaps, or market-timing systems based on entering/exiting the broad market, should be judged relative to their constraints or the market they’re trading.
I think absolute benchmarks are the way to go for system designers and unconstrained portfolio managers (including do-it-yourselfers). I have not found a single metric which satisfies me for measuring the triple aspects of minimum acceptable return, maximum acceptable volatility of return, and maximizing return relative to volatility, so I set a hurdle for the first two, and then use a ratio of the two metrics to judge competing ideas.