Manager Selection And Retention: Put The Odds In Your Favour
By: Bruce Curwood
In order to evaluate and select managers, proper strategies are needed. Bruce Curwood, of the Russell Investment Group, discusses probability, tracking error, and asset class strategies that may help in manager selection.
The investment industry generally recognizes managing money to be a long-term zerosum game. For every winning manager there is a corresponding loser. The conundrum is that pension and endowment investing is a long-term game, but choosing or retaining managers based upon unreliable shortterm performance data is a common industry practice and it is fraught with error. Manager skill cannot be evaluated statistically with five-year performance data, no matter how precise the tools or how frequent the measurements.
What is needed are long-term, consistent, value-added strategies that prescribe a better way to evaluate and select managers. So what should a prudent fiduciary do? Risk management provides that a structured approach to decision analysis makes the most sense. In dealing with uncertainty, it’s all about minimizing the probability of loss and maximizing the chance of successful decisions.
Probability Analysis: Discerning Luck From Skill
Many statistical studies have been conducted on the subject of discerning luck from skill in investment management. Probability analysis has recently yielded solutions, which computers have made easy. We recently undertook a study to determine the minimum time required to statistically determine if a manager has added value through skill. In our model, time is simply a function of the confidence interval selected (most statisticians use 95 per cent or 99 per cent to prove their point with higher confidence), the projected alpha (manager value added), and the estimated risk, or tracking error (TE). In the analysis, we recognize that each asset class performs differently, with varying opportunities for a manager to take risk in order to add value. I would not expect my Canadian bond manager to continually add more value than my Canadian equity manager, even by taking on more risk.
By entering these various factors in the computer model, the minimum time requirement for proper and adequate evaluation can be solved for each asset class. We constructed Table 1, using a 95 per cent confidence interval, along with reasonable value added and tracking error estimates for each asset class.
The results will probably be a surprise. As you can see, the timeframes for statistical significance are ludicrous: a minimum time of 16 years or 64 quarters of performance data. Certainly, these timeframes can be reduced somewhat by tightening the confidence interval, but that also reduces the chances of selecting appropriately skilled managers. The conclusion is that manager’s performance evaluation is statistically unreliable in the normal investment committee evaluation period, which is generally less than five years.
But can we learn anything from probability analysis? Zurich Scudder performed some innovative research which was quite profound. By turning the above four-factor analysis around and by solving the same equation for probability, instead of time, they determined that lower tracking-error strategies had a higher probability of success in a normal five-year time period, as shown in Table 2.
They discovered that over a given time horizon, with constant manager alpha (set at one per cent in Table 2), and assuming that active returns are normally distributed, lower tracking error created a higher probability of success. In other words, they helped to solve our manager-selection conundrum. Given the normal investment committee evaluation period of five years, the use of lower-tracking- error (risk) strategies created a lower probability of underperformance (or a higher probability of outperformance) than higher-tracking-error (risk) strategies.
Looking at Table 2, we see that tracking error of two per cent (low) resulted in a probability of underperformance of only 14 per cent and increased the chances of success to 86 per cent. On the other hand, tracking error of eight per cent (high) increased the odds of underperforming to 43 per cent and reduced the chances of success to 57 per cent.
Smallest Tracking Error
These results shouldn’t come as much of a surprise. We are observing that high-information- ratio managers – that is, the managers with the highest alpha for every one per cent of tracking error – achieve more consistent results. With constant alpha, the manager with the highest information ratio will have the smallest tracking error and, therefore, will offer the most consistent returns. In other words, returns will be more tightly distributed around their alpha and these managers will experience fewer periods with returns less than zero. Note, however, that under the assumption that managers’ active returns are normally distributed, managers with the same information ratio will have the same probability that their returns will be less than zero. Furthermore, the higher the information ratio, the smaller the probability that the managers will experience returns below zero. Therefore, we should be trying to identify the highest information ratio managers.
The problem here is that alphas and information ratios are difficult to estimate and are probably unstable. Most managers maintain a relatively stable level of tracking error, and a variety of tools can help us measure it. Therefore, one way to help put the odds in our favour is to use lower tracking error strategies. For example, Table 2 shows that a lower risk strategy (two per cent tracking error) with a 10-year time horizon will provide us with 94 per cent confidence of being correct. Not only that, but picking a lowerrisk strategy with smaller portfolio bets probably provides more downside protection if the strategy underperforms (the six per cent of occurrences when we were unlucky with a good strategy that failed to add value).
But 10 years, or a current business cycle, is a long time to wait to ensure our manager can add value and we are limiting our value added by selecting individual managers with low tracking error, mostly enhanced indexers. Why not just go passive and invest in the benchmark, avoiding any manager risk? The simple reason is that each one per cent of added value at the total fund level derived through active management can increase benefits or decrease funding costs (that is, contributions) by as much as 20 per cent over the long term. This is one of the wonders of compound interest and one of the reasons that positive, active management is desirable.
That said, an investment committee may not want to limit its opportunity set to low value-added strategies. Instead, a committee can pick specialist managers who can add high value in their respective areas of expertise and optimally combine managers to reduce risk relative to the benchmark. This limits exposure to a single manager and also allows the committee to choose from strategies with potentially higher value added. Risk reduction comes from the effects of correlation and modern portfolio theory, one of the few free lunches in investments.
In short, the whole (the asset class structure) may be more important than the sum of the parts (the individual managers). Without multiple managers, we are either limiting our outcome by using a low risk/low value-added manager with a better chance of success in a five-year period, or increasing our risk by using a high-risk manager and chancing significant underperformance. In either case, the fund has high exposure to an individual manager. A well-designed multi-manager structure will improve our odds of success through low asset class risk, while still enabling us to choose high value added managers.
Constructing A Good Asset Class Strategy
Constructing a good asset class strategy generally requires a game plan. The best asset class game plan for your fund will depend upon a number of factors specific to your fund (AUM, risk tolerance, etc) and investment committee.
Each asset class strategy should have two achievable – but often competing – objectives, usually expressed as return (that is, value added or alpha) and risk (commonly tracking error or TE). These factors differ significantly by asset class. For example, risk and return objectives for Canadian bonds (0.4 per cent alpha and one per cent TE) would be significantly different than that for Canadian equities (1.5 per cent alpha and four per cent TE). Yet, they should be aligned with your risk tolerance for each asset class and at the overall fund level. In seeking a higher value-added return, fiduciaries usually must be prepared to accept higher risk. That higher risk ultimately must be congruent with the fiduciaries’ risk tolerance and time horizons or they quickly will unwind the strategy at the first sign of significant underperformance.
The professed long-term asset class investment strategy must be well-articulated, communicated, and documented. In a specialist structure, each manager should play a separate and distinct role with differing mandates and guidelines. The combined risk of the individual managers should not exceed the risk objective for the asset class. The value added comes from picking the managers, but the risk reduction comes from combining the managers. In this structure, with multiple competing and complementary managers, the evaluation focus should shift from the individual manager to the performance of the asset class.
Practical Solutions To Put The Odds In Your Favour
No single action will eliminate the uncertainties of manager selection, but the following practical steps may reduce risk and put the odds of adding value in your favour:
- Prepare and communicate a long-term asset class strategy that is founded in research and consistent with your risk tolerance and objectives. Fundamental research generally concludes that the two factors that result most consistently in good manager selection are highquality internal research and good security selection (not market timing).
- Implement using a lower-risk (low TE) asset-class strategy to improve the odds of outperformance in the conventional five-year evaluation period and to limit the downside in case of any underperformance.
- Choose a structure with multiple specialist managers to diversify manager selection risk, reduce the exposure to one single manager, increase potential value added, and minimize asset class turnover costs.
- Communicate and document the asset class structure. This will help the committee focus on the performance of the asset class strategy.
- Use a combination of qualitative and quantitative research to analyse manager selection and retention decisions. I recommend a three-tiered approach to performance measurement: superior qualitative research; portfolio profile analysis at the manager and asset class level; and core performance measurement to ratify the outcomes3. Try to remove end point sensitivity by looking at all manager returns during the entire product history for rolling one-, three-, and five-year periods to determine consistency, average value added, and changes in approach.
Perform ongoing due diligence of the individual managers and their asset class holdings. The information gathered will help you control the asset class risk and make necessary manager replacements or reallocations.
- Continuously monitor the asset class structure and its overall risk relative to the benchmark to refine the process.
If an investment committee truly is seeking consistent performance over a reasonable time period, then focusing on a good multi-manager asset class strategy with lower tracking error is more likely to provide consistent value added. The vagaries and randomness of the market can never be removed, but a good asset class strategy and proper investment committee education and communication put the odds in your favour.
Bruce Curwood is director, institutional solutions, for Russell Investment Group.
1. Curwood, Bruce, “Measuring Performance: A New Direction”, Canadian Investment Review, Spring 2000.
2. Fortuna, Phil, “Tracking Errors: A Skeptical Examination of a Significantly Misunderstood Tool”, Zurich Scudder Investments, September 2001.
3. For further insight to this approach, see Curwood, Bruce, op. cit.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -