Luck Versus Skill in Mutual Fund Returns

Fama and French, JF (2010)

Berk and Green (2004) show that zero alpha and lack of predictability in mutual fund returns is consistent with a simple model of efficient markets, rational investors, and free flow of capital.  They argue that 80% of active managers are skilled enough to achieve returns that at least cover their fees.

The most basic question in the mutual fund literature is whether active fund managers who see high returns are skilled or just lucky.  Researchers commonly test this by looking for persistence in fund returns, since fund managers should not consistently win if winning is purely chance.  However, they also commonly use recent fund history to determine persistence. If mutual fund returns are noisy, then short-term persistence will be hard to identify even if it exists.

The Question:  What is the mean and variance of the true alpha of active fund management?

Fama and French compare the long-run performance of actual funds with simulated fund returns from a world where true alpha is zero, but the characteristics of mutual fund returns are otherwise identical.  They also simulate data with alphas from a known distribution to estimate the standard deviation of true alpha.

Simulation 1: Test the hypothesis that mean alpha equals zero

  1. For each mutual fund, regress the entire sample (1984-2006) of monthly net returns on the FF three-factor model, and examine the CDF of t-statistics for alpha.
  2. Simulate data from a zero-alpha world
    1. Subtract each fund’s estimated long-sample alpha from its monthly returns to get “adjusted” returns from a hypothetical zero-alpha world.
    2. For each fund, randomly draw (with replacement) 273 months from the time series of adjusted returns. This is a time series that could have occurred if true alpha were zero and returns were random.
    3. Regress each fund’s time series on the FF 3-factor model, separate funds into three groups based on AUM, and examine the CDF of alpha t-statistics for each group.
      1. Focus on t-statistics instead of alphas, since errors are likely distributed differently in across time periods and since funds will vary in the number of monthly in which they appear in each simulation.
    4. Repeat 10,000 times, and average to get the distribution of t-statistics from the simulated data.
  3. Repeat for gross returns, then repeat again (for both gross and net returns) with the 4-factor model of Carhart (1997).
  4. Compare the CDF of t-statistics from actual data and from simulated zero-alpha data.
    1. If the distribution from actual data is higher than the simulated data (CDF below that of the simulated data), this will indicate that observed returns did not come from a zero-alpha world—i.e., they are due to skill and not luck.
  5. Also track, at each percentile of the CDF, the percentage of simulations that produce alpha t-statistics lower than the corresponding statistic from the actual data.
    1. If this number is small (large), it is evidence against (for) fund manager skill.

This exercise actually produces 12 cross-sections of simulated returns: three AUM groups, two returns (gross and net) and two asset pricing models.

Simulation 2: Estimate the standard deviation of true alpha.

  1. Adjust each fund’s returns, as before, by subtracting the fund’s long-term estimated alpha.
  2. For each fund, draw an artificial alpha from a normal distribution with mean 0 and standard deviation 0.0, and add this to the fund’s monthly returns.
  3. Draw, with replacement, a time series of 273 months, as before.
  4. Regress each fund’s simulated time series on the FF 3-factor model, separate into AUM groups, and examine the CDF of alpha t-statistics for each group.
  5. Repeat 10,000 times and average to get a simulated distribution of t-statistics for alpha.
  6. Repeat the entire process eight more times, increasing the standard deviation of the normal distribution from which the artificial alphas are drawn by 0.25 each time.
  7. Repeat everything for the 4-factor asset pricing model.
  8. Compare the CDF of t-statistics from the actual data and from each of the 18 simulations
    1. Reject artificial alpha standard deviations that lead to too few simulations producing left tails below actual data, or too many simulations producing right tails below actual data, as unlikely.

Results:

  1. Mutual fund managers are skilled enough to produce net alpha only about 1-2% of the time (the CDF of alpha t-statistics from actual data is only below the CDF of simulated data at the 98th or 99th percentiles).
  2. After controlling for momentum, 0% of the monthly returns appear to be due to skill.
  3. Managers can produce gross alpha only 10% of the time in medium and large funds, and 40% of the time for small funds (less than $250mm AUM).
  4. There is evidence of managers producing negative true alpha.
  5. The right tails of the distribution of t-statistics, in both actual and simulated data, is smaller for larger funds, which supports Berk’s and Green’s (2004) assumption about decreasing returns to scale.
    • But the left tails are just as extreme for large funds as for small funds. Fama and French interpret this as violating another Berk and Green assumption that the bad managers are all weeded out before their funds get large.
  6. The true alpha of active management can be fairly approximated by a normal distribution with mean zero and a standard deviation between 0.75 and 1.25.

Discussion:

The takeaway from this paper is that the returns of almost all funds, even the most successful ones, could just be luck.  There are a lot of funds, as probability alone dictates that at least a few must seem consistently skilled.

As usual, Fama and French do careful and serious empirical work, and communicate their arguments clearly and succinctly.  However, they place a small handicap on active managers by assuming that passive strategies incur zero costs.  Passive fees are low, but they aren’t zero.  And your costs are not zero even if you do your own investing.  You have to spend time on Yahoo! Finance, subscribe to the Wall Street Journal, or at least design an algorithm for randomly selecting stocks for your portfolio.

I also think they don’t address a natural extension of the Berk-Green argument.  Berk and Green’s story is that after a mutual fund has high returns, the market infers a skill level and gives him more money.  If the market anticipates an evolution of skill, investors could give money to managers even before they post high returns.  The market might be even more efficient than Berk and Green assumed.  The end of that theoretical story is that managers should produce zero alpha, even if all of them are skilled.  Fama and French do find some evidence that there are managers producing negative alpha, but a complete refutation of Berk and Green would have to present convincing evidence that no managers can produce alpha, even in gross returns.  Fama and French certainly weaken the statement that 80% of managers are skilled, but they aren’t able to drive that number to 0%.

A number of things may be going on.  It could be the case, as Fama and French write, that the skilled managers have moved on from mutual funds to more lucrative careers in hedge funds.  Berk and Green admit that their analysis only applies to open-ended funds, so perhaps skilled managers prefer to close their funds.  Perhaps Fama and French are right in hypothesizing that the entry of hordes of unskilled managers have drowned out the signal from skilled managers.  If active managers use leverage, perhaps they underperform in bad times.  If there were more bad times between 1984 and 2006 than there were between 1975 and 2002, then skill could seem to disappear for that reason.

Mutual Fund Flows and Performance in Rational Markets

Jonathan B. Berk and Richard C. Green, JPE (2004).

Conventional wisdom among academics, even 12 years after this paper was published, is that the average mutual fund manager is unskilled–i.e., he cannot produce a high enough return differential vis-a-vis the market to justify his fees.  This is the case, the logic goes, because markets are efficient so that the average manager does not have private information.

Still, we know that money managers are highly paid, and presumably have a high level of skill that could be (better?) utilized elsewhere.  The conclusion to this story is that the market must be inefficient, or investors irrational.  Literature prior to Berk and Green (2004) tried to reconcile this puzzle by appealing to information asymmetry or moral hazard problems in mutual funds.  Berk and Green want to give the simple model of efficient markets and rational investors another chance.

The Question: In a model with no information asymmetry or moral hazard, what does investor rationality really imply for the returns to active fund management?

The Model:

Managers have differential ability to identify investment opportunities, but they face decreasing returns to scale.  Investors can costlessly reallocate funds between managers, but manager skill is initially unknown both to investors and to managers.  When a manager earns a high return, investors make inferences about his ability in a learning model. They then give him more funds to invest until his expected return going forward is no better than the market return.  That is, if the manager is expected to return positive (negative) alpha, a rational investor will allocate (withdraw) money to (from) his fund.  When a manager has more funds than he can actively manage, he indexes the surplus.  This means that investors do not profit by allocating funds to a good manager; the manager is the one who captures the rents.  It also undermines low returns to active management as an argument that paying mutual fund managers a lot of money is socially wasteful.  In this paper, skilled managers allocate funds to their most efficient use, and they are highly compensated for doing so.

Sirri and Tufano (1998) argue that when current performance is most predictive of future returns, then fund flows are most sensitive to performance.  In contrast, Berk and Green (2004) find the opposite result: when fund flows are more sensitive to performance, current performance is least predictive of future returns in a rational/efficient market without frictions, because funds reallocate so efficiently that return predictability is wiped out.

The outline of the model is as follows:

  1. build a model of fund returns and flows based on learning about managerial ability
  2. Derive expressions for two features of mutual funds: (1) survival probability and (2) fund flows
  3. Calibrate the model, then select the two free parameters to match the data according to the two moments named above.
  4. Examine the distribution of manager skill implied by the two parameters selected in the step above.

Implications:

  1. Managers’ future performance can’t be inferred from their past returns.
  2. Investors in mutual funds can’t beat the market–not because the managers aren’t good, but because the investors compete so perfectly for managerial talent.
    1. The rents to good management are captured by the managers, not by their investors.
  3. If returns in a sector are predictable, then investment in the sector must be inefficient.
  4. 80% of managers have enough skill to earn their high pay.
  5. Successful managers have more asset under management (AUM) and lower idiosyncratic risk
    1. successful managers have large funds, and are more likely to index surplus funds.  As they do so, their returns become more strongly correlated with the market.
  6. Managers of younger funds have fewer AUM, index less (or not at all), and have higher idiosyncratic risk and so more extreme outcomes.
  7. Fund flow is more sensitive to performance when
    1. the fund is newer — investors have less information about manager skill
    2. the fund is large — absolute flows are more sensitive to returns in large funds, but flows as a percentage of fund size are independent of fund size
    3. observed returns are less noisy — investors can make better inferences about manager skill
    4. fund fees are higher — high fees make funds less attractive to investors

Conclusion:

The takeaway from this paper is that a model with efficient markets and rational investors is consistent with manager skill appearing prima facie to be both lacking and not persistent, even when managers do have skill.  In a market for a scarce resource (here, managerial skill), competitive buyers will bid up the price until the return from purchasing the resource is no better than the return to purchasing an alternative.  This does not have to mean that the resource has no value and that buyers are irrational.