Luck Versus Skill in Mutual Fund Returns

Fama and French, JF (2010)

Berk and Green (2004) show that zero alpha and lack of predictability in mutual fund returns is consistent with a simple model of efficient markets, rational investors, and free flow of capital.  They argue that 80% of active managers are skilled enough to achieve returns that at least cover their fees.

The most basic question in the mutual fund literature is whether active fund managers who see high returns are skilled or just lucky.  Researchers commonly test this by looking for persistence in fund returns, since fund managers should not consistently win if winning is purely chance.  However, they also commonly use recent fund history to determine persistence. If mutual fund returns are noisy, then short-term persistence will be hard to identify even if it exists.

The Question:  What is the mean and variance of the true alpha of active fund management?

Fama and French compare the long-run performance of actual funds with simulated fund returns from a world where true alpha is zero, but the characteristics of mutual fund returns are otherwise identical.  They also simulate data with alphas from a known distribution to estimate the standard deviation of true alpha.

Simulation 1: Test the hypothesis that mean alpha equals zero

  1. For each mutual fund, regress the entire sample (1984-2006) of monthly net returns on the FF three-factor model, and examine the CDF of t-statistics for alpha.
  2. Simulate data from a zero-alpha world
    1. Subtract each fund’s estimated long-sample alpha from its monthly returns to get “adjusted” returns from a hypothetical zero-alpha world.
    2. For each fund, randomly draw (with replacement) 273 months from the time series of adjusted returns. This is a time series that could have occurred if true alpha were zero and returns were random.
    3. Regress each fund’s time series on the FF 3-factor model, separate funds into three groups based on AUM, and examine the CDF of alpha t-statistics for each group.
      1. Focus on t-statistics instead of alphas, since errors are likely distributed differently in across time periods and since funds will vary in the number of monthly in which they appear in each simulation.
    4. Repeat 10,000 times, and average to get the distribution of t-statistics from the simulated data.
  3. Repeat for gross returns, then repeat again (for both gross and net returns) with the 4-factor model of Carhart (1997).
  4. Compare the CDF of t-statistics from actual data and from simulated zero-alpha data.
    1. If the distribution from actual data is higher than the simulated data (CDF below that of the simulated data), this will indicate that observed returns did not come from a zero-alpha world—i.e., they are due to skill and not luck.
  5. Also track, at each percentile of the CDF, the percentage of simulations that produce alpha t-statistics lower than the corresponding statistic from the actual data.
    1. If this number is small (large), it is evidence against (for) fund manager skill.

This exercise actually produces 12 cross-sections of simulated returns: three AUM groups, two returns (gross and net) and two asset pricing models.

Simulation 2: Estimate the standard deviation of true alpha.

  1. Adjust each fund’s returns, as before, by subtracting the fund’s long-term estimated alpha.
  2. For each fund, draw an artificial alpha from a normal distribution with mean 0 and standard deviation 0.0, and add this to the fund’s monthly returns.
  3. Draw, with replacement, a time series of 273 months, as before.
  4. Regress each fund’s simulated time series on the FF 3-factor model, separate into AUM groups, and examine the CDF of alpha t-statistics for each group.
  5. Repeat 10,000 times and average to get a simulated distribution of t-statistics for alpha.
  6. Repeat the entire process eight more times, increasing the standard deviation of the normal distribution from which the artificial alphas are drawn by 0.25 each time.
  7. Repeat everything for the 4-factor asset pricing model.
  8. Compare the CDF of t-statistics from the actual data and from each of the 18 simulations
    1. Reject artificial alpha standard deviations that lead to too few simulations producing left tails below actual data, or too many simulations producing right tails below actual data, as unlikely.

Results:

  1. Mutual fund managers are skilled enough to produce net alpha only about 1-2% of the time (the CDF of alpha t-statistics from actual data is only below the CDF of simulated data at the 98th or 99th percentiles).
  2. After controlling for momentum, 0% of the monthly returns appear to be due to skill.
  3. Managers can produce gross alpha only 10% of the time in medium and large funds, and 40% of the time for small funds (less than $250mm AUM).
  4. There is evidence of managers producing negative true alpha.
  5. The right tails of the distribution of t-statistics, in both actual and simulated data, is smaller for larger funds, which supports Berk’s and Green’s (2004) assumption about decreasing returns to scale.
    • But the left tails are just as extreme for large funds as for small funds. Fama and French interpret this as violating another Berk and Green assumption that the bad managers are all weeded out before their funds get large.
  6. The true alpha of active management can be fairly approximated by a normal distribution with mean zero and a standard deviation between 0.75 and 1.25.

Discussion:

The takeaway from this paper is that the returns of almost all funds, even the most successful ones, could just be luck.  There are a lot of funds, as probability alone dictates that at least a few must seem consistently skilled.

As usual, Fama and French do careful and serious empirical work, and communicate their arguments clearly and succinctly.  However, they place a small handicap on active managers by assuming that passive strategies incur zero costs.  Passive fees are low, but they aren’t zero.  And your costs are not zero even if you do your own investing.  You have to spend time on Yahoo! Finance, subscribe to the Wall Street Journal, or at least design an algorithm for randomly selecting stocks for your portfolio.

I also think they don’t address a natural extension of the Berk-Green argument.  Berk and Green’s story is that after a mutual fund has high returns, the market infers a skill level and gives him more money.  If the market anticipates an evolution of skill, investors could give money to managers even before they post high returns.  The market might be even more efficient than Berk and Green assumed.  The end of that theoretical story is that managers should produce zero alpha, even if all of them are skilled.  Fama and French do find some evidence that there are managers producing negative alpha, but a complete refutation of Berk and Green would have to present convincing evidence that no managers can produce alpha, even in gross returns.  Fama and French certainly weaken the statement that 80% of managers are skilled, but they aren’t able to drive that number to 0%.

A number of things may be going on.  It could be the case, as Fama and French write, that the skilled managers have moved on from mutual funds to more lucrative careers in hedge funds.  Berk and Green admit that their analysis only applies to open-ended funds, so perhaps skilled managers prefer to close their funds.  Perhaps Fama and French are right in hypothesizing that the entry of hordes of unskilled managers have drowned out the signal from skilled managers.  If active managers use leverage, perhaps they underperform in bad times.  If there were more bad times between 1984 and 2006 than there were between 1975 and 2002, then skill could seem to disappear for that reason.