The arguments over investing for growth vs. value can sometime remind advisers of a water cooler quarrel. Proponents of each approach come armed with their own compelling data and analysis. But like many spats, a simple, surprising piece of evidence can quickly derail the argument. In this case, it’s the fact that the performance of both growth-oriented U.S. equity indexes and value indexes are strikingly different depending on which index provider you choose to use.

As a case in point, we can document a significant value premium over the past 19 years if using Morningstar U.S. equity indexes, but then see it disappear when using S&P U.S. equity indexes.

As you will see, this discrepancy among index providers can be central to the debate. Interestingly, it also emerges as a separate issue that illustrates the challenges in attempting to measure any aspect of our investment markets.

First, let’s consider the difference in the 19-year average annualized return in 1998-2016 for three large-cap growth indexes as shown in the table “The Growth Versus Value Conundrum.” The 19-year return of the Morningstar U.S. Large Growth Index was 3.88% whereas the Russell 1000 Growth Index had a return of 5.76%. The S&P 500 Growth Index did even better, with a 19-year average annualized return of 6.37%. (The reason for using the 19-year period is that several of the growth and value indexes in this analysis began in 1998).

The difference between a return of 3.88% and one of 6.37% is a whopping 249 basis points. Let’s put that into perspective in terms of money growth over 19 years. A $10,000 deposit with a 3.88% 19-year return would grow to $20,611. By comparison, if the money grows at 6.37%, the ending balance will be $32,326 — a difference in appreciation of $11,715.

Thus, if you are evaluating U.S. large-cap growth stocks’ performance over the past 19 years and you are using one of the three major index providers, your opinion might be very different based on whether Morningstar indexes or S&P indexes were used.

We can document a significant premium for value stocks over growth stocks in the past 19 years if using Morningstar indexes, but then see it disappear if we use S&P indexes.

We see less of a performance discrepancy among the three large-cap value indexes over the past 19 years. The 19-year return of the Morningstar U.S. Large Value Index was 6.11%, the Russell 1000 Value was 7.07%, and the S&P 500 Value Index had a return of 6.3%.

But, more to the point of this article, there was a distinct value “premium” (higher level of performance in the value-oriented index) in the large-cap U.S. equity indexes from Morningstar — 233 basis points to be exact. By comparison, the Russell 1000 Value Index outperformed the Russell 1000 Growth Index by 131 bps. Interestingly, when comparing the S&P 500 Growth Index and the S&P 500 Value Index, we observe no value premium over the past 19-year period. In fact, there was a 7 bps growth premium.

Also reported in the table is the “category average” performance between growth mutual funds and value-oriented mutual funds. In the large-cap group, there is a very small value premium.

Admittedly, these performance figures are messy in that mutual funds (as opposed to indexes) can shift their allegiance between growth and value over time and/or fluctuate between large-cap and mid-cap holdings and thus end up in different categories from year to year. In addition, there is survivorship bias, meaning there is a tendency for poorly performing funds to go out of business.

Despite such potential issues in the category averages, we observe a slight average value premium of 14 basis points among actual investable U.S. equity large-cap mutual funds.

(On a technical note, the category averages in the Steele Systems database utilize only the oldest share class of mutual funds that have multiple share classes. By doing so, the performance of multiple share-class funds is not overrepresented in the category average performance figures.)

We observe a similar outcome among mid-cap growth and value indexes. There is a pronounced value premium of 309 bps between the Morningstar U.S. mid-cap indexes, a smaller value premium of 193 bps between the Russell mid-cap indexes and no value premium for the S&P mid-cap indexes. Clearly, the logic and design of growth versus value indexes is subject to significantly different interpretations by the index makers.

Not only that, the difference among mid-cap growth indexes is also dramatic. We observe a difference of 375 bps between the Morningstar U.S. Midcap Growth Index and the S&P Midcap 400 Index over the past 19 years. This almost makes you wonder if they are measuring the same thing.

Using the category average, we see a 60 bps value premium among mid-cap mutual funds over the past 19 years. Certainly, in the years from 1998 through 2016 there were periods when growth outperformed, such as during the last gasps of the tech bubble in 1998 and 1999. But, over the entire 19-year period, U.S. equity mid-cap mutual funds with a value orientation produced a slightly higher return on average than U.S. equity mid-cap growth mutual funds.

Among small-cap indexes the same trend continues. The performance of the S&P Small Cap 600 Growth Index was significantly higher than the small cap growth indexes at Morningstar and Russell. The performance differences among the three value indexes were, by comparison, much smaller.

As for the value-versus-growth differences within a family, there was a pronounced value premium of 448 bps between the Morningstar U.S. Small Cap Value Index and the Morningstar U.S. Small Cap Growth Index. There is also a significant value premium of 269 bps between the Russell 2000 Value Index and the Russell 2000 Value Index. And, for the first time, a faint value premium of 16 bps shows up between the S&P Small Cap 600 Value Index and the S&P Small Cap 600 Growth Index.

When choosing a small-cap U.S. equity fund, look for one that has a value tilt.

As measured by the category average, we observe a 79 bps value premium among small-cap mutual funds over the past 19 years. In the small cap space, a value premium seems indisputable over the long haul.

The performance differences among the three index makers — Morningstar, Russell, S&P — are accentuated when comparing their growth indexes. The S&P growth indexes had the highest 19-year return in all three sectors: large-cap, mid-cap and small-caps.

But performance is more similar when comparing values indexes among the three index creators, and the top results are spread through all three. Over the past 19 years, Russell’s value index had the best performance among large caps, S&P had the best performance among mid caps, and Morningstar had the best performance among the small-cap value indexes. In terms of the performance of its indexes, S&P owns the growth space, but not the value space.

There is an interesting subplot in this analysis, which has to do with debate about whether active management funds or passive management funds are superior.

Advocates of index-based investing — passive management — often refer to active-versus-passive research that is conducted under the aegis of SPIVA (S&P Indices Versus Active). SPIVA research examines how actively managed funds perform against their benchmarks, and those benchmarks are S&P indexes.

Note that no one can actually invest in an index and that indexes do not reflect the impact of an operational expense ratio. Thus, the first problem is that indexes do not have expense ratios, whereas actual investable funds do — whether they are actively or passively managed. I would argue that the benchmark of any fund should be investable, not an index that no one can actually invest in.

The second problem is that S&P indexes tend to have higher returns than Morningstar or Russell indexes — particularly among growth indexes. This essentially creates a higher bar for actively managed funds to measure up to. In other words, the world can look different when measured by someone other than S&P.

My intent is not to take be critical of S&P. Rather, it is to simply observe that the U.S. equity market is measured in a variety of ways and by a variety of firms. Those differences need to be acknowledged before we accept any one index or index provider as being the ultimate measurement of any market.

In short, the percentage of actively managed funds that outperformed the Morningstar U.S. Large Cap Growth Index over the past 19 years was considerably higher than the percentage of actively managed funds that beat the S&P 500 Growth Index.

Thus, your assessment of the ability of actively managed funds to beat their benchmark will depend entirely on which benchmark you select. The simple message is that S&P is not the only maker of benchmarks. And, as we have seen, there can be considerable performance differences among the various makers of indexes.

In summary, if the active-versus-passive debate is one that you no longer fuss with, you can simply focus on the clear value premium that exists in the small-cap U.S. equity space. When choosing a small-cap U.S. equity fund, look for one that has a value tilt.

So the next time a growth vs value brawl breaks out, it’s important to clarify which “benchmark” indexes are being used in the performance comparisons. As we have seen, it matters a lot.

And one other thing…you’ll find that many small-cap value funds have already closed. The small-cap value premium is not a new finding.

Craig L. Israelsen

Craig L. Israelsen

Craig L. Israelsen Ph.D., a Financial Planning contributing writer in Springville, Utah, is an executive in residence in the personal financial planning program at the Woodbury School of Business at Utah Valley University. He is also the developer of the 7Twelve portfolio.