The NFL vs. college football: Which championship produces better matchups?

Right before the Super Bowl 538 wrote a fairly provacative article that argued that this year's Super Bowl was one of the best matchups ever, but that that might actually be bad for the quality of the game.

They argued that, "According to FiveThirtyEight's NFL Elo ratings, this year's Seahawks are the fifth-best team to participate in a Super Bowl since the AFC-NFC merger. And the Patriots aren't far behind. The average Elo rating of the teams this year is the second-best in a Super Bowl since that merger, trailing only Super Bowl XIII when the Dallas Cowboys played the Pittsburgh Steelers."

This immediately made me wonder how often the two "best" teams actually made it to the BCS National Championship. Did the BCS system, and now does the College Football Playoff, produce better championship matchups than the NFL playoff?

Thinking it through

First of all, there are a lot of ways to think about what makes the best matchup for a championship game. The best matchups might be between teams that performed the best over the course of the regular season (i.e., teams that go undefeated, or 2014 Florida State), or they could be between the best-performing teams according to some set of advanced (or unadvanced, I guess) statistics. Or, the best matchup could be based on the actual matchup and the storylines associated with the game (i.e., opposing Heisman Trophy-winning quarterbacks or legendary coaches).

Regardless, it's fairly rare that you get a championship game that combines all of the qualities of a "best" matchup. Rarely do you have a team that is so head-and-shoulders above all of the rest of the competition statistically and also played a perfect regular season (congrats, 2013 Florida State).

However, I chose to follow 538 and look at the advanced stats for the championship contenders over the last ten years. Instead of the 538's Elo rating, I used Football Outsiders' DVOA for NFL and F/+ for college football (and I chose ten years because that's how far back F/+ goes).

There was one obvious problem: the F/+ rankings listed are the ones that were calculated after the end of the season (so the final F/+ rankings reflect the two teams' performance in the championship game), while DVOA rankings are only regular season totals, not reflecting playoff victories. However, a) I don't have pre-Playoff/championship game F/+ data or post-Playoff DVOA data, and b) it still should be a decent comparison.

My first thought was that this might give the college system an extra advantage because they get the strength of schedule bump from the championship game (or playoff games), while the NFL championship contenders don't. But then again, a poor performance in the BCS Championship can lead to a lower final F/+ ranking despite the strength of schedule bump, so maybe it would all even out. I'm interested to see what others think about this.

The results

Here's the list of NCAA and NFL championship games from 2005-14, ranked by average team rankings (according to either DVOA or F/+):

• Four of the top five and eight of the top ten matchups are NCAA championship games, while the bottom six matchups are all Super Bowls.
• This year's College Football Playoff was between the top-ranked and third-ranked F/+ team, while the top-ranked Seahawks lost to the fourth-ranked Patriots according to regular season DVOA rankings. These matchups were the sixth- and ninth-best, respectively, in the last ten seasons.
• We'll need more data here to say for certain, but the College Football Playoff seemed to do a good job picking out the top teams, and at least according to the F/+, Ohio State was in that number even before defeating Alabama and Oregon. Going back to my Sugar Bowl offensive preview, Ohio State was ranked second overall in the F/+ rankings.

I also ran an unpaired T-test between the average rankings for NCAA and NFL championship contenders:

So NCAA teams have a statistically significant different mean ranking for their championship matchups compared to NFL teams. Not only are NCAA championships typically between teams ranked higher in Football Outsiders' efficiency rankings than teams in the Super Bowl, but there is typically less variance year-to-year as well.

The methodology isn't perfect here. DVOA and F/+ don't work the exact same way, and the rankings were taken at different points of the season. However, before looking at the numbers, I mistakenly expected the Super Bowl to feature better teams than the much-criticized BCS championship.

I'm guessing that the NFL playoff levels the field a little by a) having divisional representatives, b) adding more teams to the post-season, and c) any of those playoff teams could theoretically buck season-long efficiency trends and put it together for a playoff run.

For college football, it seems like expanding to an 8-team playoff and having automatic playoff bids for conference champions would certainly, over time, lower the average advanced statistical rankings for the College Football Playoff Championship contenders -- just like in the NFL.