Which college football coaches consistently produce overachievers or underachievers?

In each FSH college football stat profile, you'll see something called "win expectancy" in each team's "Schedule & Results" section. The definition from my last advanced stats glossary:

It is intended to say "Given your success rates, big plays, field position components, turnovers, etc., you could have expected to win this game X% of the time." It has nothing to do with pre-game projections or opponent adjustments.

For preview purposes, I note when teams strayed pretty far from their win expectations, one way or the other, and in 2016 two teams strayed really far from expectations: Idaho overachieved by 2.3 wins, and Notre Dame underachieved by 3.2 wins, the fourth-highest (lowest?) in 12 years.

From year to year, this is a sign of randomness. The teams on the extreme ends are all but guaranteed to regress (or progress) toward the mean the next year. Notre Dame was the fourth team to underachieve its second-order win total by at least three games; the other Power 5 team on that list: 2013 TCU, which improved from 4-8 to 12-1 the next season.

Win Expectancy mostly addresses the randomness in results. But over time, it can tell us a little bit about certain coaches.

As I noted a couple of years ago, the coaches at the top and bottom of this list tend to be the ones fans say are particularly good or bad coaches.

If a random college football fan were asked to create a list of what we'll simply call "bad coaches," he or she would probably fairly or unfairly mention Ty Willingham, John L. Smith, Dave Wannstedt, Ted Roof, Houston Nutt. Washington State fans might mention Bill Doba. North Texas fans would definitely list Todd Dodge. They're all on one particular end of the chart.

Meanwhile, asking someone to list "good coaches" might produce a list with men like Bill Snyder, Gus Malzahn, etc. They can be found anchoring the other end. Again, I'm not sure this actually means anything, but it certainly might help to confirm what some might already think about given coaches.

You can potentially distill coaching into two things: building a team that produces great stats and figuring out how to maneuver in tight games when neither team has a statistical advantage. The former matters above all (Nick Saban and Steve Kragthorpe, after all, are nearly even on this list ... and on no other list in existence), but if nothing else, this list might help us to understand which teams/coaches are good or bad at the latter.

I talked about this a bit in today’s 2017 Utah preview as well.

The idea is simple: if you won more games than Win Expectancy thought you would one year, you’re probably going to regress toward the mean the next. Same goes for if you won too few games.

That doesn’t work for every coach, though. Our eyeballs see some coaches have teams that don’t execute as well in key moments. Some always convert key third-and-5s, call timeouts at the right time, make just the right special teams play, etc. Others don’t.

The annual underachievers and overachievers list is based on comparing coaches’ average second-order win totals to their actual win totals. Navy’s Ken Niumatalolo and Kansas State’s Bill Snyder inevitably lead the list each year. Since 2005 (when my play-by-play sample begins), both have coached nine seasons, and both have averaged more than one win per year over what the stats would have expected. They are so consistent that there’s something to it.

With another year of data in the books, I thought I would share the list of coaches once more. Below is a look at everyone who has been a head coach at least three years since 2005 (I previously used four years as the cutoff) and what their average difference is per year — actual wins vs. win expectation.

Some quick impressions:

• Welcome to the party, Dino Babers. His high average comes with a qualifier, though. By lowering the bar from four years to three, I have opened the door for one season to have an even larger impact on the averages. Babers’ first season at Bowling Green produced 2.7 more wins than expected; his other two years (2015 BGSU, 2016 Syracuse) produced a combined plus-0.3.
• Your top five among coaches who have gone at least 10 years in this sample: Pat Fitzgerald, Urban Meyer, Paul Johnson, David Bailiff, and Gary Pinkel.
• Your bottom five among 10-year guys: Doug Martin, Kirk Ferentz, Al Golden, June Jones, Larry Blakeney.
• If I had lowered the bar to two years, Tom Herman would have ranked number one. His 2016 Houston team overachieved by only 0.2 wins, but his 2015 team overachieved by 2.8.
• Niumatalolo ... Johnson ... the option is still one hell of a way to manage a game. Somebody better be hiring Navy offensive coordinator Ivin Jasper to a head coaching gig soon.
• As mentioned in the Utah preview, Kyle Whittingham is gaining ground rapidly. Utah Has overachieved by 1.8, 1.8, and 1.5 wins, respectively, over the last three years. The 2016 run was aided by turnovers luck, but three is a trend!

I’ve written this before, but here’s your reminder that this is only one aspect of good coaching. Creating a good team and winning easy games is a larger aspect. Alabama’s Nick Saban is the greatest coach of his generation but is only plus-0.2 wins per year here. Meanwhile, Kevin Wilson is an extreme underachiever on this list (minus-1.08 wins per year), but he raised Indiana’s potential enough to still take the Hoosiers to back-to-back bowls.

Still, I love this list because of how it seems to hone in on what we consider good coaching. The common college football fan or writer would quickly tell you how good a coach Bill Snyder is, for instance, and this illustrates how. He doesn’t sign top-10 classes, and he hasn’t contended for a national title in a while, but Snyder, Niumatalolo, Fitzgerald, and others have consistently milked the most possible gain out of their win opportunities.