Peeking under the S&P+ hood: Second-order wins

The ratings redesign process is going well, thanks for asking! I discussed it a bit last week, and as I continue to tinker, I'm coming up with numbers that look pretty good from a retrodictive perspective (the end-of-season rating seems to be getting between 80-81 percent of past results correct) and phenomenal from a predictive perspective.

There's still some tweaking to do to make sure I'm not overfitting, but I've simulated the 2013 and 2014 seasons, week by week, and right now the new measure is hitting about 54.5 percent against the spread and about 76 straight up. Going by the results here, that's about as good as you're going to find over a two-year period. And I'm tinkering with the idea of presenting the ratings in terms of point values, which would allow anybody to quickly look at teams' ratings, apply a home-field advantage where appropriate, and come up with a general "Team A would likely beat Team B by 6.6 points" estimation.

Right after I posted my Under the Hood piece last Thursday, it hit me that I was thinking way too hard about the whole "where's the balance between predictive and retrodictive?" thing, and the responses to that post certainly solidified that. It seems the best way to approach ratings is to make them as predictive as possible, period. But in looking back at past results, it is useful to still provide clues regarding which teams have been overachieving, underachieving, lucky, unlucky, et cetera.

That's where things like Pythagorean records can come in handy. It is a popular approach for many sports -- in this instance, the Wikipedia entry basically tells you everything you need to know (the short version: look at points/runs scored and allowed, apply an exponent, and produce a team's most likely win percentage) -- and it is good at telling you which teams are likely punching above their weight and are due some regression.

There are exponents available for turning college football points into a Pythagorean win percentage, but I'm more interested in another concept: second-order wins. That basically takes the same idea but uses advanced stats of some sort to determine not simply what you did score and allow, but what you should have scored and allowed.

My new ratings are based on margins in categories related to my Five Factors: efficiency, explosiveness, field position, finishing drives, turnovers/luck. As I flesh the system out with previous years of data, I'm able to basically use these margins to determine both what was your most likely scoring margin in a given game and, based on the plays that took place, your likelihood of winning a given game.

To further explain the second part of that last sentence, it basically says "If you took all the plays in this game, tossed them up in the air, and had them land in a random order, you'd win this game XX% of the time." It is a single-game win likelihood concept, and with it, we can look at wins and losses not as zeroes and ones, but as percentages. And if you're winning a lot of "You'd have won this game 60 percent of the time" games, you're probably getting a little bit lucky. And as with everything else, that luck is likely to change over time.

So who's been particularly fortunate or unfortunate in 2014? Let's take a look.

So basically, Florida State, Arizona, and BGSU, not to mention Utah, NIU, Mizzou, Rutgers, and others, have been getting by on smoke and mirrors to some degree. Meanwhile, Pitt has been figuring out ways to lose games it had no business losing.

So what does this mean? It's hard to conclude anything too drastic before I have data set up for all previous years, but here's something worth noting: your second-order win totals, or at least the difference between your wins and second-order wins, were a pretty good predictor of progression or regression last year.

• The four schools with a difference (second-order wins vs. wins) of minus-2 or lower in 2013 (Oklahoma, Nebraska, UCF, ULM) saw their win percentage decrease by about 0.130 in 2014. Nebraska improved slightly (from 9-4 to 9-3), and Oklahoma (11-2 to 8-4), UCF (12-1 to 9-3), and ULM (6-6 to 4-8) all fell by a solid margin. Oklahoma (minus-3.08), by the way, was far and away the most fortunate team in the country last year in terms of second-order wins.
• The 18 schools with a difference between minus-1 and minus-2 saw their win percentage decrease by an average of 0.102 in 2014.
• The 44 schools with a difference between zero and minus-1 saw their win percentage decrease by an average of 0.044.
• The 43 schools with a difference between zero and plus-1 saw their win percentage increase by an average of 0.075.
• The 13 schools with a difference between plus-1 and plus-2 saw their win percentage increase by an average of 0.094.
• The three schools with a difference of plus-2 or higher (Georgia State, Temple, TCU) saw their win percentage increase by an average of 0.333. All three improved, and TCU improved significantly.

However I end up laying out my new ratings and data, I figure this is a concept worth sharing from week to week. Right now, it appears these ratings are pretty damn good, and this second-order win total might be a pretty good predictor of future good or bad fortune.

More soon.