clock menu more-arrow no yes mobile

Filed under:

A better way to measure returning experience in college football

New, 6 comments

File this one under "more interesting than useful" ... for now.

We use the best tools we have, even if they're not that great. We try to derive value from offensive line starts because it's the only individual measure of offensive linemen that we have. You can't get much from "he's been one of Team A's five preferred linemen 16 times since he started school here," but the value is probably greater than zero, so we go with it.

It's the same with returning starters. We use it because it exists, and nothing else is particularly available. It is fine as a really quick snapshot, but we know that one team's six returning offensive starters aren't another's. What about go-to guys? Returning backups? And quarterbacks are worth more than other starters, right?

You can see some people drawing reference to percent of returning yards/tackles/etc., but it's hard to find concrete data in that regard.

Of course ... I do compile all of this data, don't I? One of the goals I've had for a while now is to pull together the data I use for my team preview series. Every team gets its own tab in a giant spreadsheet, and I manually update who is or isn't returning. I collect career start information for the offensive line, too (again, because it's all we've got). These sheets also have everybody's primary stats from last season, so ... why don't I weaponize this data for use in projections and whatnot?

I had found the impetus a little too late to do anything about it in this year's projections, but as the season begins, I wanted to show you some of the data I've got so far.

So here was the intended process.

1. Compile all teams' 'percent returning' data from last year's previews. This means percentage of passing yards/attempts/completions from quarterbacks, rushes/yards from running backs, targets/receptions/yards from receivers, career starts from offensive linemen, tackles/TFLs/sacks/passes defensed from defensive linemen, linebackers, and defensive backs*.

* For now, if someone was returning from injury, or if a transfer was eligible, I just mashed their previous year of experience into the tables. For instance, Jameis Winston is gone from Florida State this year, but Everett Golson has transferred in from Notre Dame. Therefore, FSU "returns" 48% or so of passing yardage instead of nearly 0%. It's awkward, but we're still in the preliminary stages here.

2. Compare all of these new categories to how teams performed in 2014. Or, more precisely, how their performance changed from 2013 to 2014. That's what we're looking for with returnee/experience data, right? Returning 80% of your receiving yards or tackles for loss won't mean you're guaranteed to be good, but it should mean you improve, right?

3. Run a bunch of correlations. Figure out which categories seem to mean more than others.

4. Tinker with a potential overall "percent of offense/defense returning" figure. Give different weight to certain categories that seem to carry stronger correlations to actual improvement or regression.

5. Compile all teams' data for 2015, too. See what 2014 change data might mean for 2015.

6. Eventually go bigger. I have these spreadsheets for 2013 and, in some regard, 2012, too. By the time next offseason rolls around, I might be comfortable with replacing returning starters in my preview series with a deeper, richer Percent of Offense/Defense Returning number. (And it only feels like that 2016 preview series should be starting next week...)

So it took all the way until the season began, but with intern Chris Brown's help, we got through step 5. Here's some of what I found:

Is offensive line experience overrated?

So here are the correlations between different types of percentage returning and S&P+ change:

Category Correlation to change in Off. S&P+
QB (% of completions returning) 0.286
QB (% of passes returning) 0.280
QB (% of passing yards returning) 0.288
RB (% of carries returning) 0.009
RB (% of rushing yards returning) 0.012
WR/TE (% of targets returning) 0.247
WR/TE (% of catches returning) 0.254
WR/TE (% of receiving yards returning) 0.251
OL (% of career starts returning) -0.107

If you had asked me before hand, I would have ventured that the strongest correlations would be tied to the quarterback and offensive line. Instead, quarterbacks and receivers had far stronger correlations than RBs or OL, and the correlation between line experience and offensive improvement is actually negative.

Now, again, this is only one year of data, so I'm not going to jump to massive conclusions just yet. I assume with more data, the OL correlation will at least flip to positive (and tiny), so I'm not going to start saying things like "Team A returns 132 career starts up front, which is a giant red flag" or anything. But wow, that was not what I expected.

Generally, the correlation between returning starters and offensive change is around 0.2 to 0.3. By tinkering with different weights for things like % of passing yards, % of rushing yards, % of targets/receptions, and, yes, % of starts returning, I was able to come up with a Percent of Offense Returning number that correlates at about 0.35. Improvement! Again, those weights will change when I have more data, and we'll see what that does, but at the very least, that tells me that this type of number, in a future form, could help to make these projections more accurate.

(Also: in the future, I'll be attempting to tie these numbers to things like run-pass ratios -- obviously it probably doesn't matter as much when Army returns all of its passing yards as when Washington State does.)

You need a seasoned secondary

Here are the correlations between different types of defensive data and change in Def. S&P+. Since I'm now using an adjusted point total for offensive and defensive ratings, one would expect a negative correlation -- % of returning ___ goes up, adjusted scoring averages go down.

Category Correlation to change in Off. S&P+
DL (% of tackles returning) -0.135
DL (% of TFLs returning) -0.189
DL (% of sacks returning) -0.217
DL (% of passes defensed returning) -0.183
LB (% of tackles returning) -0.148
LB (% of TFLs returning) -0.171
LB (% of sacks returning) -0.163
LB (% of passes defensed returning) -0.160
DB (% of tackles returning) -0.327
DB (% of TFLs returning) -0.364
DB (% of sacks returning) -0.211
DB (% of passes defensed returning) -0.370
ALL DEFENSE (% of tackles returning) -0.347
ALL DEFENSE (% of TFLs returning) -0.348
ALL DEFENSE (% of sacks returning) -0.295
ALL DEFENSE (% of passes defensed returning) -0.424

Again, if I'd made wagers beforehand, I'd have bet on linebacker correlations being low. Everything I've read from others, and everything I've noticed myself, suggests that linebackers are a little bit less hard to replace than others. Or, the range between great linebackers and just fine linebackers is smaller than in other units.

This suggests that experience in the front seven isn't as big a deal as it is in the back of the defense. It is pretty remarkable that the correlations between returning DBs are almost as strong as those for the defense as a whole. I didn't see that coming.

And, for 2014, at least, the data suggested that the ability to get hands on passes was more valuable -- or at least, less replaceable -- then getting hands on the quarterback. I didn't see that coming either.

(Again, this is only one year of data, this is only one year of data, this is only one year of data. But still.)

One more interesting thing: by tinkering with the weights here, I was able to come up with a Percent of Defense Returning figure that correlated at -0.46 to defensive change. That's pretty damn high. And it suggests that experience matters quite a bit more on defense than on offense.

A 2015 run-through

I obviously don't feel comfortable enough with this single year of data to go and make massive changes to my projections just yet. But what if I did? What might these numbers tell us about 2015?

Actually, let's back up a year.

Largest Combined Percentage of Production Returning in 2014
Team % of Offense Ret. % of Defense Ret. Average (Off., Def.)
Memphis 87% 87% 87%
Northwestern 83% 86% 85%
Maryland 88% 79% 84%
UL-Lafayette 75% 90% 82%
Kentucky 71% 92% 81%
Central Michigan 89% 72% 80%
Houston 76% 83% 80%
New Mexico 81% 78% 80%
Virginia 70% 88% 79%
Mississippi State 78% 80% 79%
Air Force 73% 84% 78%
Nevada 70% 83% 77%
Auburn 91% 59% 75%
California 84% 67% 75%
Marshall 71% 79% 75%
Old Dominion 61% 89% 75%
Syracuse 73% 76% 75%
UTSA 68% 80% 74%
West Virginia 79% 68% 74%
UAB 57% 89% 73%
Ole Miss 60% 86% 73%

There are 21 teams listed there; 12 went bowling, and 17 improved. Most of last year's most significant breakthroughs -- Memphis, Mississippi State, Air Force, Marshall, UAB, Ole Miss -- are accounted for here.

Meanwhile, here are the teams that ended up below 50%.

Smallest Combined Percentage of Production Returning in 2014
Team % of Offense Ret. % of Defense Ret. Average (Off., Def.)
Vanderbilt 32% 33% 33%
Oklahoma State 40% 30% 35%
New Mexico State 50% 28% 39%
Wake Forest 30% 53% 42%
Arizona State 65% 21% 43%
North Texas 36% 51% 43%
Missouri 34% 54% 44%
LSU 25% 69% 47%
Utah State 56% 40% 48%
Nebraska 48% 48% 48%
Georgia State 58% 38% 48%
SMU 40% 57% 48%
Ohio 26% 71% 48%
Boston College 36% 62% 49%
Troy 38% 60% 49%

Most of last year's most epic collapses -- Vandy, Wake Forest (on offense), North Texas, SMU, Troy, Oklahoma State -- are accounted for here. Boston College was able to redefine itself pretty well, and Arizona State and Missouri were each able to use smoke and mirrors to post double-digit wins again, but the drop-off was, on average, significant.

So what would this mean for 2015 (if we were to use this data for more than entertainment purposes)?

I attempted to update teams' rosters as much as possible to account for season-ending injuries, but I'm sure I've missed some (as will always be the case). That said, here's the Percentage Returning data for 2015 in all of its inexact, preliminary glory:

Team % of Offense Ret. % of Defense Ret. Average (Off., Def.)
Massachusetts 90% 90% 90%
North Carolina 92% 81% 87%
Charlotte 89% 84% 86%
Ohio 84% 86% 85%
Georgia State 75% 94% 84%
Temple 77% 90% 83%
Ball State 96% 70% 83%
Vanderbilt 74% 91% 82%
Texas Tech 80% 83% 82%
California 84% 79% 82%
Kent State 71% 88% 80%
SMU 81% 76% 78%
Colorado 75% 80% 77%
Ohio State 82% 72% 77%
Western Michigan 93% 60% 77%
Toledo 73% 80% 76%
Western Kentucky 74% 79% 76%
Tennessee 76% 76% 76%
Tulsa 94% 57% 76%
Kentucky 79% 72% 76%
Team % of Offense Ret. % of Defense Ret. Average (Off., Def.)
Virginia Tech 78% 72% 75%
New Mexico State 74% 76% 75%
Northern Illinois 68% 81% 74%
Cincinnati 83% 64% 74%
Penn State 80% 66% 73%
LSU 86% 60% 73%
Wisconsin 70% 76% 73%
Arkansas State 88% 57% 72%
Pittsburgh 73% 71% 72%
Troy 70% 74% 72%
NC State 71% 73% 72%
Oklahoma State 66% 77% 72%
Notre Dame 56% 85% 71%
Purdue 78% 63% 71%
San Jose State 80% 60% 70%
Southern Miss 83% 58% 70%
Middle Tennessee 70% 70% 70%
Baylor 54% 85% 70%
Hawaii 74% 66% 70%
Florida International 66% 72% 69%
USC 68% 70% 69%
Arkansas 81% 56% 69%
Bowling Green 94% 43% 69%
Wake Forest 69% 68% 69%
TCU 87% 50% 68%
Nebraska 71% 63% 67%
Iowa State 70% 65% 67%
Boise State 58% 76% 67%
Oklahoma 73% 61% 67%
San Diego State 57% 75% 66%
Central Michigan 63% 68% 66%
Illinois 61% 70% 65%
Tulane 74% 55% 65%
Minnesota 57% 71% 64%
Florida 53% 75% 64%
Utah 66% 61% 64%
Colorado State 52% 75% 63%
Arizona 72% 55% 63%
Connecticut 42% 85% 63%
Mississippi State 67% 60% 63%
Miami-FL 61% 65% 63%
Navy 67% 58% 62%
Texas State 74% 51% 62%
Northwestern 46% 78% 62%
Memphis 79% 45% 62%
UCLA 62% 62% 62%
North Texas 69% 54% 62%
Georgia Tech 48% 75% 61%
Idaho 58% 64% 61%
Akron 63% 59% 61%
Ole Miss 62% 58% 60%
Michigan State 71% 49% 60%
Texas A&M 56% 64% 60%
Michigan 66% 53% 60%
Utah State 63% 55% 59%
Army 46% 72% 59%
Louisiana Tech 56% 61% 58%
Indiana 60% 56% 58%
Washington State 58% 57% 58%
South Florida 36% 79% 58%
Oregon 66% 49% 58%
Eastern Michigan 52% 62% 57%
UL-Monroe 37% 76% 57%
Florida Atlantic 65% 49% 57%
Arizona State 47% 66% 57%
Houston 45% 69% 57%
Duke 34% 78% 56%
Buffalo 73% 38% 56%
Georgia 55% 55% 55%
West Virginia 34% 77% 55%
New Mexico 62% 48% 55%
Missouri 57% 53% 55%
Miami-OH 37% 70% 54%
Stanford 73% 33% 53%
Air Force 61% 45% 53%
East Carolina 41% 64% 53%
BYU 51% 54% 53%
UNLV 58% 47% 52%
Syracuse 74% 31% 52%
Clemson 64% 40% 52%
Iowa 43% 59% 51%
Texas 56% 46% 51%
Rutgers 47% 54% 51%
Rice 68% 33% 51%
Team % of Offense Ret. % of Defense Ret. Average (Off., Def.)
Louisville 50% 50% 50%
Maryland 41% 59% 50%
Virginia 52% 46% 49%
Florida State 38% 60% 49%
South Carolina 30% 67% 48%
Nevada 38% 54% 46%
Marshall 35% 56% 46%
Auburn 34% 57% 46%
Old Dominion 49% 43% 46%
Alabama 26% 64% 45%
Washington 43% 44% 44%
South Alabama 51% 36% 44%
Oregon State 53% 33% 43%
Kansas State 25% 60% 43%
Fresno State 31% 52% 42%
UTSA 35% 48% 41%
Boston College 31% 49% 40%
UL-Lafayette 40% 38% 39%
Central Florida 51% 26% 39%
Wyoming 30% 46% 38%
UTEP 24% 50% 37%
Kansas 20% 38% 29%

Not surprisingly, some of the teams on the "nobody's back!" list from 2014 are on the "everybody's back!" side of the ledger this year. That tends to be the way it works.

Perhaps noteworthy: Ohio State returns 77% of its production from last year, which is genuinely unfair. (Equally unfair: Kansas is dead last.) Meanwhile, both Alabama (45%), Auburn (46%), and Florida State (49%) on the wrong side of 50%. And BC has managed to be near the bottom for a second straight year.

Anyway, food for thought. Hopefully I can get this data to where I can trust it next year. This was an exciting first step. Well .. an exciting first five steps, I guess.