Why college football recruiting rankings are flawed metrics

Everyone agrees that recruiting is an immensely important part of college football. Big picture points focusing on how only teams recruiting blue chippers are winning titles have obscured what still seems to be an obvious fact within the world of college football, which is that lots of talent is overlooked annually and the recruiting rankings regularly fail to predict the success of major teams each season.

So how do we reconcile these two data points? Recruiting sites might be our most useful resource, but are they really a very good one? A closer look at the world of recruiting penetrates through the large sample sizes and inconsistent criterion to reveal some interesting truths.

Why do the recruiting rankings seem to matter?

Most of the major programs are fishing in the same spots, which are where the recruiting services all congregate with their scales and tape measures to validate the catch.  Within that world, recruiting rankings serve to demonstrate which schools are landing the known fish, the obvious whoppers who bring a team great depth of size and speed.

For picking out which three-star loaded also rans might be sleeping giants? The services are nearly entirely useless.

And how many of the "big fish" blue chippers are good enough to guarantee results? Very few it seems. Here's 2014's top ten teams ranked by Football Outsiders' F/+ followed by their recruiting rankings the previous six seasons based on 247's composite class rankings which combine the service rankings.

Team	2014	2013	2012	2011	2010	2009
Ohio State	3rd	2nd	5th	7th	16th	4th
Alabama	1st	1st	1st	1st	5th	2nd
Oregon	21st	19th	13th	11th	13th	28th
TCU	42nd	35th	29th	34th	33rd	20th
Georgia	8th	11th	8th	6th	11th	5th
Georgia Tech	55th	76th	52nd	44th	42nd	41st
Ole Miss	15th	8th	47th	20th	23rd	19th
Baylor	26th	27th	26th	46th	40th	52nd
Clemson	18th	15th	15th	10th	24th	31st
Florida State	4th	10th	3rd	2nd	7th	13th

From this table you could reasonably draw a few conclusions. One is that there really are blue chip athletes that are so good they can make for nationally dominant teams...but they are in much more limited supply than commonly assumed. Ohio State and Alabama were the only two schools who seemed to land enough of them to effectively leverage that advantage in 2014 and both of those schools also have exceptional coaches in addition to exceptional talent.

The other conclusion is that the difference between a top 10 class and a top 40 class is potentially negligible if the team recruiting classes ranked outside the top 10 is grabbing the "right" types of players.

To further demonstrate the point, let's look at at a table from 2013 that shows all the Division 1 conference champions (excepting the independents) and how their six prior recruiting classes ranked within their own conference:

Conference	Team	2013	2012	2011	2010	2009	2008
AAC	UCF	9	11	2	1	5	2
ACC	Florida State	1	1	1	1	2	3
Big Ten	Michigan State	5	5	7	4	4	9
Big 12	Baylor	3	3	7	6	8	7
CUSA	Rice	7	5	9	7	5	7
MAC	Bowling Green	2	4	3	2	6	10
Mountain West	Fresno State	3	11	6	1	4	1
Pac 12	Stanford	10	7	4	6	3	9
SEC	Auburn	7	4	2	3	10	6
Sun Belt	La-Lafayette	1	4	1	5	5	6

This table makes it clear that how well 2013 teams had been recruiting compared to their conference peers was a very limited factor in determining the league's champion. Again we see one team in Florida State that recruited at a truly dominant level and saw results, but they were also in a league where most of their competition didn't have the same kind of access to the main blue-chip talent pool.

So it seems that the best strategy in college football is to combine one of the few elite coaches with top five recruiting classes every single year in order to gain a decisive advantage over your competitors.

Obviously that's not going to happen for most teams, so why do they insist on playing by the same rules, spending massive resources squabbling over the cream of the crop? Most Power-5 teams are not going to finish as the "king of the mountain" very often so why even approach recruiting in that manner?

The recruiting process

When you consider that the people responsible for coaching college football teams are also responsible for coaching said teams it becomes rather easy to see how the recruiting process might not always be the most efficient or thorough system imaginable.

As hard as some staffs work, one coach can't keep his eyes on all the potential prospects within his assigned turf. Imagine one coach from a Big Ten staff being responsible for recruiting "South Florida" and it's easy to see how quickly this becomes an impossible task. That coach's eyes have to be guided in order to see anything, perhaps by recruiting writers, street agents, high school coaches, skill coaches, or anyone who is plugged in on the local scene and can help a coach get "in" with a player.

So in the interest of time, recruiters will focus on specific schools, coaches, regions, agents, or connections to help them find players. If a player participates in various summer camps his chances of getting noticed rise considerably, but it's easy for talent to slip through the cracks and go unnoticed.

The "big fish" known blue chippers are usually the ones that participate heavily in the recruiting process, are well-represented or located in key schools, and who have the obvious size and speed to play college ball. A five-star ranking is intended to be a near guarantee that a player will be good so it's generally given out to the athlete who, at 18, is already nearly big, strong, and fast enough to project to the NFL. Obviously these players are rare and hardly representative of all the talent that will be NFL caliber four years later.

When you consider this process, or the way in which much more thorough organizations like the NBA scour the earth and then nearly miss on players like Hassan Whiteside, you realize how easy it could be for schools and recruiting services to regularly overlook even some obvious talents.

And judging by our tables above, they miss quite a lot.

What's getting overlooked?

The statisticians at FiveThirtyEight measured how teams recruit vs. how they perform in terms of wins over multiple seasons, and the results are helpful for demonstrating the recruiting services' oversights.

Let's start with Wisconsin, who consistently "get the most out of their recruits" if you assume that the services are accurately assessing talent. Where are the Badgers getting all of their players?

On a national level they follow the same program as every one else, attempting to cull recruits from developed connections in Florida, Texas, and California. However within their state, the Badgers regularly find several big time players despite having a population of only 5.8 million people. The key is the type of athletes who live in the state, namely big farm boys who grow up playing multiple sports and are far from their peak size and potential when they enter college.

Do you think the recruiting services regularly send scouts to rural games in Midwest America? How often do they even spot all the key recruits in a big metro area like Dallas-Ft. Worth with its nearly seven million people? If you are a gifted receiver playing on a running team with a terrible QB in a metro area you are unlikely to be effectively scouted. If you are that same player in nowheresville, Minnesota? Even less so.

Another issue is when players reach their full size and physical potential. The boys of Wisconsin, with their Northern-European ancestry, more commonly have frames and genetics that lend to creating big men who are well built for a contact sport. Northern European lineage comprises over half of Wisconsin's population.

The JJ Watt who balloons by 50+ pounds with college strength and conditioning is not outside the norm for the cheese-eating state.

Similarly, Utah also ranked rather high than "expected" and their rivals at BYU have also tended to accomplish a good deal without "blue chip" talent. Even just a cursory look at the demographics of Utah and the teams' rosters reveals that these schools make a habit of recruiting Pacific Islanders.

The missional efforts of the Mormon Church in Polynesia has led to many Islanders relocating to Utah and helps make young men from Samoa, Tonga, Hawaii, or other talent-rich locales very willing to come to school within the state. For players like Motekiaia Langi, there is little to eval other than bloodlines, sheer size and athletic ability, or perhaps rugby film. Naturally, the recruiting services have little to offer on such a player other than their customary two-star stamp that goes on all unknown players (why even have the one-star rating if it's never handed out?) regardless of whether they've been scouted or not.

So it's easy to see that rural players are typically overlooked, services aren't exactly doing studies to get scientific projections of player growth and size potential, and even talent in well known urban spots can be totally missed.

Finally there's the simple matter of fit and development. Why are Baylor, Michigan State, TCU, Georgia Tech, and Oregon perpetually under ranked by the services? Because each of these schools have strong systems on offense and/or defense that allow them to plug skill sets in and get a multiplier effect thanks to fit and development.

It's a team sport, after all, an eleven on eleven competition where no player can be free of the impact that his surrounding teammates have on every play.

Taking advantage of market inefficiencies

Power conference programs rarely dive into rural recruiting or do exacting, scientific measurements of which players they should be profiling. Instead, they try to spend their resources on getting a top coach and hoping he's more successful in the main talent pool than the competitors.

Consequently, schools like Baylor or Michigan State, who are outside of the elite in terms of resources, are generally the ones to figure out how to game the system and "over achieve" by choosing the right non-blue chippers to build competitive rosters.

However, what if a program with the resources of a Michigan or Oklahoma determined to quit the rat race of trying to build elite rosters in the midst of heavily-contested regions and instead starting spending piles of money on finding overlooked players and figuring out how to best develop them on campus?

Sooner or later some program is going to give it a try, and then we may see that final shield held up by the recruiting rankings, the lack of non blue-chip champions, finally lowered, and a more accurate and scientific tool placed in its stead.