Third Downs Continued: What is happening on third-and-13?

An in depth look at conversion rates at third-and-13 yards to go.

This is the 2nd installment of a series of articles on third downs in college football. You can find the most recent one here.

In my first post I showed third down conversion rates by distance to go for a first, as well as the pass % for each distance to go. The one thing that stuck out to me was an apparent spike in conversion % at third-and-13. Yes, it could easily be a small sample size issue, but my hope is after this article I will be able to convince both myself and you that there is something else at work here. First, I need to thank CFB Stats for making a ton of college football data available for the past eight seasons; this wouldn't be possible without them. Let's now get a look at exactly what I am talking about when it comes to third-and-13.

Screen_shot_2013-05-12_at_1

This is the same chart that shows us conversion %, pass conversion %, and rush conversion % for all third downs in games between 1-A teams in non-garbage time plays over the last 5 seasons. Non-garbage conventions are the same ones used by the F/+ ratings. This shows a clear spike at third-and-13 pass conversion % compared to third-and-12 and third-and-14. In addition this spike is only seen in passing plays. No season showed a spike in run conversion % at this distance and no year showed a decline in pass conversion % like you would expect if there was nothing going on here. So in the last five years, third-and-13 has a higher pass conversion % than third-and-11 or 12. But does this actually tell us anything about predicting third down performance in the future?

Sample Size?

Here are the sample sizes for third downs where a passing play occurred from 11, 12, 13, 14, and 15 yards to go for a first down.

Distance to Go 11 12 13 14 15
Total 2480 1924 1547 1235 1278
Conversions 642 431 408 264 217
Conversion % 25.9% 22.4% 26.4% 21.4% 17.0%

Something to note: Fifty-six successful third-and-13s over the last five years (3% of all passing third-and-13s that occurred) would have to change to unsuccessful in order for the conversion % to be the same as third-and-12. Perhaps that gives more strength to the thought that there is a large enough sample size we could run some statistical tests on the data. One statistical test we can run to determine if two samples are significantly different is a Student's T-Test. The Student's T-Test tests the difference of means of two samples to determine if there is significant difference between the samples. The test takes into account sample size and variance, so the T-Test can measure small effects over large samples and also large effects over small samples accurately.

A low P-Value (something less than .1/.05/.025, depending on the rigor you want) tells us that the difference in means of the two samples is significantly different than 0. In addition the Student's T-Test was developed by a statistician at the Guinness Brewery, which is far and away the coolest statistics history lesson I have heard. We can test each distance-to-go against the others to determine how statistically likely it is that they are in fact different. The P-value in this table tells us the chance that we got a bad sample and that the two distances are essentially the same.

Distance To Go 11 12 13 14 15
11 X 0.007 0.733 0.002 ~0
12 0.007 X 0.007 0.496 0.001
13 0.733 0.007 X 0.002 ~0
14 0.002 0.496 0.002 X 0.005
15 ~0 0.001 ~0 0.005 X

You read this chart like this; the chances that in the given data set the conversion % at third-and-13 to go is the same as third-and-12 to go is less than 1% (.007). There are some caveats I kind of ignored here. The T-Test assumes a normal distribution of the means, and the data isn't quite a normal bell curve.

In addition, these were two-sided T-tests even though I was specifically looking at if third-and-13 was greater than the others, not just different. But what is important is that the sample sizes do give us enough evidence statistically to say that there is a difference in means.

Performance On Third Downs

A reader in my first post suggested that I should look at how teams are performing on a yards-per-play basis to see if anything was happening. So let's look at a frequency histogram of how many yards teams gained on all third-down passing plays in my data set within the last five years.

Screen_shot_2013-05-27_at_9

The average passing (including sacks) third down yardage gained was 5.7 yards, but the median yard gained was zero (there were about 25,000 third down passing plays that gained 0 yards, nearly half of all third down passing plays). This is neat to know, but what I am really interested in are when teams have 11, 12, 13, 14, 15 yards to go. And these histograms are by percent as opposed to total number of observations with bucket widths of 1 yard.

Screen_shot_2013-05-27_at_9

Screen_shot_2013-05-27_at_9

Screen_shot_2013-05-27_at_9

Screen_shot_2013-05-27_at_9

Screen_shot_2013-05-27_at_10

(I am using R to make these plots, and I could think of no better way to display the information. If you have any ideas or suggestions to make these easier to compare then please shoot me an email or comment.)

It's tough to get much out of these plots -- it is kind of information overload. What can the histograms of the individual distances tell us? One interesting thing to note is that one of the least frequent yards gained on all downs to go is the exact yardage needed for a first down. I think there is at least one thing at play here. The measuring process for first downs could cause this gap. If the spot is just past the first down marker the spotter/recorder might just round up and say they are at the next yard line, and may round down if they didn't get it. What do you guys think? I almost feel like this would be the opposite though. If a team knows they have to get to a certain distance, I feel like that distance would show up more than on average because teams might do all they can to get as close to the first down marker instead of just going out of bounds normally. I'm not sure what to make of it quite yet.

Back to what we wanted to know in the first place, is third-and-13 any different than other distances? Here is the chart we looked at earlier, now with the average yards gained at each distance.

Distance to Go 11 12 13 14 15
Total 2480 1924 1547 1235 1278
Conversion % 25.89 22.4 26.37 21.38 16.98
Avg Yards Gained 5.76 5.60 6.56 6.20 5.58

As you can see teams gained almost one yard more per play than they did on third-and-11 and third-and-12. Teams at third-and-13 are losing a yard at the line of scrimmage compared to third-and-12, but are making it right back up with the success of their plays. Heck, the average yards gained on passing plays at third-and-13 is higher than the average yards gained on ALL passing third downs ... wow. The average distance "short of a first" is basically the same for third-and-12 and third-and-13. So if a higher percentage of plays on third-and-13 gain 7.5 more yards than average on third-and-13 than third-and-12, the third-and-13 conversion % would be higher. And that effect is exactly what we are seeing in the histograms.

Screen_shot_2013-05-27_at_10

The presentation of this sucks, I know, I just have no idea how else to present it. Fortunately, I think we are done with the histograms.

Potential Causes

But what is causing this effect? I believe third-and-13 represents an optimal distance where offenses can still run their base packages and gain 13+ yards, defenses have to respect the run as a possibility, and offenses don't have to sacrifice extra blockers to go run routes. At third-and-13 perhaps defenses are blitzing less because they don't want to get "burnt" but it is still a short enough distance that offenses can get yards while holding more potential receivers back as blockers than they might at third-and-15. This last part is testable, here are the percentage of passing third downs that end in sacks:

Distance to Go 11 12 13 14 15
Sack % 10.2% 9.6% 8.6% 10.1% 11.0%

The sample sizes are a little smaller than the conversions (133 sacks vs 408 conversions on third-and-13) but there is still a clear dip at third-and-13. The total sack % on all third downs was 7.9%. I think this tells us that teams can still get more yards on third-and-13 while not sacrificing any increased risk of a sack.

What does all this tell us? From my data I have shown that teams convert a higher percentage of passing plays, gain more yards per play, and also get sacked at a lower percentage of plays at 13 yards to go for a first than either 12 or 11 yards to go for a first. All of these factors together make it seem like there is some real effect going on here. But there is one final test we can run to be sure.

Just one last test, and ... uh oh

Lets just re-run the original analysis on the years 2005-07. I didn't know these years were available when I started the analysis and someone suggested I test these years, so here it goes. Here are the third down conversion % by distance to go for 1-A games in 2005-07.

Distance to Go 11 12 13 14 15
Total 1593 1203 935 782 820
Conversion % 25.7% 25.3% 20.4% 18.9% 18.4%
Avg Yards Gained 5.25 5.77 5.01 5.35 5.50
Sack % 10.6% 10.5% 10.8% 11.9% 10.7%

Well.......

Tumblr_m8zey5wu0r1qzeg2so1_500_medium

via 24.media.tumblr.com

Well that settles it, there is literally ZERO evidence of the third-and-13 effect we were seeing from 2008-12 in the 2005-07 data. Either something has changed since 2007 that would make it easier for offenses to complete third-and-13s, or this whole exercise was a lesson in small sample sizes. Either way it was fun, please comment away with any questions or suggestions.

(If anyone is interested, here is the compiled third down data from 2008-12 I used for this analysis.)

** Edit: Better Gif **

Trending Discussions

X
Log In Sign Up

forgot?
Log In Sign Up

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior users will need to choose a permanent username, along with a new password.

Your username will be used to login to SB Nation going forward.

I already have a Vox Media account!

Verify Vox Media account

Please login to your Vox Media account. This account will be linked to your previously existing Eater account.

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior MT authors will need to choose a new username and password.

Your username will be used to login to SB Nation going forward.

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join Football Study Hall

You must be a member of Football Study Hall to participate.

We have our own Community Guidelines at Football Study Hall. You should read them.

Join Football Study Hall

You must be a member of Football Study Hall to participate.

We have our own Community Guidelines at Football Study Hall. You should read them.

Spinner.vc97ec6e

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.

tracking_pixel_9341_tracker