Bill C.'s recent piece on mobile quarterbacks got me thinking. He looked at how a quarterback's probability of rushing influences his probability of taking a sack. His gut feeling was that rushing quarterbacks were bust-boom in that they might run for more yards but they were also more likely to take a sack. And, his data demonstrated that trend. Pretty cool.
But, it reminded me of a statistical technique more often used in medicine : number needed. Typically, this is employed in determining the number of patients one would need to treat in order to achieve one successful outcome, or to prevent one bad outcome. It can also be used to calculate how many patients would have to be treated in order to incur a single instance of a bad side effect. Here is how it is calculated :
- Define two groups (control group, and experimental group) and define the event you want to examine
- Determine how often that event occurs in the control group (CER, or control event rate) and in the experimental group (EER, or experimental event rate).
- Number needed for one event = 1 / (CER - EER).
Note : this number is always rounded up, because you can't have a partial instance.
Example : say you have twenty people with an infection that is 50% fatal if untreated. You divide them into two equal groups : one group gets a new treatment, and one does not get it. In the untreated group, five die ; in the treated group, two die. CER = 5/10 = 0.5. EER = 2/10 = 0.2. Number needed to treat = 1 / (0.5 - 0.2) = 3.33... = 4. So, you would on average need to give four people that treatment in order to save one. The lower the magnitude of your number needed, the more effective it is to make that decision. In our example here, if number needed is 1, then the treatment is extremely effective ; if number needed is 20, then it's probably not all that great.
This strikes me as potentially useful for football. After all, many things we are trying to examine can be modeled in this same way - as a binary decision. If we ask ourselves "How many times do we need to pass instead of run in order for X event to occur?", number needed can provide that answer.
I don't have a lot of data to work with - all I have is what I could cross-reference from the NCAA stats website's tables, and they don't tally sacks from the quarterback's perspective. But, there are some interesting numbers to play with.
For instance, we could look at how many times a quarterback needs to attempt a pass (rather than to rush) in order to score a touchdown. I would call this term number of passes needed per touchdown, or, NPN-TD. In 2015, Trevone Boykin had 33 touchdowns in 492 pass attempts, and 8 touchdowns in 152 rushes. If we divide all his plays into passes (experimental group) and rushes (control group) then EER is 0.067, CER is 0.053, and number needed is 70, or, 6 per game. What this calculation effectively states : if Boykin had chosen to pass instead of rush 70 times over the course of the season, he would have put one more passing touchdown on the board. Compare this with Tanner McEvoy, whose NPN-TD is -2 ; this signifies that he was more likely to score when he rushed, rather than when he passed.
We could also look at how many times a quarterback needs to attempt a pass (rather than to rush) in order to throw an interception ; I would term this NPN-INT. In 2015, Jake Waters had 7 interceptions in 397 pass attempts, and 0 interceptions in 154 rushes. His NPN-INT is 5.
There are weaknesses to this calculation. For one, it is blind to situation unless the data is segmented out by situation (down-and-distance, field location, etc.) So, it does not do much to tell us exactly what a quarterback should do in a specific situation. Second, these are not necessarily "real" numbers. If CER and EER are very close, the magnitude of number needed may be in the hundreds. In medicine, this doesn't become much of a problem because we are treating thousands of people at a time. In football, however, number needed can produce an outlandish figure : Dak Prescott would have needed to throw 661 more passes in order to gain one more passing touchdown, or, 51 more per game. Third, it's not opponent-adjusted (nor am I certain how to do that in this case).
That said, number needed effectively compares two mutually-exclusive options, based on their event rates. And that may help in making standardized comparisons between players with complex data. We can look at J.T. Barrett, whose NPN-TD is 2 per game ; interestingly though, Justin Thomas (Georgia Tech) and Kale Pearson (Air Force) have the same NPN-TD. We can see, too, that J.T. Barrett's NPN-INT is 3 ; Brett Hundley's is 7, Marcus Mariota's is 8... and Drew Hare's (Northern Illinois) is 12.
Moreover, number needed allows us to assess balance. Going back to Tanner McEvoy, his NPN-TD is -2 per game, IE, switching two passes to rushes per game should result in an additional touchdown, somewhere down the line. What this tells us is that McEvoy is a much more effective scorer on the ground than in the air (because fewer decisions need to be changed in order to have an impact). Compare that to Dak Prescott at 51 per game (!) or 661 over the course of the season. In order to add an additional passing touchdown, the number of run-to-pass changes would exceed the total number of plays he actually ran. This large magnitude indicates that his scoring rates were very similar, and thus that he was very balanced.
Lastly, we can transition from (passing attempts, rushing attempts, passing touchdowns, rushing touchdowns, interceptions) to two (NPN-INT, NPN-TD) which encapsulate the quarterback's performance. My gut is that you want both of these numbers to have high magnitude ; this would mean that their interception rates were lower, and that their touchdown rates were balanced. Quarterbacks who have a low NPN-TD would not be using their most effective strategy as often as they should and would still be far from their Nash equilibrium. That said, it is hard to argue with a quarterback who, effectively, only needs to throw two more passes per game to gain a touchdown somewhere along the line. Ultimately, it may be that these metrics have to be interpreted in context of each other.
Here's the table of NPN-INT and NPN-TD :
| Player | Added passes for 1 INT | Added passes for 1 TD |
| Trevone Boykin, TCU | 4 | 6 |
| Marcus Mariota, Oregon | 8 | -4 |
| Rakeem Cato, Marshall | 3 | 20 |
| Jake Waters, Kansas St. | 5 | -26 |
| Dak Prescott, Mississippi St. | 3 | 51 |
| Grant Hedrick, Boise St. | 3 | 19 |
| Fredi Knighten, Arkansas St. | 5 | 23 |
| Brett Hundley, UCLA | 7 | -12 |
| Marquise Williams, North Carolina | 4 | -5 |
| J.T. Barrett, Ohio St. | 3 | 2 |
| Tyler Jones, Texas St. | 5 | 4 |
| Tommy Armstrong Jr., Nebraska | 3 | 4 |
| Jaquez Johnson, Fla. Atlantic | 6 | -21 |
| Jacoby Brissett, North Carolina St. | 6 | 3 |
| Nick Marshall, Auburn | 4 | -22 |
| Cody Fajardo, Nevada | 3 | -3 |
| Terrence Broadway, La.-Lafayette | 3 | 5 |
| C.J. Brown, Maryland | 3 | -6 |
| Chad Voytik, Pittsburgh | 4 | 3 |
| Drew Hare, Northern Ill. | 12 | 15 |
| Greg Ward, Jr., Houston | 3 | -15 |
| Kale Pearson, Air Force | 5 | 2 |
| Reginald Bell, Eastern Mich. | 4 | 6 |
| Tyler Murphy, Boston College | 2 | -19 |
| Justin Thomas, Georgia Tech | 3 | 2 |
| Matt Davis, SMU | 4 | -6 |
| Lamar Jordan, New Mexico | 3 | 4 |
| Keenan Reynolds, Navy | 4 | -3 |
| Tanner McEvoy, Wisconsin | 2 | -2 |
| Angel Santiago, Army West Point | 6 | -3 |
A few guys stuck out to me † :
- Tanner McEvoy needs. to. stop. throwing. He's a far more effective scorer on the ground, and he's dreadfully prone to interceptions.
- J.T. Barrett has a passing game that seems boom-and-bust, with a low number of added passes per game to get a touchdown, but only a slightly higher number of added passes per game to get an interception.
Same goes for Justin Thomas (Georgia Tech) - Drew Hare (Northern Illinois) is my pick for the best numbers here. He has the best NPN-INT, and a mid-range NPN-TD which favors the pass. Thus, he's not turning the ball over, and he's a threat to score on the ground and through the air, slightly better when throwing.
- Unsurprisingly, quarterbacks from Navy, Army, and Nevada were some of the most effective runners (negative, low-magnitude NPN-TD). Instead they were stratified by their NPN-INT.
- Nick Marshall and Jake Waters have extremely similar stat lines in terms of interceptions and touchdowns : respectively, 7 INT / 20 passing TD / 11 rushing TD, and 7 INT / 22 passing TD / 9 rushing TD. But, Waters has a better NPN-INT, and a more-balanced NPN-TD. So, he's the better quarterback.
Lastly, I cobbled together a sort of quarterback score, based off these values. It's a modified harmonic mean (and, probably, in no way rigorous) † :
My reasoning is that a quarterback who is just as likely to score when rushing and when passing is a more dangerous quarterback (of course). So, a larger-magnitude NPN-TD should be rewarded. But, the practical difference between NPN-TD of 15 and NPN-TD of 150 is not that big. At the very least, it's much smaller than the difference between NPN-TD of 15 and NPN-TD of 1. So, I needed an expression that exhibited diminishing returns. Again, not all that rigorous.
Here are the results :
| Player | Score |
| Drew Hare, Northern Ill. | 13.17 |
| Brett Hundley, UCLA | 8.58 |
| Jaquez Johnson, Fla. Atlantic | 8.55 |
| Marcus Mariota, Oregon | 8.47 |
| Dak Prescott, Mississippi St. | 8.42 |
| Jake Waters, Kansas St. | 8.21 |
| Fredi Knighten, Arkansas St. | 8.02 |
| Nick Marshall, Auburn | 7.36 |
| Rakeem Cato, Marshall | 6.70 |
| Grant Hedrick, Boise St. | 6.61 |
| Jacoby Brissett, North Carolina St. | 6.39 |
| Angel Santiago, Army West Point | 6.39 |
| Tyler Murphy, Boston College | 6.22 |
| Greg Ward, Jr., Houston | 6.19 |
| Tyler Jones, Texas St. | 5.72 |
| Trevone Boykin, TCU | 5.37 |
| Reginald Bell, Eastern Mich. | 5.37 |
| Matt Davis, SMU | 5.37 |
| Kale Pearson, Air Force | 5.19 |
| Marquise Williams, North Carolina | 5.13 |
| C.J. Brown, Maryland | 4.67 |
| Chad Voytik, Pittsburgh | 4.56 |
| Keenan Reynolds, Navy | 4.56 |
| Terrence Broadway, La.-Lafayette | 4.40 |
| Tommy Armstrong Jr., Nebraska | 4.09 |
| Lamar Jordan, New Mexico | 4.09 |
| Cody Fajardo, Nevada | 3.72 |
| J.T. Barrett, Ohio St. | 3.30 |
| Justin Thomas, Georgia Tech | 3.30 |
| Tanner McEvoy, Wisconsin | 2.43 |
The only thing that seems weird to me is J.T. Barrett bumping elbows with Justin Thomas and Tanner McEvoy at the bottom of the rankings. Of course, I've watched next-to-none of these guys' film, so by all means, CMIIW.
But hey - that's what this is all about, right? No one can ever watch all the games, so it'd be nice to have a way to make sense out of the torrent of numbers generated over the course of a season. Moreover, a calculated stat of this kind is just a question ; if it doesn't reveal something unexpected, then it's mostly useless. The key is how we answer the questions it raises.
And, this is only a start. If you have suggestions I would love to hear them.
† : The online LaTeX equation editor is cool
