Quarterback salary as a percentage of the salary cap

The following data set in regard to Super Bowl winners was presented on OverTheCap.com in January:

Year	Name	Cap %
1994	Young	13.10%
1995	Aikman	6.70%
1996	Favre	10.20%
1997	Elway	5.20%
1998	Elway	5.00%
1999	Warner	1.30%
2000	Dilfer	1.60%
2001	Brady	0.47%
2002	Johnson	9.60%
2003	Brady	4.40%
2004	Brady	6.30%
2005	Roethlisberger	4.90%
2006	P. Manning	10.40%
2007	E. Manning	9.20%
2008	Roethlisberger	6.80%
2009	Brees	8.30%
2011	E. Manning	11.70%
2012	Flacco	6.60%
2013	Wilson	0.56%

Now, I don't want to rip on the author too much, because he's pointing out something that looks interesting more than he's drawing any firm conclusions. But conclusions have been drawn. To wit, the history seems to indicate that paying a quarterback -- such as Russell Wilson -- too much of the cap -- such as $20M+ per year, which is 14% -- is likely to eliminate a team from Super Bowl contention.

And that's just dumb.

It's An Apples to Oranges Comparison

The chart shows only the quarterback's one-year cap hits, without regard to variations caused by cap growth, uneven contracts, or extensions.

Superficially, paying Russell Wilson $22M a year (the same as Aaron Rodgers) would eat up 15% of the $143M salary cap. But if Wilson gets exactly that on an extension, and the cap grows at its current $10M/year rate, his cap hit will never be 15%. Assuming a four-year $88M extension ($22M per year) with $35M paid as a bonus,

[EDIT: Numbers corrected, thanks to Jason_OTC]

2015: $3M salary + $7M bonus = $10M / $143M = 7.0%
2016: $13.25M salary + $7M bonus = $20.25M / $153M = 13.2%
2017: $13.25M salary + $7M bonus = $20.25M  / $163M = 12.4%
2018: $13.25M salary + $7M bonus = $20.25M  / $173M = 11.7%
2019: $13.25M salary + $7M bonus = $20.25M  / $183M = 11.1%

Now, you can debate those specific numbers, but that's not the point. The point is that these one-off cap hits don't really tell us what Super Bowl winning teams paid for their quarterback. NFL teams are constantly juggling their cap chargers, and if a quarterback cashes in one season it likely means that another player is taking a small cap hit relative to his own contract. A season or two later, the quarterback's cap percentage shrinks and that other player will be taking a bigger chunk.

(Note that Tom Brady's 2010 contract made him the highest-paid player in the NFL. That didn't stop the Patriots from winning 14 games that season and leading the league in DVOA, nor did it prevent Brady from going to two Super Bowls in the next five years and winning the Lombardi -- when his cap hit was listed at about 10%.)

More Like Apples to Pork Chops

Recklessly omitted from OTC's chart is an NFL team's companion to the salary resource, namely, draft capital. Every team gets seven picks per year, with players coming on for four years at a significant discount. In effect, each team has about 28 discount tokens available at any given time for stocking their rosters.

The expenditure of these tokens is not counted on the chart, making it (again) severely flawed. That much is obvious.

(Less obvious is another limited resource, namely, time under center. The direct cost of Wilson's discount token was a third-round draft pick, but the cost of finding a discount quarterback is markedly higher. Whether it's an undervalued veteran like Drew Brees, an unproven new player like Tom Brady or Kurt Warner, or a backup quarterback like Aaron Rodgers, teams have to spend wasted seasons, draft picks, and free agent money to identify their man.)

Or Mabye Apples to Yellow Cake Uranium

The NFL & NFLPA began a new CBA in 2011. OMG. Did you know that? Did you forget?

The new deal includes a rookie salary cap. The #6 overall pick in 2010, Russell Okung, signed a rookie contract for $8M/year. The #6 overall pick in 2011, Julio Jones, signed a rookie contract for $4M/year.

That difference of four million dollars is a lot of money. It's even more for higher draft picks, and extends (somewhat less spectacularly) through all 7 rounds. This means that, in 2011, not just more money overall but a higher percentage of the cap was available for paying veteran players.

In 2011.

The percentage of cap space going to veterans would increase every season under the new CBA, as more players rotate in on cheaper deals. It would not reach its peak until four rookie classes were in place. Ergo, when considering future veteran contracts, the historical list of NFL seasons that are valid for comparison looks something like this:

2014

Short list.

Math?

The problem with the math is that there is none. Simply looking for patterns in the numbers is akin to staring at the sky and observing that sheep-shaped clouds like to go left and the shredded cotton balls like to go right.

Some specific flaws:

1 The causal explanations are unsupported. If success comes from paying a quarterback less, the distribution should be bottom-heavy, dominated by quarterbacks earning less than 5% with very few outliers in double-digits.

Alternatively, it may be said that there is a "proven model" of paying quarterbacks in the 6-10% range, and therefore 15% won't work. But there are winning quarterbacks at 1.6% or less. How can a 50% increase on the "proven model" be considered an excessive divergence when the bottom end of that proven model is 300% higher than four historical winners?

2 The source population is ignored. Seahawk fans will remember a year ago when Seattle was dismissed as a contender based on the historical fact that such teams were unlikely to return to the Super Bowl. The fallacy in that reasoning is that the Seahawks' chances were being measured against 31 other teams (or 15 others in the Conference). So while it's always true that Defending Champ vs The Field is a bad bet, the Defending Champ is a much better bet than any single other team.

We are not given the league-wide distribution of quarterback cap hits over the last 20 years. If such is dominated by quarterbacks in the 6-10% range, with very few outliers over 11%, then the alleged "proven model" may actually be the worst for winning the Super Bowl, with highly-paid quarterbacks having the best chance. Math, folks.

3 Trying to infer causal relationships exclusively from post-hoc data of unique events is basically superstition. If you don't follow that, let me give an example:

Season	Winning Quarterback	Team Points	Points Rank	Home Stadium Latitude
2014	Brady	468	4	42.4
2013	Wilson	417	8	47.6
2012	Flacco	398	10	39.3
2011	E. Manning	394	9	40.7
2010	Rodgers	388	10	44.5
2009	Brees	510	1	30.0
2008	Roethlisberger	347	20	40.4
2007	E. Manning	373	14	40.7
2006	P. Manning	427	2	39.8
2005	Roethlisberger	389	9	40.4
2004	Brady	437	4	42.4
2003	Brady	348	12	42.4
2002	Johnson	346	18	28.0
2001	Brady	371	6	42.4
2000	Dilfer	333	14	39.3

It appears that teams have the best chance of winning a Super Bowl when they score points in the upper 300's or very low 400's. It's definitely better to drop below 350 (four winners) than go over the dreaded 500 mark (just one).

Of course, scoring frequency changes with the rules, so we can also look at scoring rank. The proven model for success is to be between the 6th- and 14th-most prolific scoring team. Stay out of the top five! The 9th-, 10th- and 14th-ranked teams are twice as likely to win the Super Bowl as the #1 overall team. And if you're closing in on the end of the season in position #3, you better hope for a Sunday night game so you can tweak your gameplan to avoid the curse position.

Even more telling is the latitudinal sweet spot-- situating your home stadium between 39 and 43 degrees North is obviously a smart move (11 out of 15 winners). Worse, teams starting the season with a home stadium north of 44.5 degrees latitude have just a 1/88 chance (1.1%) of winning the Super Bowl. Seahawk fans can take some consolation in the fact that we won't have to go all the way to California to get back into contention-- a move to Medford, Oregon would put us at 42.3 degrees North, with a mere 7 hour drive according to google maps.

A Real Comparison

We know that including rookie contracts and doing a post-hoc analysis are worthless. If we want to look for trends, then, we should look at veteran quarterbacks and their salaries across all teams and compare team success. The only salary data readily available came from 2013 and 2014, but that's fine because previous years are skewed by higher rookie contracts preceding the current CBA.

Adjusted Salary is just a small tweak to make the numbers for 2013 and 2014 (which have different caps) similar (I could have used cap percentage, but the math was more fun this way).

DVOA (Defense-adjusted value over average) is a good omnibus number for measuring a team's performance across all facets (offense, defense, special teams). Only quarterbacks with at least 10 starts are included.

ANY/A is adjusted net yards per pass attempt (counting sacks, interceptions, and touchdowns). We'll get to that and the "expected ANY/A" further down.

Season	Player	Team	Adjusted Cap ($M)	Team DVOA	Playoff Games	Salary- expected ANY/A	ANY/A	ANY/A Difference	Standard Deviation
2013	Eli Manning	Giants	22.73	-15.7	0	8.96	4.55	-4.41	-2.83
2013	Matt Stafford	Lions	19.42	-1.5	0	8.10	6.4	-1.70	-1.09
2013	Peyton Manning	Broncos	19.08	32.7	3	8.01	8.87	0.86	0.55
2013	Drew Brees	Saints	18.97	19.3	2	7.98	7.51	-0.47	-0.30
2014	Eli Manning	Giants	18.77	-5.8	0	7.93	6.67	-1.26	-0.81
2014	Ben Roethlisberger	Steelers	17.38	12.1	1	7.58	7.82	0.24	0.16
2014	Jay Cutler	Bears	17.02	-13.8	0	7.48	5.57	-1.91	-1.23
2014	Drew Brees	Saints	16.93	-1.7	0	7.46	6.77	-0.69	-0.44
2014	Aaron Rodgers	Packers	16.15	23.0	2	7.26	8.65	1.39	0.89
2014	Matt Ryan	Falcons	16.10	-5.4	0	7.24	6.71	-0.53	-0.34
2014	Peyton Manning	Broncos	16.10	29.4	1	7.24	7.68	0.44	0.28
2014	Philip Rivers	Chargers	15.33	-0.6	0	7.04	6.45	-0.59	-0.38
2013	Tom Brady	Patriots	15.04	18.9	2	6.97	6.13	-0.84	-0.54
2013	Philip Rivers	Chargers	15.02	6.4	2	6.96	7.79	0.83	0.53
2013	Ben Roethlisberger	Steelers	14.82	0.9	0	6.91	6.24	-0.67	-0.43
2014	Matthew Stafford	Lions	14.55	4.4	1	6.84	6.03	-0.81	-0.52
2014	Joe Flacco	Ravens	13.62	21.9	2	6.60	6.66	0.06	0.04
2014	Tom Brady	Patriots	13.62	22.1	3	6.60	7.01	0.41	0.26
2013	Tony Romo	Cowboys	12.88	-2.8	0	6.41	6.54	0.13	0.08
2013	Jay Cutler	Bears	11.30	6.6	0	6.00	6.23	0.23	0.15
2014	Tony Romo	Cowboys	10.83	13.7	2	5.88	8.11	2.23	1.43
2013	Matt Ryan	Falcons	10.46	-10.4	0	5.78	5.72	-0.06	-0.04
2013	Alex Smith	Chiefs	9.27	17.5	1	5.47	5.94	0.47	0.30
2013	Joe Flacco	Ravens	7.41	-6.7	0	5.00	4.5	-0.50	-0.32
2013	Chad Henne	Jaguars	5.07	-38.2	0	4.39	4.86	0.47	0.30
2014	Josh McCown	Buccaneers	4.37	-28.3	0	4.21	4.3	0.09	0.06
2013	Carson Palmer	Cardinals	4.36	10.0	0	4.21	5.67	1.46	0.94
2014	Alex Smith	Chiefs	4.23	10.4	0	4.17	6.14	1.97	1.26
2014	Kyle Orton	Bills	3.68	10.5	0	4.03	5.69	1.66	1.06
2014	Ryan Fitzpatrick	Texans	3.11	-4.5	0	3.88	7.15	3.27	2.10
2014	Brian Hoyer	Browns	1.15	-6.7	0	3.38	6.11	2.73	1.75

Some quick observations:

Among teams with a DVOA of +12%, 8 out of 10 paid their quarterback at least $13.6M (adjusted).

Among teams with a positive DVOA, 14 out of 17 paid their quarterback at least $9.2M. Two of the remaining three-- Carson Palmer and Alex Smith-- were on low-cap years of much more lucrative contracts ($16.5M/yr and $17M/yr).

The lowest cap hits among quarterbacks who won a playoff game were Tony Romo's 2014 (adjusted) hit of $10.8M (contract average $18M/yr) and Joe Flacco's 2014 hit of $13.6M (contract average $22M/yr).

Overall, a higher QB salary has a positive correlation both with team success (DVOA) and playoff games:

Some Reasonable Exclusions

I'll make one concession in that if you pay your quarterback like an elite player and he performs below league average, you aren't in good shape to win a Championship.

To determine which quarterbacks are underperforming (and by how much), we start with ANY/A. This would not be an accurate statistic for quarterbacks who contribute a lot of rushing, but as the veterans table does not include Wilson, Luck, Newton, Griffin, or Kaepernick, it should work just fine for the nonce.

Now, a quarterback who records 10.0 adjusted net yards/attempt is not worth three times as much as a quarterback at 3.3 adjusted net yards/attempt. He's worth more like 20 times as much, so the salary ranges are considerably wider than the ANY/A range. To line those up, I computed an average ANY/A among all 2013-2014 QB's with at least 300 attempts (6.16) and a standard deviation (1.56). I did the same for the veteran salaries ($11.9M average, $6.0M standard deviation). It's then a simple calculation to show that Joe Flacco's 2014 adjusted salary of $13.62M (+0.28 standard deviations) produces an expected ANY/A of 6.597.

Now let's re-examine the data and leave out those quarterbacks who performed a full standard deviation below expectations. Note, first, that this is not a sweeping arbitrary exclusion-- all three quarterbacks who are removed (Cutler '14, E.Manning '13, and Stafford '13) still show up for their other season; and all lower-paid quarterbacks who over-perform are still included.

Note secondly, and most importantly, that the original premise under discussion is the best strategy for winning a Super Bowl. Spreading $20M in cap among four players gives you the best chances of winning a minimum of 6 games, but putting all your eggs in one basket ($20M to a worthwhile quarterback) is better for a championship. It's far more likely for a single player to outperform expectations in a single season than it is for four different player to do so simultaneously.

And with just those three exclusions, the correlations look like this:

For normal statistics with typically random sampling, you'd want a correlation coefficient of .7 to show significance for these these sample sizes. But in the chaotic world of NFL football, .46 and .49 are pretty strong.

Finally, it bears repeating that all NFL teams have the same salary cap. So any measure of team success versus spending across all positions would necessarily produce a flat correlation line. Therefore, the data show that spending money on a quarterback is a better strategy than spending it elsewhere.