Does running lead to losing?

I recently penned an article about John Schneider's drafting strategy. In my opinion, the Seahawks got excellent value by picking up three running backs while most other teams settled for UDFA pickings. At least one respondent disagreed. He did not question the relative quality of these particular draftees, but the ultimate value of the running game itself:

"[The Seahawks] just spent an entire draft to get better at the one thing that is least likely to win them more games."

Strong words, but intelligently supported: Over the last 10 seasons, the relative ranking (1 to 32) of each team's top rusher has a correlation with team wins of -0.17.

That correlation coefficient is negative because higher rankings are lower numbers, so having a highly-ranked top rusher does help-- a little. Correlation coefficients range from -1 to 1, which means -0.17 is pretty darn weak. By comparison, the win correlation for yards per pass attempt ranking is -0.57, and for team defensive ranking (using DVOA) it is -0.52.

That assessment is hardly new. Dr. Ed Feng, writing for The Power Rank, states it bluntly:

"When it comes to winning in the NFL, passing is king. Rushing hardly matters."

Refined evidence comes from Advanced Football Analytics, a site of which I am definitely a fan. Brian Burke has studied the correlation between winning and a variety of statitistical performances, and his chart looks something like this [1]:

Fig 1

Stat	Win Correlation
Off Pass Yds/Att	+0.61
Def Pass Yds/Att	-0.47
Off Fumble Rate	-0.46
Off Int Rate	-0.45
Def FFumble Rate	+0.41
Def Int Rate	+0.39
Off Pen Rate	-0.37
Off Run Yds/Att	+0.18
Def Run Yds/Att	-0.04

According to these numbers, the only thing less important than offensive rush efficiency (measured in yards/carry) is defensive rush efficiency. Yikes! It would appear that teams are making a big mistake when they invest resources in the running game on either side of the ball.

The Passing Premium

Throwing the ball gives a bigger payoff. A 2015 season summary shows that the league averaged 4.1 yards per rush and 6.4 net yards per pass attempt (a number which includes sacks). Football Outsiders' Offensive DVOA metric shows that 24 teams had a positive DVOA when passing, peaking at +48.5%, while just 8 teams had a positive DVOA when rushing, peaking at +14.0%.

The quality of the passing game is also more variable from team to team. In 2015, the best and worst rushing teams averaged 4.8 and 3.5 yards/attempt, a difference of just 1.3 yards. The best and worst passing teams averaged 7.8 and 5.6 net yards/attempt, a difference of 2.2 yards. Using Football Outsiders' stats (all team-seasons from 2012-2015), the standard deviation for offensive passing DVOA is 20%, compared to 14% for offensive rushing.

If it's hard to be very good or very bad at running, then you'd expect an investment in the passing game to have a bigger absolute payoff in terms of production. Add in the fact that roughly 60% of all scrimmage plays are passes [2], and the argument starts to look very strong.

Despite the numbers, teams still run the ball because it is safer (fewer sacks & turnovers) and more consistent. It's also just as good at moving the chains, so it's vital that we understand the importance of the passing premium.

Consider three running plays that gain 2, 5, and 6 yards: you have one "failure", two "successes", and an average gain of 4.3 yards. Contrast with three passing plays that garner 0, 6, and 14 yards. As before, there are two successful plays and one failure, but the average gain is 6.7 yards. Both strategies make the same progress towards a first down, but passing the ball gives you greater progress towards scoring.

And you absolutely need those chunk plays in order to score. Long touchdown drives average close to 9 yards per play. Long touchdown drives averaging 4 yards per play literally do not exist [3]. And 83% of all plays which go over 20 yards are passes.

The Running Attitude

"It's the most consistent, proven championship formula in the history of this game."
- Pete Carroll, recently quoted by Danny Kelly in The real value of rushing yards in the NFL

Attitude is real. Do a google search on "fun in the workplace" and you'll find myriad studies showing that good attitudes have a positive effect on performance. The two most successful coaches over the last four seasons [4] are the masters of attitude. New England's Bill Belichick has instilled a cool, professional "do your job" culture that mutes the effect of negative attitudes. Seattle's Pete Carroll has fostered a more dynamic, individualistic culture that embraces fun and competition.

"I want my team to play like the Seahawks."
- Steve Kerr, head coach of reigning NBA champions Golden State Warriors.

Running the football may not be about having fun per se, but there is a limit to how much fun you can have while subjecting your body to the violence of a professional football game. So running embraces the physical nature of the game and, more imporantly, makes things not fun for the other team.

"If we ran 70 offensive plays in a game, we expected to knock opponents down at least 105 times to meet our goal. In some games, we had as many as 130 or 140 knockdowns. On those days, we were very productive offensively."
- Tom Osborne, Nebraska head coach and inspiration to Pete Carroll

Running the football is part of a comprehensive strategy (somewhat detailed in my 2012 article on physical domination). The effect on attitude is difficult to isolate, but there are more tangible benefits. First and foremost, a balanced attack is more consistent at producing first downs, even if it seems less effective at producing touchdowns, and this allows your defense more time to rest. When the Seahawks' offense became consistent in 2012, their defense made the jump from "pretty good" to "elite". Having the ability to run is especially valuable in the red zone, where a compressed field eliminates the passing premium.

Running speeds up the game (fewer plays), and the ability to run allows a team with the lead to force an end to the game, not only sealing the victory but reducing injury risk. Running frequently allows your quarterback to throw fewer passes and reduces the pressure when he does pass, meaning he takes fewer hits. These are season-long benefits, carrying over from one game to the next.

With all that, those statistical measures of team rushing efficiency don't really capture the benefits of running. In fact, they don't even tell us how good a team is at running the football.

Game Theory and Offensive Balance

The threat of running affects the efficiency of passing, and vice versa. To demonstrate, imagine that football is a simple guessing game with only two options for each side. The payoff matrix might look something like this:

Fig 2

	Pass Play	Run Play
Pass Defense	0 yards	6 yards
Run Defense	12 yards	0 yards

Passing is better when it works. But if the offense passes all the time, the defense will play 100% "Pass D" and shut them down. On the other hand, if the defense predictably plays pass on every down, the offense will just keep running. As it happens, the numbers dictate an optimum mixed strategy for both sides:

Fig 3

	Pass Play (67%)	Run Play (33%)
Pass Defense (67%)	44% gain 0 yds	22% gain 6 yds
Run Defense (33%)	22% gain 12 yds	11% gain 0 yds

Three typical pass plays will garner 0, 0, and 12 yards, for an average of 4 yds/play. Three typical runs will net 0, 6 and 6 yards, for the same average of 4 yds/play. Go ahead and play this game with a friend or a calculator and try out different strategies. They won't help. Neither side can improve by tweaking these percentages with the given payoff matrix.

But we're talking about resource investment, so let's consider some scenarios where the offense can improve by adding a big wide receiver that boosts their passing payoff from 12 yards to 18 yards; or they can add a stud running back and boost their rushing payoff from 6 yards to 8 yards. The strategies change as follows:

Stud Receiver Added

Fig 4

	Pass Play (75%)	Run Play (25%)
Pass Defense (75%)	56% gain 0 yds	19% gain 6 yds
Run Defense (25%)	19% gain 18 yds	6% gain 0 yds

Stud Running Back Added

	Pass Play (60%)	Run Play (40%)
Pass Defense (60%)	36% gain 0 yds	24% gain 8 yds
Run Defense (40%)	24% gain 12 yds	16% gain 0 yds

With the stud receiver added, pass attempts gain 18 yards 25% of the time and rush attempts gain 6 yards 75% of the time. Both play types improve to 4.5 yards/attempt.

Adding a better running back, pass attempts gain 12 yards 40% of the time and rush attempts gain 8 yards 60% of the time. Both play types improve to 4.8 yards/attempt.

This simple game demonstrates why rushing efficiency doesn't measure a team's ability to run. Any improvement or decline in running ability will be statistically distributed among the passing and running averages after the offense (and opposition defense) adapt their strategies. Likewise, to some degree, for passing ability. The real-world manifestation is very familiar: A defense will stack the box against an opponent who is bad at passing; linebackers will play deeper against an opponent who rarely runs the ball; play-action passing can exploit an aggressive run defense; and running the ball can exploit an aggressive pass rush. Not surprisingly, a team's rushing DVOA and passing DVOA have a strong positive correlation of .409 (2012-2015).

How convenient, then, that Brian Burke has already provided evidence of how these strategy adjustments occur in the real world. In previous studies, Burke concluded that teams weren't passing enough to maximize scoring. In the ANS article How Coaches Think: Run Success Rate, he determined that NFL coaches were playing the grid efficiently if one looked only at Success Rate.

"Based on SR, passing and running correlate at 0.41.... Coaches are optimizing Success Rate, although they’re probably thinking of a simpler version of SR.
So NFL coaches are playing minimax after all! They’re just using a very simple payoff function."

It's important to acknowledge that success rate is a measure of progress towards a first down. So coaches are only optimizing first downs, presumably at the expense of reducing their chance of scoring (because fewer passes means fewer big plays and less yardage gained).

But here's where it gets good. Burke's study absolutely confirms that good running teams are likely to "run down" their yards/carry average by running more often. But the reverse is not necessarily true: If a team adds a good deep-threat receiver, it will improve its yards/attempt because more successful passes happen to be deep passes. In other words, the sum of improvements to the passing game will be split between the passing premium and passing success rate; and because coaches are playing the grid for success rate, a smaller part of the improvement bleeds over into rushing efficiency.

Furthermore, the mathematics of mixed strategies show that an improvement in the weaker play type (played at a lower percentage) is much more valuable than an improvement in the stronger play type. Above, you'll note that an increase of +5 yards on successful passes only raised the overall play average to 4.5 yards, whereas an improvement of +2 yards on successful runs raised the overall play average to 4.8 yards.

A Better Rushing Metric

So far, we know that rushing efficiency statistics are a poor measure of a team's true rushing ability, because much of that ability is gifted to the passing stats. We also know that, late in the game, teams with a lead will run the ball very predictably [5], which means that winning causes a team to produce bad yards/carry stats regardless of how good they are at running. So everything we've heard about the correlation between rushing efficiency and winning is utter crap.

Other studies purport to show that running doesn't help defense, and rushing attempts don't lead to winning, but these studies are flawed. You can read about them in the footnotes [6] [7].

So what about rushing frequency?

We know for a fact that teams "run down" their rushing average to maximize success rate, so the frequency of rush attempts should be just as good, if not better, at revealing the true rushing ability of a team.

We also know that the passing game benefits running, but we cannot simply measure passing efficiency (it is far more dependent on quarterback play and other factors); but because rushing frequency reveals rushing quality, it should isolate the benefit to the passing game.

Finally, the suggested unmeasurable benefits of running-- such as defensive rest time, protecting the quarterback, and demoralizing your opponent-- depend more on frequency than efficiency. If real, these effects should show up in improved overall performance when a team runs more often.

With that in mind, I tabled all team-seasons from 2012 through 2015, counting rushing attempts, passing attempts, and DVOA (win totals are far too choppy). With a sample size of 128, the resulting correlation between rush frequency and total DVOA was a whopping +0.405.

But let's be honest, we know that's bogus. As noted previously, teams with a late lead will run the ball to kill clock, so good performance causes more rush attempts, not the other way 'round. To eliminate this effect, I recalculated rushing frequency using plays under more restrictive "non-situational" conditions:

All first quarter scrimmage plays excluding those when a team is up by 10+
All 2Q plays excluding those when a team is up by 8+
All 3Q and 4Q plays excluding those when a team is up by 7+
All overtime plays

That probably excludes more than it should, and fails to count the value of running well for teams that manage to close out games, but I'm erring on the side of caution.

I specifically contend that trailing teams should run the ball more, and evidence suggests that trailing teams pass because of increased risk tolerance and not time pressure [8], so no adjustment is necessary on the other side of the ball.

With a stricter sample of rushing attempts, the correlation drops to 0.243. Smaller, yes, but the fact that it's positive at all bucks the conventional wisdom pretty good. And there's more to tell.

I've never denied that passing efficiency is important, and even crucial. But some teams are bad at passing. And by "bad" I mean "horrible". The 2012-2015 league average, measured by DVOA, is +10.67%. In that span, 31 teams were at least 15 percentage points below average, ranging from -5.3% to a whopping -35%. By comparison, the league average rushing DVOA was -4.51%, and just 8 teams slotted in at 15 percentage points below that.

Those bad passing teams are going to run the ball a lot (especially early in the game) simply because they have no other choice, and it does little to help their passing (an essential component of the value of running). They will include most teams that have a quarterback injury mid-season, teams that should have never been counted in the first place when looking at whole-season correlations. Therefore, we'll exclude the bottom quartile of passing DVOA teams, leaving a sample of 96 team-seasons.

Consider that, if the passing proponents are right, this refinement should reduce or eliminate the correlation. They say passing leads to winning, and we've limited our sample to teams which are at least decent at throwing the ball, so the benefit of passing should hit home even more. On the other hand, if I am right...

Fig 5

Boom. Let's go even further and look at the top half of passing DVOA teams (top 63, actually, because there was a tie at 64/65). Again, if the stat-grinders are right about passing, we should expect zero or negative correlation with rushing frequency, because these are average and above passing teams. But in fact...

Fig 6

Think I'm cherry-picking with this limited set? In fact, I don't really care if running the ball knocks your 6-10 team down to 5-11 instead of raising them to 7-9. Pete Carroll is talking about a Championship Formula, which means you darn well better have a good quarterback to give you a decent passing game. When that is the case, your best strategy is to protect him, make him more effective, and help out your defense by running the football. (If you're curious, the correlation for the top quartile [1st-32nd in passing DVOA] is 0.430, and the second quartile [33rd-63rd passing DVOA] is 0.503. Consistency defies noise.)

Putting The Nail In The Coffin

So long as the rest of the league is spending high-round draft picks on pass blockers and 8-figure salaries on wide receivers, teams that spend resources on the running game will get more bang for their buck [9].

The mathematics of game theory show that you compound this advantage even more because it is better to improve the weaker side of the play grid (running) than the stronger side (passing) if you optimize your play frequency.

And that's a nightmare for an opposing defense. If they continue to structure the personnel and play calling based on a league-average 60/40 pass-to-run ratio, they are not playing the optimum grid for success rate, and you can exploit that by increasing the run frequency even more. Success rate will skyrocket. On the other hand, if they do optimize to limit your success rate by playing more to the run, they must necessarily give up a greater percentage of successful pass plays, which means more big plays.

In other words, you can earn back the passing premium.

Sticking with the non-situational plays (i.e., excluding those with a big lead as per above), I tabled "explosive plays" of 20+ yards, separating run and pass, for all teams. For all teams, the correlation between non-situational rush frequency and explosive play percentage when passing is a positive 0.273. For teams in the top 63 of passing DVOA, the correlation is a whopping 0.478.

Passing adherents might be surprised by the magnitude, but not the effect. The more often you run, the more successful you'll be on the (perhaps rare) occasions when you throw, especially throwing deep.

Curiously, there is also a strong positive correlation between rushing frequency and explosive play percentage when rushing. Among all teams, it is 0.353; among the top-63 in passing DVOA, it is 0.441. The more frequently you run, the greater the percentage of rushing plays that go for 20 or more yards. That's counter-intuitive to the simple game theory model, but it makes sense in football: Defending the pass with smaller, faster players and deeper linebackers exposes you to short runs but gives you better control of backs who get past the line of scrimmage. Defending the run with bigger, slower players crowded at the line exposes you to breakout runs when the back gets into the second level.

Well, that all looks great for running the ball, but let's keep in mind that the "explosive play" percentage for the average team in this sample is 8.5% while passing and a meager 2.5% while running. The explosive passing plays are far more important, and even if you slightly increase the percentage within that statistical "bucket", you are dipping into that bucket much less often (because you run more). We ought to look at the total explosive play percentage.

Okay. The correlation is muted, of course, but it remains a positive 0.117 among all teams. And for the top half of the league in passing efficiency...

Fig 7

Astonishing, what? This also validates my "non-situational" criteria [10]-- if it still included a lot of "running because we're already winning" plays, the explosive play% would shrink for teams that run the ball a lot.

More importantly, though, we've already shown a positive effect on team performance when rushing more. Isolating the effect on big plays demonstrates one of the mechanisms by which this happens, and confirms a prediction. That's, like, science, am I right?

"Yeah."
- Marshawn Lynch, professional demoralizer

Footnotes:

[1] The website has a negative sign on the defensive forced fumble rate, but I'm certain this is a misprint and so it was changed to positive.

[2] Raw stats show a lower frequency, but if you throw out kneel-downs and count sacks as passing plays, it's closer to 60%.

[3] From 2012-2015, among 2823 touchdown drives that covered more than 70 yards, teams averaged 9.5 yards/play. A paltry 1.2% of those drives averaged 5 yards/play or less. The absolute bottom was 4.44 yards/play.

[4] Certainly the Denver Broncos would qualify as equally successful over the same time span, but those years include two different head coaches.

[5] Previous Defense helps offense helps defense article compared quarter-by-quarter numbers for teams with a 7+ point lead. Fourth quarter yards/play plummeted.

[6] From Brandon Gdula at numberFire, Establishing the Run Is a Recipe for Losing in the NFL:

"Over the past five seasons, first-half rushing attempts have actually had a slightly negative correlation with season-long win totals."

This unwittingly measures pace. Fast-paced offenses may be be good offenses, but they don't give much rest time. In 2014, the Philadelphia Eagles led the league in total plays (a whopping 10% more than average) but were dead last in time of possession per drive (a staggering 20% below average). The frenetic pace made them 7th in total rush attempts, but could not be expected to help the defense.

I found that first half rushing attempts had a positive correlation with DVOA of .086, and 1st Half pass attempts a positive correlation of .092. First half rushing percentage correlates with DVOA at +0.01. So all the study found was a bit of noise by using win totals instead of a more advanced performance metric.

More offensive plays of both types correlates with good offense (you keep the ball longer) and gives the defense more rest time, hence the positive correlations. They should actually be much bigger, but here's what's going on: When a team scores quickly with a big play, creates a turnover deep in the opponent's territory, or has a big special teams play, they gain a big advantage towards winning but run fewer plays. A defensive or special teams score costs them an entire possession. Likewise, teams which give up these plays will themselves have more time and more possessions. Thus the correlations are muted (for both play types).

The rush frequency correlation is also disturbed by the end-of-half strategies. Whichever team is able to run the last possession has an advantage towards winning, and this last possession usually involves a lot of passing. Teams which run that 2-minute drill successfully will have even more pass plays; teams which run it badly will give the ball back and allow their opponent to run the 2-minute drill.

Teams that are really dedicated to running the ball will make hay in the second half when the opposition is worn down. For all teams, passing a bit more in the first half (when conservative play is expected) is a good idea, so a whole-game comparison is really necessary to see the difference in strategies.

Most importantly, it is a well-known phenomenon that coaches play to extend the game (maintain a plausible chance of winning) at the expense of forgoing smart risks. If a team is bad on defense or, especially, bad at passing, they will run the ball a lot in the first half. Coaches of these very bad teams pass up their best chance to win to avoid the real possibility of an ugly, fast blowout. As they inevitably fall behind, they start passing against an opponent who expects it. The first-half-only analysis misrepresents bad teams as heavy running teams, when in fact they are not.

I compared first-half rushing frequency to total DVOA among the top 63 teams in passing DVOA and the correlation was a solid +0.305. Case closed.

[7] Also from numberFire, Joe Redemann states that No, Elite Running Backs Don't Help Their Teams' Defenses

"The truth is, there's no correlation -- the r-value is 0.10 -- between rushing and defensive performance according to our schedule-adjusted Net Expected Points (NEP) metric. It may seem intuitive that holding the ball would help a defense -- and it might to a small degree -- but there are plenty of instances where that's not the case."

This is simply the wrong metric. If Adrian Peterson breaks off a 50-yard touchdown run, that's a huge chunk of Net Expected Points, but it results in a very short possession. On the flip side, better defensive teams will tolerate a negative rushing NEP (on the offensive side of the ball) as part of conservative play and clock management, especially late in the game.

To correctly test the premise of offense helping the defense rest, they should have looked at plays per drive, then ran a second test to ascertain whether rushing frequency (or efficiency) was of any help.

[7.5] There is no footnote 7.5, so you must be reading these sequentially. I worked hard on the details, so I appreciate that.

[8] Note that a team which is trying to kill clock takes off an extra 40 seconds for a running play versus an incomplete pass, but a trailing team loses only about half that time under the same comparison (they can choose to hurry after the play), and gains no time advantage for a completed pass over a running play.

If a team has no hope of winning, they are adopting a losing strategy by prolonging the game and risking injury. Trailing teams pass more because they are willing to take more risks in order to score (I looked at offensive performance with a 7+ point deficit, comparing quarter by quarter; the 3rd quarter was no different from the 2nd quarter, and yards/play did not change in the 4th quarter; there was only a spike in touchdown percentage and turnovers).

Situations where a team has to pass because of time constraints are relatively rare. A good way to assess these situations is to look at onside kicks. From 2012-2015, there were 118 games (122 kicks) featuring a 4th quarter onside kick when the kicking team was trailing by 1 to 15 points. There were 1024 games played in that span, or, more importantly (because we already accounted for the strategy change by winning teams) 2048 game-sides. That means 118/2048 = 5.76% of game-sides actually reached a time-critical situation requiring multiple scores AND where the team actually managed to, you know, score.

Of these kicks, 90 were in the last 3 minutes (74%). Allow 3 minutes for the prior scoring possession and we're looking at 6 minutes of game time in 6% of game-sides, accounting for the greater part of situations where a team had to pass to save time for a chance at winning. That's about half a percent.

[9] I won't be looking at rush defense, but the same principle applies. Ahtyba Rubin is a bargain at $4M a year, and Jarran Reed was steal in the 2nd round of the draft.

[10] I originally ran the numbers comparing non-situational rush% and total explosive play% (all plays), and the correlations were similar, but predictably muted by conservative/clock-killing drives when winning teams did not get explosive plays. The correlation between (non-situational) rush% and explosive play% for all teams was 0.095, for the top 63 in passing DVOA it was 0.254.