An anatomy of the sack

Haha, jokes.

When you compare things you need a common unit to measure them with. Well, you don't need one but it makes it much easier. If I tell you the newtons produced by one spaceship engine and the exhaust speed of another you can't decide which you're going to use on your space yacht. Not only do you not have enough information, you don't even have the same information. No ranking is possible.

All you've learned is that it's time to go to a better space yacht engine dealer.

Sports fans and analysts spend an awful lot of time creating ways to compare players and arguing about which ones should be used in specific situations. Somehow, after all this collective effort, ESPN still managed to invent QBR which is to QBs what phrenology is to crime solving. If you're using it right now your either fucking insane or a racist time travelling detective.

Still, there are plenty of very useful stats on the offensive side of the ball. Without ever having seen an offensive skill player take a snap I can create a pretty good idea of what I'm going to see if I do watch him with 500 snaps worth of data. Sometimes much less.

Tragically, on defense almost all the individual stats are utter garbage. Therefore I don't trust stat based comparisons of defensive players. But I do think that there is room to improve.

Right now I'm going to complain about defensive sacks.

Why Defensive Sack Rates Suck.

We know that sacks are an excellent QB stat. Sack rate stabilizes amazingly quickly for QBs - faster than completion percentage. Intuitively they seem like a full offense stat but college sack rates are well correlated with NFL sack rates for QBs. So I tend to think there is a lot of individual skill involved in being a David Carr.

Because Sack rates are very stable for QBs I expect them to be highly variable for defensive players. I believe end of season sack totals for defenses and defensive players are more dependent on QBs played than defensive skill.

The 2012 offensive sack rate range: 3.58%- 8.70%, Variance: .0003

The 2012 defensive sack rate rate: 3.60% - 8.68%, Variance: .0001

If the offensive sack rates approximate the true skill measured by defensive sack rate then the different teams' defensive sack rates are (non-random) samples of that skill across multiple QBs. They should be (bad) approximations of the population mean. Therefore, I expect the range of defensive sack rates to be less than offensive. Which it is, but only just (.04). And the variance to be lower. Which it definitely is.

The same relationships hold up in past years but the difference in ranges is actually much larger (1.8 in 2011). Actually, in 2012 the ranges were atypically close. The average difference in ranges is right around 1.5.

Finally, despite the greater range within each year I expect the correlation from year to year to be higher for offensive sack rate since the QB sack skill is likely to remain the same while the schedule is guaranteed to change.

And R^2 for offensive sack rate is .20 versus .02 for defensive sack rate.

Graph!

In the end team defensive sack rate has almost no predictive value. The evidence points toward sacks being a primarily offensive skill and any inherent defensive talent in sack rate is completely overshadowed by schedule.

Essentially, I think individual QBs decide/manage to respond to pressure by taking a sack or not at a pretty consistent individual rate that the defense is not in any control of.

A Proposal for Evaluating Pass Rush.

We all know that there are good pass rushers. We all know that good pass rushers get sacks. Good strikers have to score in soccer. Good batters have to get hits in baseball. Good pass rushers have to get sacks. With fans at least, there is very little patience for arguments that good pass rushers aren't necessarily the ones with the gaudy sack totals.

They aren't though. Necessarily.

Once the ball snaps, if there isn't a rush, throw, or fumble in the next few seconds, there is going to be a sack. That is an immutable fact. One of the offense's most important jobs (on passing players) is to make sure that the time to sack is greater than the time to pass.

The optimal strategy for an OC (assuming pocket passing) should be to equalize sack times across the line. In other words if Player A is going to sack your QB in 2.5 seconds and Player B is going to do it in 7 the OC should move pass protection from player B to A. The solution with the greatest minimum time to sack should always be the one where the entire line has the same time to sack.

(If you suggest that this makes removing all pass pro an optimal solution you're clever but fuck you.)

Since pass protection resources aren't completely fungible the OC is never going to get to the optimal solution. An OC can't move 2/3 of a TE to the other side of the line. This means that some pass rushers are more likely to get a sack than others.

For great evidence of this watch Green Bay's offensive adjustments in the Green Bay-Seattle game from this season. After it becomes clear that Irvin can get to Rodgers in 1.5-3 seconds in a one on one he starts getting chipped and double teamed. Then the tackles start getting through a little faster.

Does that mean that Bruce Irvin is a better pass rusher than Brandon Mebane? No, he probably is, but still no. It just means that protection resources were misallocated to start.

To me this suggests that average time to sack or pressure is a better stat for measuring defensive pass rush ability. Obviously it is still dependent on offensive skill but I suspect there is some defensive skill to be found in the numbers - unlike in sack rate.

As support for that assertion, note that sack rate and time to sack aren't very well correlated for QBs.

For individual players my dream stat would look at the player's success rate for getting pressure/sacks over time since snap. Where that success rate is equal to the league average individual success rate (by position: DT, DE, OLB) I'd take the time. Call it Time to Average Sack Rate. It could be written TASR and pronounced taser. Which is bad ass.

This method is still vulnerable to OCs targeting some players but not others. If an OC decides he'd rather die than let Bruce Irvin through everyone else on the line is going to look better by comparison. It's also vulnerable to different success rate curves. Some guys may get all tuckered out after five seconds and lie down. Others might have incredible motors and work rate that let them overpower offensive players after four seconds. I doubt it but If i ever get the wherewithal to make this stat I'd have to ensure the success curves are broadly similar.

Also this is still football so the team sport caveat applies as usual.

The Bad News. Also the Good News.

Time to sack and time to throw numbers are out there. So checking the accuracy of my hypotheses regarding evaluations of pass rush is technically possible. But the numbers are floating around in excel sheets on other computers. Unfortunately I don't have access to them and, even if I did, as near as I can tell everyone who has done it has viewed them as QB stats, not defensive stats. So when I get my hands on some - and I plan to - there could very well be some tedious matching of plays to play by play info. So that will take time.

I don't like to tear down a method of evaluation without proposing a new one. And, in a dream world, I'd be able to offer the method fully formed and tested. Here, I can't. Sorry.

But at least now we know to be cautious of defensive sack numbers as a measure of talent.