A Defensive Stat Worth a Damn? Introducing Brian Burke's WPA+

So I’ll wait and see where it goes. Right now I think it looks like an interesting direction, but one that will be unkind to certain “non-impact” positions. Kind of like how football outsides struggles to quantify the individual offensive lineman’s impact.
 
But a stat is only limited to the creativity of the stat-creator. If he breaks enough of the obstacles, or even if he doesn’t, this looks like a good line of research.

It’s a number of different measurable factors determining the probability of success. The sum of logistic probabilities can be described by the binomial distribution. Neither of these distributions have as strict limitations on sample size and equality of variance as does the normal distribution. I think if WPA sum were tested against a binomial, you’d be better able to measure the contributions of situational pass rushers or nickle backs who aren’t in as many downs and it should better account for players that actually do vary wildly in their performance.

The key assumption here is stated: “Unlike a company, however, (as Wall Street sadly learned a couple years ago) a player’s individual performance from play to play almost certainly follows a normal distribution.”

Why is that the case? Some players make the occasional brilliant play and the occasional terrible play; others are more consistent. Some defensive players, as we all know, occasionally take downs off entirely. And even if all do follow a normal distribution, there are other characteristics of the bell curve that we don’t know (does everyone have the same standard deviation?).

The article tries to explain that away with this paragraph:

“Not every defender would have the same normal profile. As I mentioned above, there are the ‘gamblers,’ players who shoot gaps when they should be reading the play or cornerbacks who jump pass routes when they should stay in position. Certainly, +WPA and +EPA would be biased in favor of these types of players. But if their gambles were really hurting a team, I doubt they’d be given much slack and playing time by their coaches. Only ‘winning’ gamblers, who are taking smarter risks, would tend to survive long in the NFL.”

A rational actor model for football, brought to you by the Cato Institute? I have a hard time swallowing this. The best and most consistent players do not always start (but enough about Brian Russell already). Coaches are sometimes impressed by big plays or by, oh I don’t know, grittiness. We know this.

I don’t want to be altogether dismissive of this, but I’m having a tough time accepting the premises.

It seems like WPA+ should be interpreted as a measure of the tendency of a player to be in a position to make plays. As others have discussed above, players could have high WPA+ in a given season because:

1) They are good, and hence get to the right spot to make plays consistently
2) They play with good surrounding talent, which allows them make plays because of holes/opportunities created by their teammates
3) They happened to be in the right place at the right time on a few key plays

To me, it seems like WPA+ (particularly over one season) is too noisy to quantify true talent. And if that’s the case, I don’t really see how it’s useful.

would be to model an entire defense like the parts on a car. Engineers use a time-to-failure exponential model to compute the probability of failure over time. Since the car as a whole also has an expected value and variance for probability of failure at any given time, you can give weight to the different parts and then compare the part’s performance to the expectation for the whole car.

To model defenses, individual plays could be substituted for time units. Failure could be missed assignments, missed tackles, penalties, etc. You could also look at the effect of groups of players on the probability of a failure. The exponential could be expanded to a gamma distribution so that you could look at the probability of any number of failures over a specific number of plays.