A Defensive Stat Worth a Damn? Introducing Brian Burke's WPA+
There is likely to be a strong correlation between a defender's visible positive impact and his overall net impact. In other words, we should expect better defenders to tend to have both more positive plays and fewer negative plays. This is because of the symmetric nature of the distribution of human performance.
This pretty squarely summarizes the Brian Russell debate. Russell was supposedly contributing in the shadows, but whenever the play neared him, was weakened like Kryptonite by the panning camera. For two years, some would insist that though Russell looked like a muggle on plays he was involved in, he somehow tied together the Seahawks secondary like a pissed-on rug. Well, I didn't buy it and neither apparently did the league.
Brian Burke constructs a very solid case for WPA+ as a measure of individual defensive performance. It's one of the first non-traditional defensive stats I find interesting -- even exciting. I tread carefully with statistics. There's more garbage than good. However, at their best, statistics provide a scope and objectivity scouting is incapable of. This might be one. I think this is one. I am excited this could be a quality statistic that actually measures defensive performance. !!!
21 comments
|
0 recs |
Do you like this story?
Comments
Yes. I went reference wacky for no reason.
Early symptom of advanced gonorrhea? Early symptom of advanced gonorrhea.
If my dad is reading this...
He’s likely drowing in tears of joy. The Big Lebowski is, by far, his favorite movie.
ET looks like he's freezing his ass off.
It seems intuitive, and I find it interesting, but I don’t really feel convinced. So, if we see a player, make a positive play, then he’s good. Alright, got that. But, I am not seeing how +WPA takes the other players out of the equation.
Ray Edwards makes a Tackle for Loss, so we can measure that in +WPA, but it doesn’t take into account that the Williamses just smashed the pocket, Jared Allen cut the LT’s throat, and the QB is walking into Edward’s arms. I suppose that would be considered the peak of the bell curve.
Basically, what it says to me, is that “Players who make plays are good. If a player makes more good plays, they are better than a player who makes less good plays.”
If for nothing else, I like it because it included the greatest picture of BRuss ever.

Or when you have a Revis, or an Asomugha type player.
How would you measure the +WPA for the opposing QB not wanting anything to do with them, and so never throwing it to their side? It looks like they are just out there, not making any plays, when in actuality they’ve negated an entire portion of the field.
Opposing quarterbacks targeted Revis. He recorded 31 passes defended and 6 interceptions in 2009.
I’ve always been a little wary of heaping praise on Asomugha. Maybe he’s shutting down his receiver. Maybe his teammates suck so bad other receivers are wide open. Whatever the case, Oakland was awful defending number one and number two wide receivers, so something went wrong. It’s in my nature to praise good play I see and not assume if I don’t see a corner that he’s “shutting down” his man. The ultimate shut down corner is Deion Sanders, and Sanders had two or more interceptions in every season he played in. I can’t imagine a corner so good he’s never challenged.
That's a fair response.
I think I may be letting my natural skepticism and distrust of statistics cloud things here.
It doesn't take other players out of the equation
But neither does YPA or YPC or tackles or sacks. It gives us a measure of a player’s total good plays and through the good plays a sound measure of their total impact. So it’s closer to ERA than FIP, but ERA is a lot better than what we have.
Hmm, yeah, I can see that.
In that light, I went back and reread it. I’ll give it the benefit of the doubt, because it is intuitive, and I’m excited to see where it goes. It may need some work (ignoring costly plays), but like all science, it’s just error kept up to date.
I wonder if you could make/quantify a +WPA profile for every individual player? Now that would be useful.
There is no point to mentioning that there are plenty of obstacles, as BB already knows that.
So I’ll wait and see where it goes. Right now I think it looks like an interesting direction, but one that will be unkind to certain “non-impact” positions. Kind of like how football outsides struggles to quantify the individual offensive lineman’s impact.
But a stat is only limited to the creativity of the stat-creator. If he breaks enough of the obstacles, or even if he doesn’t, this looks like a good line of research.
WPA sound logistic to me.
It’s a number of different measurable factors determining the probability of success. The sum of logistic probabilities can be described by the binomial distribution. Neither of these distributions have as strict limitations on sample size and equality of variance as does the normal distribution. I think if WPA sum were tested against a binomial, you’d be better able to measure the contributions of situational pass rushers or nickle backs who aren’t in as many downs and it should better account for players that actually do vary wildly in their performance.
I'm having trouble with the assumptions.
The key assumption here is stated: “Unlike a company, however, (as Wall Street sadly learned a couple years ago) a player’s individual performance from play to play almost certainly follows a normal distribution.”
Why is that the case? Some players make the occasional brilliant play and the occasional terrible play; others are more consistent. Some defensive players, as we all know, occasionally take downs off entirely. And even if all do follow a normal distribution, there are other characteristics of the bell curve that we don’t know (does everyone have the same standard deviation?).
The article tries to explain that away with this paragraph:
“Not every defender would have the same normal profile. As I mentioned above, there are the ‘gamblers,’ players who shoot gaps when they should be reading the play or cornerbacks who jump pass routes when they should stay in position. Certainly, +WPA and +EPA would be biased in favor of these types of players. But if their gambles were really hurting a team, I doubt they’d be given much slack and playing time by their coaches. Only ‘winning’ gamblers, who are taking smarter risks, would tend to survive long in the NFL.”
A rational actor model for football, brought to you by the Cato Institute? I have a hard time swallowing this. The best and most consistent players do not always start (but enough about Brian Russell already). Coaches are sometimes impressed by big plays or by, oh I don’t know, grittiness. We know this.
I don’t want to be altogether dismissive of this, but I’m having a tough time accepting the premises.
by Suburban Shocker on May 4, 2010 8:13 AM PDT reply actions
Yeah
I get a little suspicious of getting a common sense type explanation of the distribution instead of the results of a goodness of fit test.
by BurtonOerney on May 5, 2010 12:01 PM PDT up reply actions
Doesn't WPA depend heavily on game situation?
A sack on 3rd and short with the offense on the opposing teams 33 yard line in a close game in the 4th quarter would be worth far and away more than a sack in a blowout but theres little reason to think that one sack is more indicative of talent than the other.
but
WPA also measures consistency right?
http://www.youtube.com/watch?v=cZDUh9yboqI
Your culture is primitive; yet so funky!
Interpretation
It seems like WPA+ should be interpreted as a measure of the tendency of a player to be in a position to make plays. As others have discussed above, players could have high WPA+ in a given season because:
1) They are good, and hence get to the right spot to make plays consistently
2) They play with good surrounding talent, which allows them make plays because of holes/opportunities created by their teammates
3) They happened to be in the right place at the right time on a few key plays
To me, it seems like WPA+ (particularly over one season) is too noisy to quantify true talent. And if that’s the case, I don’t really see how it’s useful.
I think a good way to get at individual contributions to the team overall
would be to model an entire defense like the parts on a car. Engineers use a time-to-failure exponential model to compute the probability of failure over time. Since the car as a whole also has an expected value and variance for probability of failure at any given time, you can give weight to the different parts and then compare the part’s performance to the expectation for the whole car.
To model defenses, individual plays could be substituted for time units. Failure could be missed assignments, missed tackles, penalties, etc. You could also look at the effect of groups of players on the probability of a failure. The exponential could be expanded to a gamma distribution so that you could look at the probability of any number of failures over a specific number of plays.

by 































