FanPost

QBR: not a Russian conspiracy against Russell Wilson but actually a pretty neat tool

Hi, I'm Ben and I like #analytics. Since I've been "fighting" with people on twitter about QBR for the past week, I figured I might as well make a post to show why I like it.

Here are 3 basic requirements that I think a good measure of QB play should contain:

  1. Context-dependent. An 8 yard pass on 1st and 10 is substantially more valuable than an 8 yard pass on 3rd and 10. An interception on 1st down is more harmful than an interception on 3rd down.
  2. It should be a combined measure that includes rushing and passing plays. Many QBs add value with their legs.
  3. Adjusted for opponent. Playing in the NFC South is different than the NFC West.

Commonly-used advanced stats don't meet these 3 requirements:

  • Pure passing stats (passer rating, Y/A, ANY/A, etc) fail all 3.
  • TAY/P and its variants fail #1 and #3.
  • DVOA comes close, but fails #2: DVOA is calculated separately for passing and rushing and there's no way to combine them (I verified this with Vince Verhei of FO). It's also less transparent than QBR. I would have no idea about how to even begin calculating DVOA myself, while for QBR, as I show below, does not have this problem.

QBR meets all 3 because it is built on expected points added (EPA) and then adjusted for opponent.

What is expected points added?

Expected points is the expected value of the next score. See PFR's writeup here:

"Our crack staff of statisticians have used a decade's worth of play-by-play data to develop a formula that estimates the expected number of points given a combination of down, distance, and yard line."


Context-dependent? Check. Passing and rushing? Check.

So what is the "formula" for QBR?

  1. Expected points added = [expected points after the play] - [expected points before the play]
  2. QBR = (total expected points added) / (total plays)

That's (almost) it. I can come close to replicating ESPN's QBR list from the 2015 using PFR box scores. The correlation between my measure (constructed from PFR box scores) and ESPN's is 0.93, which is very high.

Here are the 2015 rankings for my calculation's vs. QBR. Again, very close. When you think of QBR, don't think "black box", think "EPA with minor modifications".

qCu55xI.0.png


Why don't my rankings exactly match ESPN's QBR? I don't take into consideration the following:

  • How many yards came after the catch (a QB gets credit for how many yards after the catch are "expected" given target location, but no more than that)
  • Which plays came in garbage time (these plays are given less weight in QBR)
  • Which plays came while the QB was under pressure
  • Which plays contained dropped catches or INTs

Since without using any of this information I can come close to duplicating QBR, none of these play vital factors in QBR.

Let's look at some objections to QBR (these are taken from actual FG comments) (sigh)

Objection 1: QBR is a black box

I can come close to duplicating ESPN's rankings by following their methodology, as shown above. It's simply EPA with some minor modifications and thus hardly a black box.

Objection 2: QBR isn't predictive of anything

It's very good at out of sample predictions, especially compared to measures like ANY/A and passer rating.

Objection 3: QBR is an ESPN conspiracy to drive clicks

*bangs head against wall*

To close, just to give some examples of what QBR is doing, here are the 3 highest and lowest EPA plays from Wilson in the Seahawks-Jets game.

Highest

  1. "2Q/3:54/1-10","Russell Wilson pass complete deep left to Tanner McEvoy for 42 yards touchdown", (EPA: 4.21)
  2. "2Q/13:16/2-8","Russell Wilson pass complete deep middle to Doug Baldwin for 38 yards (tackle by Calvin Pryor). Penalty on Calvin Pryor: Unnecessary Roughness 5 yards", (EPA: 4.08)
  3. "4Q/13:06/1-10","Russell Wilson pass complete deep left to Paul Richardson for 27 yards (tackle by Calvin Pryor)", (EPA: 2.45)

Lowest:

  1. "4Q/4:55/3-11","Russell Wilson sacked by Buster Skrine for -14 yards", (EPA: -1.9)
  2. "3Q/1:10/3-3","Russell Wilson pass incomplete short left intended for Tyler Lockett", (EPA: -1.79)
  3. "1Q/4:47/1-10","Russell Wilson sacked by Leonard Williams for -7 yards", (EPA: -1.49)

Russian conspiracies and Russell Wilson

Since the title of my post argues that QBR isn't a Russian conspiracy against Russell Wilson (this was a (hopefully) joking comment in this QBR thread), here are his opponent-adjusted ranks in QBR by season:

  1. 2012: #3
  2. 2013: #6
  3. 2014: #6
  4. 2015: #4
  5. 2016: #20

QBR doesn't hate Russell Wilson. It loves him because he's a great quarterback! It just notes that he's providing zero value with his legs this year, because he has been injured. In fact, while he's #12 in the league in pass EPA despite already having his bye week, he's literally last in the league in rush EPA.