Football's Biggest Junk Stat
by BrianL on Feb 9, 2011 11:35 AM PST
Email
PrintIf you've been in comment threads for any length of time, there's a good chance you've seen me pull my hair out over the use of QB Rating as an evaluation metric. The answer I've probably given you is something along the lines of "QB Rating is football's version of WHIP*. It's a junk stat."
*If you want to see good baseball pitching stats, head to Fangraphs and read up on tRA and FIP.
Let's start off by taking a look at the formula behind one of sports' worst commonly accepted statistics.
a = (((Comp/Att) * 100) -30) / 20
b = ((TDs/Att) * 100) / 5
c = (9.5 - ((Int/Att) * 100)) / 4
d = ((Yards/Att) - 3) / 4
a, b, c and d cannot be greater than 2.375 or less than zero.
QB Rating = (a + b + c + d) / .06
What we have here is a needlessly convoluted mess that doesn't tell us anything of value. This is a statistic that overvalues completion percentage and doesn't put enough value to actual yardage. Let's do a demonstration.
QB #1 goes 35 for 40 and Mark Sanchezes his way to 125 yards, passing for one touchdown and zero interceptions.
QB #2 goes 20 for 40, good for 300 yards. He also had one touchdown and zero interceptions.
Plugging these numbers into PrimeComputing's QB Rating Calculator, we return an 88.02 rating value for QB #1. QB #2, on the other hand, scores a rating of 83.33.
Just what has happened here? The first quarterback in this little exercise has been handsomely rewarded for checking down and hitting his outlet receivers. The second quarterback has taken a beating for missing on half of his attempts.
I suspect you're beginning to see the issue now.
QB Rating adores players who put up high completion percentages. The problem with this is that it largely ignores the context of those passes. Why value a quarterback so highly for completing a boatload of passes when that's done next to nothing to move the offense forward? Speaking strictly in terms of moving the ball*, you should want the performance our second hypothetical quarterback put up.
*And I understand that an argument could be made that a higher completion percentage is valuable because it helps keep the clock running. Overall, I think being able to put up yardage wins out.
This only scratches the surface as to why QB Rating is such a flawed tool. I haven't even discussed that it was designed to correlate to quarterbacks in the 70s or how it ignores the context of the game situation. Is an 80-yard drive and a touchdown pass from Seneca Wallace really all that valuable when you're down by twenty with two minutes left in the game? QB Rating doesn't care.
Edit: Something I left off that Nate Dogg points out, QB Rating gives the quarterback full credit for touchdowns. Is it really fair to do that when someone (say, Justin Forsett) gets the ball in the backfield and jukes the entire defense out of their cleats for 20+ yards on the way to the endzone?
I don't want the takeaway here to be that complex statistics are bad. There are other measures that (while not perfect) do a better job valuing player performance. This stat, on the other hand, is mostly noise.
63 comments
|
1 recs |
Do you like this story?
Comments
F*$^ a QB rating like you said...
Its all about moving the team down the field. Playing QB is so much more about being a winner in a game than on a stat sheet. Most successful QB’s have good great ratings but no one cares about what you threw for or how long you didn’t turn the ball over if you lost. Tom Brady crazy good QB a million passes without a pick, knocked out by the Jets. Drew Brees awesome QB 1000/1000 passes complete, Defense ran over by “where the wild things are” out in the first round. All that being said the best QB in football right now is Aaron Rodgers because he is the superbowl champion and thats what defines you when your career is over, with the exception of Marino.
"I was 11 or 12 and Quinton was like 30. He was the only dude on the Pop Warner team who had a full mustache and a beard. And he used to drive to Pop Warner games. You're not supposed to be doing that. It was crazy." -Marshawn Lynch
By this logic the best QB of 2000 was
Trent Dilfer?
sub-60% completion percentage, 1500yards, 12TD, 11Int, 4.5 ANY/A.
And how exactly can we make a metric of “being a winner” when attempting to evaluate QB talent in the draft?
"The" Superbowl champion...
All that being said the best QB in football right now is Aaron Rodgers because he is the superbowl champion
I’m not really sure this is the proper measure of a quarterback, unless you define “best” as “played on a Superbowl-winning team”.
I would think a more useful definition of “best” quarterback is something like “gives the team the best chance at scoring points when the offense is on the field.” The problem with your line of reasoning is that it gives the quarterback all the credit for a win, when that simply cannot be the case. Quarterbacks don’t play defense, so around half of the game is utterly out of their hands. Even the best quarterback won’t be very good if he has a defensive player in his face as soon as the snap gets off, so the line is pretty important, and of course the quarterback usually doesn’t have much to do on a running play. Threading a pass between a couple defenders into the hands of your receiver, in stride, is a function of the quarterback, but getting yards after that catch is a function (largely) of the receiver.
The real issue is that, for whatever reason, the record of a team is tacked onto a quarterback; but that does not reflect reality. It’s rather like a pitcher’s record: next to useless. People know this to some degree, I think (complaining about drops, a defense giving up, etc), but the fact that a team’s record is carried around by one player really does seem to skew how people view players’ contributions.
your right there are so many other factors but right now I would want Rodgers, Brady, or Vick...
those are the best in my opinion.
"I was 11 or 12 and Quinton was like 30. He was the only dude on the Pop Warner team who had a full mustache and a beard. And he used to drive to Pop Warner games. You're not supposed to be doing that. It was crazy." -Marshawn Lynch
Another huge issue with QB rating is that it gives full credit for touchdowns to the quarterback.
A 20 yard touchdown pass on a drive that started from your own 15 is equal to a dump off to the full back for a touchdown after your defense forced a turnover on the one. Overvaluing completion%, undervaluing yards per attempt, giving no context to touchdowns and ignoring sacks makes QB rating a far inferior stat to things like AY/A, ANY/A and DVOA or DYAR (none of which are perfect themselves).
Also an enormous problem.
Why should Charlie/Matt/Warm Body at QB get so much credit for Justin Forsett receiving the ball in the backfield and going to town on the secondary?
by BrianL on Feb 9, 2011 12:49 PM PST up reply actions 1 recs
and what happened to Force getting the ball in the backfield and going to town on the secondary...
Hope we see more of that in 2011/12.
"I was 11 or 12 and Quinton was like 30. He was the only dude on the Pop Warner team who had a full mustache and a beard. And he used to drive to Pop Warner games. You're not supposed to be doing that. It was crazy." -Marshawn Lynch
yeah Brian i get that but that screenpass to Forsett is what i'm talking about.
If Force even gets one decent block on a checkdown or Screen he’s going for good yardage and I can’t even remember seeing that play called once. Guess thats part of the reason Bates is finished with Seattle. Marshawn got a will get majority of the run plays but where was the two back set with Forsett looking to get open and he can block too, under utilized i think.
"I was 11 or 12 and Quinton was like 30. He was the only dude on the Pop Warner team who had a full mustache and a beard. And he used to drive to Pop Warner games. You're not supposed to be doing that. It was crazy." -Marshawn Lynch
I think Brian meant any passes to the flat/backfield
were received by Lynch not Forsett. But yes, Force screens are pretty awesome in general.
Go Pack?
QB rating is awful, just awful
However, as a baseball lifer, I do think WHIP holds value.
It’s definitely not a “defining” stat, but it certainly isn’t junk. The basic idea is the fewer base runners, the fewer opportunities the opposing team has to score. And please, I am not trying to start a debate regarding WHIP, I just don’t think it’s “junk.” I could get into pitch counts, efficiency, etc, but I won’t.
QB rating on the otherhand, is just terrible. It’s goofier than clown poo.
All WHIP can do (and do well) is tell you what happened, just like ERA.
It has little to no predictive value, and you can’t really apply any of it to the pitcher, as it doesn’t separate what he did from what the rest of the team did. So yes, it has its place, and has some value in that place. But because nobody uses it for the only purpose for which it has value, it ends up having little practical value.
"Retarded isn't a race." -Thingray
by Matt Erickson on Feb 9, 2011 6:01 PM PST up reply actions
If you wanted to know how many baserunners (regardless of circumstance or what kind of baserunners)
reached base per inning for a particular inning, WHIP can tell you that. So by definition it has SOME value. However, that knowledge isn’t very useful, or at least isn’t useful past any sort of surface level. That’s why I said that it ends up having little practical value. I was just trying to find the common ground between BrianL and MTJ.
"Retarded isn't a race." -Thingray
by Matt Erickson on Feb 9, 2011 6:12 PM PST up reply actions
It's not completely without value...
In general, I’m willing to bet that a QB with a season rating of 105.0 is probably better than one with a 70.0.
But that’s about as far as it can be taken, heh.
http://17power.blogspot.com
Quarterbacks can luck entire entire seasons where QB rating makes them look better than they really are.
Derek Anderson comes to mind.
Yeah, I think there is some truth to that.
Just like id be willing to bet that a pitcher with a 2.80 era is better than one with a 4.55 era, but there are just better stats to use to measure how good/effective a player is.
This is the point.
We can figure out pretty easily that Miguel Batista and his 5ish ERA is worse than Felix and his 2ish ERA. But we didn’t need ERA to tell us that.
Tom Brady might run a 111 QBR for the season, but that number doesn’t make him better than Jimmy Clausen. Being Tom Brady does.
ERA (and you can sub in QB rating here, that’s fine) might even outrright lie to us at times, whereas AY/A is much, much less likely to do so.
Just picture Randichitupu.
Maybe
This is a world of relativity. Your first complaint had merit in a sense. But the value of high completion percentage is not just to keep the clock mving, but rather to keep the CHAINS moving. For instance, a RB who gains 4 yards every attempt is better than one who loses -1 every time except for 1 in 20 runs hits a home run for 90 yards, even if they end up with the same yardage and total runs at the end of the game. Because a 4 yard run puts in ahead of the chains on 2nd, and on 3rd and short on 3rd, etc., while a -1 kills your drive right off the bat.
But i digress, the first complaint has some merit, a high completion percentage is very important, but so are yards. But then again, what good are yards if they don’t lead to TDs? What good is driving 90 yards on 1 play to turn it over? That’s why more value is put on TDs and INTs.
But the last part of your complaint is ridiculous. It doesn’t take into consideration circumstance? Well of course not, it is a STAT. Statistics are not people, they don’t tend to take into consideration circumstance. Not to mention, in this world of relativity what is your better stat for QBs?
Putting up yards is perhaps the best correlation to scoring points.
Keep in mind that it’s not just the QB who is responsible for touchdowns going on the board. What he can do, however, is efficiently move the ball through the air to put the offense in the redzone.
As for better stats, AY/A, ANY/A, DVOA, and DYAR. They’re not perfect, but there’s much less noise in there than QB Rating. Furthermore, stuff like DVOA does attempt to take context into consideration.
Also keep in mind that having a high completion percentage doesn't necessarily mean you're moving the chains.
If your quarterback is Captain Checkdown and doesn’t attempt much in the way of midrange throws, you’re probably not going to be collecting much in the way of positive yardage. You can hit a solid number of passes and fail to keep the offense moving.
To QB Rating, that doesn’t matter much. It weights completion percentage so highly that it will override the lack of yardage.
Sam Bradford.
"Retarded isn't a race." -Thingray
by Matt Erickson on Feb 9, 2011 6:06 PM PST up reply actions 1 recs
Gotta disagree here...
But the value of high completion percentage is not just to keep the clock mving, but rather to keep the CHAINS moving.
An enormous problem with QB rating is that it conflates completion percentage with “moving the chains.” But they are not the same. The guy who takes the 8 yard check down on 3rd and 16 isn’t moving the chains. QB rating treats him AS IF he is. That’s misleading information, which in most respects is worse than no information.
In this way football is fundamentally at odds with parametric stats in a way that baseball is not. You basically get at this when you ask “what good are yards if they don’t lead to TDs?” I would amend it to ask “what good are yards if they don’t lead to first downs and points?”
I really like the intuition behind “success rate and stuffed rates” for running backs where you get a “1” or “0” for the outcome of a given running play. If I were creating a QB stat from scratch the underlying intuition would go like so.
An offensive possession is successful if and only if it results in (ascending order of importance):
1. withholding possession from the opposing offense (i.e., controlling the clock with a lead) — minimal points tied to the number of plays per drive
2. field position for kicking or punting — points scaled to the length of a drive
3. touchdowns — bonus
I might consider passing plays exclusively (or assign fractional points for running plays to #1). Just typing out loud. I haven’t thought this through.
"Those who fear disorder more than injustice inevitably produce more of both." -- Rev. William Coffin
by dcrockett17 on Feb 9, 2011 1:37 PM PST up reply actions 2 recs
answers
No one said that completion % is the exact same as getting first downs, but it is a great indicator. If you want a better indicator, then you should probably add a yards per attempt component. This doesn’t change the fact that everything being equal, a higher completion % is better and indicates more first downs than a lower completion %.
I don’t quite get your second point, could you clarify? When i say “tds” it was meant to represent points. It was implicit in the statement.
Regarding your attempt at creating your own QB stat, it doesn’t seem like a bad idea, but things like this need to be tried out and applied. Also, they need to be readily available for all QBs over all time periods, if they are deemed a worth stat. Right now, we don’t have much for QBs.
I wonder if we shouldn't be looking at more non-parametric stats
something similar to what FO does for running backs with success rates. For that matter, it’s amazing to me that adjusted yards per attempt isn’t more commonly used.
"Those who fear disorder more than injustice inevitably produce more of both." -- Rev. William Coffin
Historically, it was designed to try and create a baseline for quarterbacks.
In the seventies.
Perhaps then it made a vague amount of sense, but it makes even less now.
Was a rationale offered for it at the time?
e.g., regressing the various inputs against things like team points scored or wins or something? Or is the formula really as arbitrary as it’s made out to be?
by Suburban Shocker on Feb 10, 2011 7:27 AM PST up reply actions
Back then it was WAY harder to complete a pass
That’s why completion percentage is so highly valued.
by Jackrabbit5683 on Feb 10, 2011 9:02 AM PST up reply actions
Which is fine and good
But did they have a justification for using the constants they used, other than waving a finger in the air? If there was something, based on the data of the day, comparing the relative values of completions and yardage that factored into QB rating, I’d be happy — and I’d also say let’s just tweak it with the data of the last five or ten years and come up with a new, improved QB rating. It would then be an okay stat, subject to the usual caveats that it still doesn’t measure everything and that football is a team sport. But if it was all just made-up, then yeah, let’s junk the whole thing.
by Suburban Shocker on Feb 10, 2011 11:49 AM PST up reply actions
I think they're just to properly scale the stat
which is odd because maxing out at 158.3 doesn’t seem like a very intuitive scale.
I'm pretty sure it's set up like IQ tests
100 is average in IQ tests. 100 is supposed to be a “good” QB rating. The constants were construed so as to make the good QB stats of the day fit the 100 rating.
by Jackrabbit5683 on Feb 10, 2011 6:16 PM PST up reply actions
Heh, I used to have it memorized.
My brother and I would track QB rating in Madden 93 and Madden 96.
The user formerly known as Phildopip
by Phil Hatzenbuehler on Feb 9, 2011 1:31 PM PST up reply actions
Considering the sometimes strong resistance to complex statistics
The fact that QB rating got to be so popular is a little amazing. Only a savant could possibly even think of just approximating it in their head. Even when you do know how it’s calculated, it looks likes one of your algebra-class inspired nightmares. It’s capped at 158.3, adding to what already seems like an arbitrary stat.
It’s just so…hokey.
by Gihyou on Feb 9, 2011 1:28 PM PST reply actions 4 recs
I think I read it for a few years on the backs of football cards as a number that was completely made up out of thin air that tried to tell me how good everyone thought the QB was
I was actually pretty close, as it turned out.
by Gihyou on Feb 9, 2011 1:35 PM PST up reply actions 4 recs
its always been stupid.
I want Rodgers, Brady, Vick, Ryan etc. as my QB if i’m a coach no matter what type of QB rating they have for I know those guys give me the best shot to win a game.
"I was 11 or 12 and Quinton was like 30. He was the only dude on the Pop Warner team who had a full mustache and a beard. And he used to drive to Pop Warner games. You're not supposed to be doing that. It was crazy." -Marshawn Lynch
Brady ran a 111 rating this season!
Damn AP classes.
Just picture Randichitupu.
by fiftyone on Feb 9, 2011 8:08 PM PST via mobile up reply actions
It's the maximum that gets me
and the fact that it maxes out at such a weird number just rubs it in. You shouldn’t be able to max out a QB stat without throwing a TD on every attempt. That right there tells you it’s bogus; the rest is the details of why and how.
by The Ancient Mariner on Feb 9, 2011 3:07 PM PST up reply actions 1 recs
As in all things, context is key.
QB rating has to be the worst stat of all time, even “non advanced” stats like Yards per Attempt blow it out of the water. When somebody quotes QB rating, I stop reading them.
Football stats are so murky, I think the best way to go about it is to use several just like (BA/OBP/SLG). For a QB that might be (YPA, Air Yards/A, TD/INT).
The only QB stat worse the QB rating is win-loss percentage
Great piece, BrianL, really informative.
by Greetings from the Lord Humongous! on Feb 9, 2011 3:43 PM PST reply actions
Interceptions/attempt carry the most weight in that formula.
If you take this year’s league averages for completion percent, TDs/attempt, INTs/attempt and yards/attempt, the most important components to QB rating, (in order, before dividing by .06), are INT/Att (1.625), Completion%(1.5), Yards/Att(1) and TD/Att(.8).
I think it's all in the context
The problem is that it’s the only advanced stat presented to the general public and so to the uninformed it’s taken as the be-all representation of a QB. It’s indictive of decision-making, the intent is to highlight the QB who makes less mistakes. The majority of fans don’t want to be bombarded with advanced statistics and QB rating’s an easy one to throw out there and explain to casual fans. I agree that it’s flawed but how many people watchin games on TV want to listen to some long winded explanation of advanced stat metrics. QB rating is a stat that can be thrown out there in a broadcast without needing to be explained and therefore chiewing up airtime, it’s prolific use as “THE QB STAT” is directly related to television.
by CMoney87 on Feb 9, 2011 7:06 PM PST via mobile reply actions
Another thought
It also seems to me that when QB rating is presented it’s usually alongside other stats like comp/att, TDs, YDs, INTs and sometimes even first downs. I think it’s a stat meant to be presented amongst others that help understand the context, though it’s often misused or misrepresented without that context.
by CMoney87 on Feb 9, 2011 7:25 PM PST via mobile up reply actions
I'm pretty sure that pre-game shows can fit in 5 minutes of a "Stats 101" section in between the other 55 minutes of pure fluff.
Is that the light at the end of the tunnel, or the headlights of an oncoming train?
Thing is
I bet that networks feel that fans prefer that fluff to explanations of advanced statistics. The dating life of insert hunky quarterback, and the dramatic personal ties of far extended branches of whatever coaching tree is the kind of stuff that the polled audiences would rather see. We live in an age where “Celebrities, Entertainment, Gossip” is a section on the front pages of MSNBC, everybody wants to be in everyone elses business and most people don’t care about the intricacies of the game. I heard somewhere in the media run up to the Super Bowl that something like 42% of a polled audience who would consider themselves NFL fans didn’t even know there was labor unrest in the league and that there was no CBA .
Great post!
The version of The Hidden Game of Football that came out in the 90s talks about the evolution of QB rating. It’s an interesting read, and while it comes to the conclusion that you do: that QB rating is pretty damn awful (in fact, that’s the book that introduced, so far as I know, adjusted net yards per attempt) but it gets into exactly why, at least from another perspective. Football, it seems, went through several iterations of how best to rate a QB, from straight yards passing (bad because a guy could just throw a lot or have a good receiver) to yards per pass completed (also not great because it doesn’t take how many of those passes were successful into account) to yards per pass attempted (still not perfect) to completion percentage (a step backwards in my opinion and even the book points out that Milt Plum was considered the best QB of his time thanks to being able to dump the ball off to Jim Brown a lot), and finally QB rating. QB rating came out in the late 60s or early 70s and has remained essentially unchanged since that period.
So… around 30 years of innovation followed by 40 years of stagnation. It’s not hard to see why QB rating doesn’t work well: it’s based on an outdated understanding of the game.
Colts game anyone?
I think part of what you’re saying that may not have been articulated but I think is major: QB Rating values yards gained by short passes (easy for QBs to do) the exact same as deep passes (much harder to do in general). That’s what I take away from this anyway.
I definitely want a stat that more accurately depicts how well a QB played – I want to look at the two starters’ QBR and be able to guess pretty well which team won just from that.
Go Pack?
Erm I realize my comment was non sequitur from
my subject line…plus there’s a picture already…blegh…but for anyone who doesn’t understand its use, almost all of our yardage in that game was achieved in the 4th quarter when Indy was playing their 2nd-stringers against our 1st-string offense.
Go Pack?
This makes me wish
That we had someone like Graham and Matthew at LL to come up with a good stat for measuring a QB like they did with pitching and tRA.
I hate to advocate drugs, alcohol, violence, or insanity to anyone, but they've always worked for me.
i would think era or rbis to be a more blatant junk stat comparison
But that is neither here nor there…your points concerning qb rating were well stated.
Thanks for the piece Brian.
"C'mon Joe, GET WET!" - Jon Miller
by Omerta on Feb 10, 2011 1:23 AM PST via mobile reply actions
At least RBI has the courtesy of being stupid and simple to calculate.
QB Rating is stupid, yet has such a ludicrously complicated formula it’s impossible to take seriously.
Is that the light at the end of the tunnel, or the headlights of an oncoming train?
I don't want to start an argument, but I think QB rating has some merit
When you’re watching the ticker at the bottom of the game and you see QB ratings over 110 or so, you know the QB probably had a pretty good day (providing he threw more than 3 passes). If you see one below 60 or so, you know he probably had a pretty bad outing. Another thing about QB rating is that it’s relatively easy to find as opposed to ANY/A, DVOA, etc. Also, advanced stats are sometimes a little too complex for the casual fan.
I’m not saying I prefer QB rating to advanced stats, but I think it does have its place. That being said, I agree that you can’t use it to make accurate QB comparisons and that you instead need to use some kind of advanced stat.
It has a guesstimate/quick glance merit
It has no merit in indepth analysis. I think that’s more BrianL’s point, so what you’re saying is true and probably not in contradiction to what he’s saying.
by Thomas Beekers on Feb 10, 2011 9:31 AM PST up reply actions
I think you're giving the casual fan too little credit.
People certainly seem willing to go by a stat that they couldn’t even begin to calculate. I’m sure they could utilize the advanced stats out there if they wanted to.
So it's merit is that it's used by ESPN.
AY/A could be used in those situations just as easily and give you a better idea of whats going on.
Yes
ProFootballReference – Adjusted Yards per attempt (AY/A) and Adjusted Net Yards per Attemp (ANY/A, includes sacks).
Football Outsiders – DVOA (measures performance above average and is weighted for opponent) and DYAR (a counting stat based off of DVOA)
Advanced NFL Stats – Win Probability Added per Game (WPA/G) and Expected Points Added per Play (EPA/P).
I think you just nailed it...
Nobody wants to hear all those acronyms and QB Rating took the cool name for their metric.
by CMoney87 on Feb 10, 2011 1:27 PM PST via mobile up reply actions
That's a pretty damned good point.
The abbreviations for those stats all come out sounding awful and awkward. That, plus they all kind of sound the same…
I suppose if I spent more time looking at them, I’d be able to seperate them easier.
Comments For This Post Are Closed
Secondary Sidebar
User Tools
FanPosts
Community blog posts and discussion.
Recommended FanPosts
Recent FanPosts
In Association With
Shutdown Corner - NFL - Yahoo! Sports
- Former high-school star Brian Banks does serious time on trumped-up charge, now hopes for an NFL shot
- Former ‘idiot kicker’ Mike Vanderjagt in trouble after accosting junior-high heckler
- Adrian Peterson blows teammates away in footraces; appears to be on the mend
- UPDATE: Charlie Sheen didn’t buy Lawrence Taylor’s Super Bowl ring
- Alex Smith gets blown up by Jon Beason over Cam Newton comments
