Pro Football Prospectus 2008 Reviewed
For the ardent NFL fan, or even the wannabe ardent NFL fan, Pro Football Prospectus 2008 is an indispensable compendium of information and analysis. Its combination of intelligent analysis and efficient, approachable writing is unmatched in the crowded field of magazine rack glossies and big-name, big-book bossies. For many, that's enough review to decide, and I won't bat an eye if you scamper off to your local bookseller to buy. But Football Outsiders and Pro Football Prospectus long ago distanced themselves from their peers, and as the industry leader in intelligent analysis, they now must face a somewhat daunting task. They must compete with themselves. And in the competition to be smarter, more entertaining and more accurate, PFP 2008 is not the great leap forward one might want.
PFP 2008 is subdivided into 3 main parts: team specific essays and unit reports, NFL related research and essays and fantasy related projections. That's not exactly how it's organized, but it's an easy way to classify its disparate sections.
The team specific reports, on the whole, are succinct, readable and do a good job of providing both an effective recapitulation of last season and an accurate look at next season. They may come off as prosaic at times, but one must consider the breath and diversity of their potential readership. The works of Mike Tanier and Doug Farrar stand out.
Tanier is an entertaining and engaging writer. Further, he's a real football fan with an obvious passion for the game. Working long form, and with no doubt more time to revise, the sometimes precious cleverness displayed in his Football Outsiders writing becomes instead energized prose, with a strong voice that is informative and independently interesting. Special praise should be given to his opening lines.
Farrar's work is polished and efficient. It has the easy feeling of a Pixar film. You find yourself deeper, more engaged, more informed and farther along than you expect. Compared to Tanier, his writing is a little more subtle, journalistic, but he excels at creating a strong thesis and holding it throughout the essay.
While not every team essay is a standout, most are well written and comprehensive. Two essays stand out for the wrong reasons. Kevin Lynch's work on the 49ers is busy, lacks cohesion, seems incapable or unwilling to take a realistic look at the franchise and really thuds in its closing paragraph. Mike Nolan is seemingly the focus, but from him we're sort of taken everywhere and nowhere. We're told that hiring offensive coordinator Jim Hostler was a mistake, but also told that San Francisco had "few options" and that "Hostler's game plans were generally good and he called a game similar to Turner". Hostler's problem? "[H]e lacked Turner's authority and gravitas..." Further frustrating is the assertion that Nolan hiring Mike Martz was a "brilliant move" and that "no one can resuscitate an offense better than Mike Martz." The tone and logic of the piece, a piece about the worst team in football, down to the curiously late mention of the 49ers tampering charge, seems better home in a media guide or on sf49ers.com than in a non-partisan, analytical publication like Pro Football Prospectus.
The other essay that runs astray does less so and for less damnable reasons, but must be pointed out nevertheless. Ben Riley's essay on the Seattle Seahawks is an otherwise light, well-written account of Seattle's 2007 and prospects for 2008. But his evaluation of Tim Ruskell's offensive drafting leaves much to be desired. The thrust of his argument is a "hypothetical starting lineup" for Ruskell acquired talent, offense and defense. Not surprisingly, the defense shines. Unfortunately, it ignores one very essential consideration and relies on some very suspect arguments. The essential consideration is simple: Ruskell inherited an above average offense but a well below average defense. It's little wonder then, with no need at quarterback, either tackle spot or running back, that the talent acquired on offense is going to look much worse than the talent acquired on defense. Can we really gripe that Ray Willis is "Never seen" when he's playing behind Walter Jones and Sean Locklear? Is it right to slag third string quarterbacks? Should Ruskell have anticipated the injuries of Deion Branch and Alvin Pearman? It's telling Mansfield Wrotto is knocked as "equally invisible" as Willis, but then appears in a later chapter on the "Top 25 Prospects". Needless to say, whatever the truth of Ruskell's talent evaluation abilities, Riley's argument falls flat.
The NFL related research and essays is the best portion of the book. Bill Barnwell's look at how weight and 40 times combined provide a better idea of a rusher's potential, Five Seconds Can Be a Lifetime, should be credited for admitting its limitations. Still, given the relatively low correlation between Speed Score and DYAR (.37) and the fact that running backs are pre-selected for both qualities (think how many times you heard Jonathan Stewart's weight and 40 time mentioned together this past draft) I can't help but wonder if Speed Score offers anything truly groundbreaking. It may reveal a potential bust or steal and do so with some level of accuracy.
Will Carroll's look at painkillers, Painkillers: The Dirty Secret, left me cold. Injury and debilitation from sacrificing one's body to the NFL sounds concerning enough, but as a blue collar guy, the son of a mechanic, I have a tough time summoning sympathy for rich athletes who must play with pain or injury. The blue collar attitude of the NFL, its ethos of toughness and selflessness, is part of its appeal. Working with pain is an unavoidable part of a huge portion of the population's life. My father, nearing 60, has worked as an auto mechanic for 15 years on a back badly in need of surgical repair. With all due respect for the suffering of others, and I know what it's like to work through back spasms, twisted ankles, bleeding, throbbing fingers, I couldn't embrace Carroll's dirty secret. Playing hurt sucks, but football is entertainment, and football players are paid handsomely for their sacrifice.
The most interesting and perhaps confusing essay is Rob Pitzer's Wide Receivers: Size Matters. In short, we learn that 23 of the top 25 wide receivers in football, as measured by yardage per game, fit one of 4 generic body types. Those body types are squares plotted at the intersections of height and BMI. I don't know enough about this type of analysis to openly question it, it's certainly interesting, but I found no rational explanation for the phenomenon. Height and BMI really should have no bearing on player's ability to get separation, and only an incidental relationship to a player's ability to shield a defender. But sometimes the research outpaces the explanation and this is an theory to look out for. It's certainly interesting.
Touching briefly on PFP 2008's fantasy content, I have two thoughts. If you're interested in dynamic and thoughtful ranking of a player's fantasy value, I'd skip to Football Outsiders' downloadable KUBIAK spreadsheet. Though, I'm not sure any fantasy forecaster is that much better than a little research and a little common sense. As I think Tom Tango said, and I'm very loosely paraphrasing here, any projections system extensive enough will have its share of surprise hits, but that's simple probability.
The second thought is about the top 10 risers and fallers article that's become a bit of a mainstay in the seasonal prospectuses. Listing players who are old, injured, both or will see their playing time increase or decrease this season from obvious causes (new personnel, a change of role) is not particularly insightful. I would hope most people already know that Ryan Grant's rushing totals are likely to increase. He only started 7 games in 2007. Equally, I think most can figure out that Justin Fargas's rushing totals are likely to decrease. The Raiders did draft Darren McFadden. That's two examples, but you can lump in Derrick Mason, Hines Ward, Jon Kitna, Willie Parker, Anthony Gonzalez and Matt Schaub into the "no duh" category. I get that fantasy football is not particularly conducive to thoughtful analysis, but is a however-many million dollar industry that must be appealed to. Nevertheless, this list of risers and fallers is a step below what I would expect out of Athlon.
This all might seem kind of negative, but it shouldn't be thought that I'm really down on Pro Football Prospect as much as kind of disappointed. My final reflection upon reading is kind of my own dirty secret. See, when I approached PFP 2005, 2006 and 2007, I did so as a fan of football but by no means an analyst. It might be just further bastardization of an already pissed on word, but I think I approached PFP 2008 as a Seahawks analyst. Of the few things I truly know in this world, the Seahawks 2007 season is among them--paramount possibly. It is then with some perplexity that I first read Football Outsiders independent defensive statistics for 2007. Though Aaron Schatz prefaces them by saying they are not "unassailable", I couldn't help but wonder if they have any use at all. Deon Grant ranks 1st among all safeties in Run Stop Rate, but 69th in pass stop rate (Success%). Leroy Hill ranks 5th in pass stop rate. Hill made strides, but he's still very much a work in progress as a cover linebacker. On Grant, if I need to offer this, he's a much stronger cover safety, among the best in the NFL, than run stopping safety, where he had a part in some Seattle's longest runs allowed. The problem is that the stats overemphasize the plays a player makes, a very, very small sample, and put no worth into the plays a players doesn't make, the hundreds of snaps that comprise his season. The greater problem is that these stats are so far divorced from a player's actual ability they probably do more harm than worth. To think we complain about yards per carry, but you'll never find a truly awful running back (given any reasonable sample size) over 5, or any truly great running back under 3. These stats do just that, imply bad players are good and good players bad.
Unfortunately, after considering Football Outsiders defensive stats, I began to think about the rest of their numbers. The team totals are strong and reflective of that teams overall quality, and the quarterback numbers are decent, but DVOA, DPAR and DYAR (don't get me started) just do not accurately reflect the ability of an individual skill position player. Famous misses, like "[Randy Moss] A cog not a superstar" or "[Kevin Jones] 2005 rushing champ" were each built off a faulty premise. For Moss, the explanation is simple, Moss' stats, traditional and advanced, were way down after playing for the Raiders. KUBIAK, a system that does projections partially based on similarity scores (I believe) irrespective of the player's innate abilities, wrongly concluded Moss was in the midst of a decline. But Moss couldn't control the overall quality of Oakland's offense and without similarity scores that factor in Moss' incredible speed and athleticism, his comparables are not going to accurately reflect his potential. And they didn't. It sounds eminently retrograde, but the best information about Moss, about Steven Jackson or Dallas Clark is still found through careful, intelligent scouting. Scouting has been maligned thanks to the snowjob done in Moneyball, but from high school, through college and into the pros, across almost every major sport, scouting is still the premiere tool in talent evaluation. While statistical analysis has made major strides in baseball, football is nowhere near as structured or as statistically detailed. Every play in the NFL is akin to measuring a baseball player's defense, except with 11 interworking players, 11 interworking opponents, and no single, defined method of accomplishment. It's hard enough to calculate a centerfielder's ability to run in straight lines to a flyball, how far off are we from quantifying a running back's ability to accelerate, read his blockers, pick through opponents, break tackles, cutback, accelerate or decelerate midstride, and on and on.
That's why, to conclude this too long review, I think the next logical step for Football Outsiders is not a metric or formula, it's a scouting department. Game charting is nice, but attempting to quantify every happening on a football field often misses the forest for the trees. The next step is a rigorous, analytical and logical scouting department. One that doesn't boil everything down into a number stripped of meaning, but instead describes a realistic set of skills. Many publications, from ESPN to the Sporting News, claim to do it, but their "experts" seem ignorant of modern ideas of what is repeatable, the aging curve, and, frankly, reality. Whoever these unnamed and unaccountable scouts are, their reports read too much like scout speak mad libs and their conclusions too often look dubious to even a casual fan.
Pro Football Prospectus 2008 is a great book, indispensable, irreplaceable, but as a longtime reader, and nascent analyst, I do not see a lot of room for growth in the current model. DVOA is a great metric for evaluating a team, but evaluating a player is still best left for one's own eyes.
0 recs |
9 comments
Comments
Good write-up
You make some very good points about the drawbacks of the book (and I say that as a huge fan of FO and PFP).
I’m wondering why they seem pessimistic about the ‘hawks this year—one of their basic philosophies is that a team that is good on 1st and 2nd down but sucks on 3rd down is likely to improve the next year and the ‘hawks were awful at that. As bad as Julius Jones comes out in their metrics, he’s an improvement on Shaun and the D is likely to still be good… It does come out to 10 expected wins, but that seems low considering last year’s 10-6 record without a running game.
There are at least three shots taken at Charlie Frye (and I probably missed one somewhere). Poor Charlie Frye…
by Strang on Jul 15, 2008 3:43 PM PDT reply actions 0 recs
Oh, not to reply to my own comment but...
the article reviewing team medical staffs was interesting, but I felt like it punished the Seahawks’ for Branch blowing out his knee and for Tubbs being made of low grade plastic. I don’t see any way the medical staff could have predicted that Branch’s knee would pop.
They were right about the amount of injuries our receivers tend to have as a unit, though…
by Strang on Jul 15, 2008 3:46 PM PDT up reply actions 0 recs
I don't think 10.5 wins is supposed to be pessimistic.
Their win projections are always kind of conservative. I think conservative projections are common in any kind of regression analysis. The mean win projection should always take a back seat to the splits. I think it’s more telling that 48% of their trials resulted in Seattle winning 11+ games.
by John Morgan on Jul 15, 2008 3:56 PM PDT up reply actions 0 recs
Fair enough
I may just be projecting a little butthurt at it as a typically sensitive Seattle sports fan—beyond the numbers, I just didn’t feel much enthusiasm from the teams (especially compared to the Eagles, but then KUBIAK always loves Philly).
One thing that made me smile—in the Cardinals chapter when they are discussing coaching staffs, it stresses the importance of o-line coaches and how they have the biggest impact among position coaches (followed by secondary coaches).
by Strang on Jul 15, 2008 4:02 PM PDT up reply actions 0 recs
JM, really, really interesting stuff
I love reading Outsiders material, but I agree with you that the individual metrics do not always appear to reflect reality. To me, it’s simply because the statistical premises that DYAR, etc., are based on are still in their nascent (cool word!) stages, and need to be constantly refined, especially on the defensive side of the field. The same problems are in baseball as well, as you can consistently read whenever the guys at USSMariner talk about rating defense.
The actual idea surrounding the individual stats is one of value over some hypothetical replacement player, which, as a statistical premise, is already slightly flawed, especially because, as they articulate, the replacement player itself would vary based on the position and because the number is really saying that this is their value when you place the player with the exact team and coaching style that they are in. Thus, you have a fundamental hypothetical replacement player with an unchanging value that is compared to real players with unique situations and positions.
This isn’t to negate their work, but just to say that they still have a ways to go. They are still by far the best out there to date, and I really enjoy reading their work.
"Why is it every time I need to get somewhere, we get waylaid by jackassery?" - Dr. Venture
by Eegah on Jul 15, 2008 4:25 PM PDT reply actions 0 recs
Nice...and good timing...
I’ve been pouring through said Prospectus for about a week now. It is my first experience with the series and I love it. I can see your point about how the numbers don’t always tell the story. However, reading about Seattle’s linebackers, specifically, told me that Hill was a stud at stopping the run; not far off in his coverage ability, and needs to be resigned. (hello!) It seemed that the numbers for our linebackers were close to what I thought about them and have watched, which in this and other cases, including the unit analysis, validated what I thought about certain players. I bought the book to better scout for fantasy football, primarily, and will take into account you opinions of the book’s numbers. I thought the projections for individuals were mostly in line with my other research, if not seemingly more accurate. The writers captivate with informative, straight-forwardness and humor, which makes it an easy read for such a technical tome of football .
by Misfit74 on Jul 15, 2008 7:39 PM PDT reply actions 0 recs
Wait a second here...
Hi John. Ben Riley from Football Outsiders here. First off, thanks for all the kind words in support of FO and the Pro Football Prospectus. I think you know I’m an avid reader of Field Gulls and I think this is one of the best sources for Seahawks related analysis and insight on the web. Plus, you gave my brother Jimmy a prominent post here once, which literally made me spit up my beer. So, we cool.
I do want to respond to your argument, which believe it or not I did consider - and rejected - because it’s the same argument Mr. Ruskell makes. Here’s the problem: Ruskell has used almost as many high picks on offense as he has on defense. Yes, he inherited a better offense, but the rosters I created were fake, a montage of his best picks and acquisitions at every position since he took over. Looking at Chris Spencer and Deion Branch, I see two first-round picks we’d like to do over, don’t you? And you can’t just say “is it right to slag third-string quarterbacks” when they were supposed to be more than third-stringers (cough David Greene cough). But reasonable minds can disagree.
by Ben Riley on Jul 16, 2008 7:54 PM PDT reply actions 0 recs
I did not know you were an avid reader, but...
that’s gratifying.
I guess we’ll have to see how Spencer, Sims, Wrotto, Willis, Burleson and, yes, Branch contribute in the next few seasons and revisit this.
by John Morgan on Jul 17, 2008 12:12 PM PDT up reply actions 0 recs
John:
First of all, the review was a blast to read. I think that any FO staffer would take it as a compliment that you or any other reviewer would take so much time and care with this review. I already know from last year’s review that you’re part of the core audience that will dog-ear this book to Kingdom Come. And the kind comments about my writing were very much appreciated.
I wanted to reply to the review in an overall sense, but first, I think it’s important and interesting to address two things you brought up that I can relate to very well. First, the idea that as you are advancing as a football writer, you seem to see the writing of others in a different way. I agree completely, and I can’t tell you how many sportswriters in general don’t hold the same weight with me from a content perspective now that I’ve seen the elephant and have ridden it a few times. Conversely, the ones that do still make the cut are held in much higher regard. I think you’ll find that as your own voice develops over time (I don’t say that to condescend—you’re further along than most at any level), you’ll be able to interpret the voices of others with more and more expertise. The more we speak this dialect, the better we listen.
Second, the notion that FO’s scouting acumen much mix and match with the numbers? Bulls-eye. I think that in the end, that’s what’s going to be FO’s legacy. Not just the advanced numbers for a statistically confused sport that didn’t start counting sacks until 1982 and has no database for vintage play-by-play, and is way behind the curve from an official perspective when it comes to the concepts of replacement value, positional fungibility, age trends, and other concepts that, if heeded, can save teams millions of dollars and net them enough wins to make thinking out of the box worthwhile. No, it’s going to be just as much about the ability of the numbers guys to mix and match the ability to look at on-field activity with a careful eye and discern what it all really means.
Recently, I was asked by Scout.com’s Cleveland Browns site to put together certain proprietary FO numbers for Cleveland’s inside linebackers. I did so, and wrote an article about those numbers, with the caveat that what I was saying was that, according to Football Outsiders, Andra Davis had the third-best Stop Rate among 3-4 inside linebackers in 2007. That’s it. I stressed the point that without my watching Davis play, and comparing his ability to stop opposing offenses in a host of different situations with other players who do what he does, the numbers are just that – numbers. Just that and no more. I didn’t have time to watch Davis, as I had made time to watch Cleveland’s offense line the year before, when I wrote an "Every Play Counts" article about that particular unit. There’s where I was able to look at Jamal Lewis’ numbers and say to myself, "Okay – these numbers don’t jibe." After talking with FO Big Kahuna Aaron Schatz about it, we discovered that a series of situations and circumstances created a divide between the line stats and Lewis’ own actual productivity, and that forced me to write an afterword explaining why everything I just said didn’t apply in one sense.
And that’s what we’re dealing with in the overall picture – the fact that as we put these numbers out there, the ground is still shifting beneath our feet. These things have to pass the eye test as well, and we’re very aware of the need for a 50-50 solution. That’s why we have Audibles at the Line, and Every Play Counts, and Every Stat Tells a Story, and Too Deep Zone. At the same time, Aaron is tweaking replacement level and so many other stats to get closer to the truth. And as much as we’d like it all to be out-of the-box perfect, that’s not the way it goes.
Aaron’s motto, "The best is the enemy of the better," means to me that as much as we strive for the numbers and methods to be perfectly indicative o performance, they’re not, and we’re aware they’re not. Still, hiding away in the lab until they supposedly are is a futile thought process – these things need air and light to grow, so to speck. They need the review of other staff members and FO fans and FO non-believers. That’s the prime ingredient, right there – intelligent dialogue.
Sometimes, as you discovered with the fantasy projections when you suggested that downloading KUBIAK would be the better way to go, the machine gets in the way. We are internet writers, and we live on internet time, and when we have to assemble our projections, jump off the spaceship, and fire up the Gutenberg Press in order to hit some very imposing lead times for book publication … let’s just say that the process doesn’t facilitate seamless action.
In conclusion, I think we all find it incredibly encouraging that as much as people accept the inevitable limitations of what FO does, our readers still find more than enough to enjoy - in the furtherance of those numbers, and in the writing and analysis that is just as much a part of what we do as the stats themselves. That’s the primary goal as we go forward - to give people enough to enjoy as we’re refining what is still a very young process. July 30 will mark the five-year anniversary of the light going on at Football Outsiders, and as you said in your review, the challenge is now just as much about competing with ourselves as it is about setting out a series of new ideas. Pro Football Prospectus 2008 is pretty reflective of that challenge. And because of that, I’m just as eager to see what PFP 2009 will look like as you are.
Really, we’re just getting started.
by Doug Farrar on Jul 20, 2008 5:37 AM PDT reply actions 0 recs

by 















