clock menu more-arrow no yes

Filed under:

Bye Week Narrative Check: Is Pete Carroll challenge flag challenged?

New, comments

“Narrative Check” is where commonly held assumptions and beliefs about the Seahawks are held up to statistical study. Sometimes they survive! Often they don’t.

do it do it do it

The Narrative

Seahawks coach Pete Carroll isn’t very judicious with his use of the red flag and often wastes perfectly good timeouts on wild goose replay chases. Sometimes it’s like he lets his big balls do the heavy thinking when it comes to throwing the red challenge flag.

The Method

Pro-football-reference keeps a record of all coaches' challenges. I plan to:

  • compare his challenge record with other NFCW coaches from 2010-2015
  • compare his challenge record with that of all other active NFL coaches
  • examine, for nerdy fun, if there is any correlation across the league between challenge winning percentage and game winning percentage.

Likely the answers will bring up more questions that are hard to answer definitively. The scope of this post is narrow -- do the numbers reveal Pete to be a subpar replay challenger? -- but the discussion can swiftly become wide. Therefore, this slippery question is best left for the comments:

“Did Pete's losing challenges present large enough rewards if overturned to warrant the risk, or was he careless and capricious at times with the flag? In other words, were his gambles worth it, win or lose?”

(As always with these Narrative Checks, I decide on what data to use before I compile it. This prevents me from manipulating it to suit my preferred narrative.)

The Data: Set 1

Pete Carroll vs. NFC West coaches, 2010-2015

The numbers are all over the place, with some pleasant surprises.

Carroll: 24 calls successfully overturned, 29 upheld. Challenge win %: .453

Jim Harbaugh: 12 successes, 23 failures. Challenge win %: .343

Jim Tomsula (that happened): 2 successes, 7 failures. Challenge win %: .222

Mike Singletary (yep that too): 4 successes, 3 failures. Challenge win %: .571

Jeff Fisher: 7 successes, 8 failures. Challenge win %: .467

Bruce Arians: 12 successes, 12 failures. Challenge win %: .500

Ken Whisenhunt: 16 successes, 12 failures. Challenge win %: .571

A) First observation is that the division’s two most successful coaches overall, Carroll and Harbaugh are near the bottom in flag victories. Conversely, tops in replay performance are two failed coaches, Singletary and Whisenhunt. Maybe winning challenges doesn’t matter?

(Predictably, Jim Tomsula is the notable exception: bad at replay victories, bad at football victories.)

Placed in order of challenge win percentage, we get:

Whisenhunt, Singletary, Arians, Fisher, Carroll, Harbaugh, Tomsula.

It doesn’t take a quantum surgeon to notice the obvious: winning replay challenges does not correlate with success. At least not within the crazy confines of our division.

Unless... B) we observe that there might be a correlation between total challenges won and success on the field. Toss the percentage of challenges won aside, and order the teams instead by volume of challenges won. Now it goes Cards, Hawks, Niners, Rams.

Cardinals .28 challenges won/game

Hawks .24 challenges won/game

49ers .19 challenges won/game

Rams .13 challenges won/game

Seeing the Rams there at the bottom, winning challenges only half as often as the Cards and Hawks, makes one want to investigate this further as the sample size grows. Are you better off winning replays more often, and often taking the accompanying loss of timeout, rather than playing it safe? Maybe.

So let’s try it by individual coaches and see.

Whisenhunt .33

Singletary .27

Harbaugh .25

Carroll .24

Arians .23

Spagnuolo .13

Tomsula .12

Fisher .10

The cream is rising to the top. Whiz and Singletary are wrecking the scale a bit, sure — but there is definitely reason to continue investigating along these lines.

C) Third observation is that Carroll's not bad, when it comes to his immediate rivals. His .453 challenge win percentage is respectable. He’s winning enough by percentage, and enough by volume. He’s not outclassed at all by his immediate competition.

(Idea: Maybe the 49ers should’ve retained Singletary for replay duties.)

The Data: Set 2

Pete Carroll vs. all active NFL coaches

Too much work to break down all coaches who served in the league since 2010. I like you guys, but not enough to donate an entire night of research to you. I'll gather all the career challenge records for all active coaches instead.

NFC North:

Jim Caldwell .593, Mike McCarthy .488, John Fox .366, Mike Zimmer .300

NFC South:

Dan Quinn .500, Ron Rivera .459, Sean Payton .433, Dirk Koetter no challenges

NFC East:

Jason Garrett .586, Jay Gruden .214, Doug Pederson .000, Ben McAdoo no challenges

AFC West:

Andy Reid .475, Mike McCoy .455, Gary Kubiak .345, Jack Del Rio .329

AFC North:

Mike Tomlin .517, Hue Jackson .500, Marvin Lewis .469, John Harbaugh .439

AFC South:

Chuck Pagano .482, Bill O’Brien .444, Mike Mularkey .391, Gus Bradley .381

AFC East:

Rex Ryan .508, Bill Belichick .403, Todd Bowles .286, Adam Gase no challenges

In case you’re buried under an avalanche of digits, or adult ADHD’d your way through the list, the total combined percentage of challenge wins is a neat .438, so you’re welcome.

Hey, wasn’t Pete’s percentage .453? Yes. Good memory, reader. In fact, Carroll’s mark here is the very definition of “slightly above average.”

Have another sip of coffee/beer/warm human blood if you’re reading this at 3:15 a.m. These are the only active coaches who win as many or more challenges than they lose.

Jim Caldwell, .593

Jason Garrett, .586

Chip Kelly, .526

Mike Tomlin, .517

Rex Ryan, .508

Bruce Arians, .500

Hue Jackson, .500

Dan Quinn, .500

Small sample size is in effect for Kelly, Jackson and Quinn, who’ve combined to throw the flag only 27 times. It’s hard to win replay challenges, apparently.

Carroll is 15th among 28 active coaches. (Wait, 28? Henceforth excluded are Koetter, Pederson, McAdoo and Gase, who are 0-for-1 combined in replay challenges.)

Some coaches are not good at the red flag business. Shocker. The bottom ten are:

Bill Belichick (!!), .403

Mike Mularkey, .391

Gus Bradley, .381

Jeff Fisher, .380

John Fox, .366

Gary Kubiak, .345

Jack Del Rio, .329

Mike Zimmer, .300

Todd Bowles, .286

Jay Gruden, .214

So Pete Carroll is an average challenger, maybe a little better, and that’s that? Not so fast. Like I alluded to earlier, compiling these charts made one important detail stand out: some guys win many challenges because they throw the flag a lot, so they accumulate many successful overturns, while others are quite conservative instead. What if I made a subset of each type of coach and compared their winning records? I wanted to do this for the NFC West but it felt small-samply. Now there’s tons of data.

So I organized coaches into three tiers:

Tier 1 is for those who win at least .20 challenges per game

Tier 2 is for those who win .11-.19 challenges per game

Tier 3 is for the cautious ones who win .10 challenges per game, or fewer.

Tier 1 includes Harbaugh (.27), Ryan, Payton, McCarthy, Carroll (.24), Arians, O’Brien, Pagano, Tomlin, Fox, Rivera and Kelly (.20)

Their coaching record is a combined .596 — they are a highly successful group. Only one member (Ryan) has a career W-L under .500, and scarcely at that.

Tier 2 includes Caldwell (.19), Lewis, Garrett, Reid, Mularkey, Bradley, Del Rio, Kubiak, Belichick.

Their coaching record is a combined .552 — even with Darth Hoodie in the mix. It’s quite lower without him but it would be unfair analysis to throw him out as an outlier.

Tier 3 includes McCoy, Quinn, Jackson, Bowles, Fisher, Zimmer, Gruden (.08).

(We’re still excluding Koetter, Pederson, McAdoo and Gase for lack of participation.)

The coaching record in Tier 3 comes out to a combined .512, leading me to believe volume matters. Perhaps don’t worry about how many challenges you lose? Just try to maximize replay wins at the expense of replay winning percentage.

By the way, organizing the coaches into four tiers instead of three yields the same result. While it’s not safe to conclude that high volume of successful challenges will result in more wins, the data point strongly in that direction.

Pattern-busters like Belichick obviously exist. But it’s telling that among the top half of coaches in volume of challenges won, only one has a losing record; among the bottom half, six have losing records.

And so Pete’s a high-volume, medium-percentage challenger. Or, you might be able to say, above average overall with the red flag.

Miscellaneous Concluding Thoughts

Carroll's best challenge year was 2013, when he went 7-4 in challenge situations. Coincidence? Maybe. He won zero challenges the year before and went a mediocre 2-5 the year after. Both highly successful seasons that ended with losses. Maybe 2013’s replay performance was worth an extra win, and therefore home-field advantage? I could see it if you consider the Texans game in Week 4, right about when Doug Baldwin decided to do a very Doug Baldwin thing. Backpedaling out of his own end zone, Russell Wilson heaved the ball at the intersection of Sprinting Doug and Left Sideline.


The pass was ruled incomplete and Carroll, out of either desperation, hope, confidence, or calculated risk, or all four, challenged. And won.

It was third and seven and RW was backpedaling out of his own end zone. The Hawks were trailing by 14 in the fourth quarter. The drive ended 12 plays later with a Lynch touchdown run that helped force overtime.

Maybe without the mentality of gambling for high rewards, Carroll doesn't pull the trigger there. And as one win becomes a loss, maybe home-field advantage slips away. The NFCCG suddenly isn't at the Clink, and maybe the Lombardi case is still empty.

My conclusion, which you’re free to dispute: a high-risk high-reward high-volume red flag strategy is in fact a positive. All you can lose is a timeout and a suspension of privileges. Whereas what you can gain is more significant: a play that might alter the outcome of the game, the year, the career significantly.

Keep it up, Pete. It’s working.