NFL Playoff expansion is objectively wrong, Part I

The NFL Commissioner is pushing hard to expand the playoffs to 14 teams. Although the proposal has been postponed for the 2014 season, popular opinion holds that expansion is an unstoppable force. An impending sense of doom hangs over its critics. Google "NFL Playoff Expansion", and then search again with the word "inevitable" tacked on, and you'll get exactly the same results.

Expansion proponents make many argments, most of them illogical and all of them bad. They say it will create more competitive interest in earning the #1 seed (the only bye), ignoring the diminished interest in gaining the #2 seed. They say it will create more excitement as the season winds down for teams that have a shot at the #7 seed, ignoring the fact that it will equally reduce interest in competition for the #5 and #6 seeds (where an added margin of error makes late-season games less relevant).

ESPN's Kevin Seifert boldly states that "It would also decrease the impact of a poor start and late-season injuries that might slow a team's momentum." His logic baffles me. Or at least it would if I expected any logic. Expanded playoffs will increase the impact of late-season injuries by denying better teams the more exclusive playoff berth (and, in particular, the playoff bye) that they earned prior to injuries.

Seifert cites as a "poor start" example the 2013 Pittsburgh Steelers, whose 8-8 record included six wins over their final eight games. But if we admit the possibility that Pittsburgh's bad start was some kind of fluke that does not reflect their "real" quality, then it's equally possibile that the strong finish did not represent true quality, and Pittsburgh was actually a very bad team that rose to .500 on a fluke.

For every alleged "hot" team that gets admitted after a bad start, we will also have a "cold" team coasting in on a good start.

The final argument here is that "cold" teams don't matter, and we absolutely have to make sure these "hot" teams with a legitimate shot at the Super Bowl are in the tournament. Dan Pompei of Bleacher Report sums it up like this:

"It's a fallacy that a team's regular-season record is the primary determinant of how efficient and dynamic that team can play in January.... The playoffs are about which team is best in the fifth quarter of the season."

That's logical and generally true. But still stupid.

Pompei (like many others) does not want teams to earn prestige by making the playoffs and he does not want the NFL Championship determined by anything other than playoff performance. Because the regular season's win-loss record is such an imperfect indicator of a team's efficiency at the end of December, he wants to provide the largest possible margin for error by including more teams in the playoffs.

To put it more bluntly, Pompei views the regular season as a means of providing a rough efficiency ranking of NFL teams, and that's all he wants it to be used for. He does not want those 256 games to actually count.

And therein lies the crux of my own argument: Expanding the playoffs will add two meaningful games at the cost of making the previous 256 games less meaningful.

Roger Goodell's Limitations

Before we go further, let's answer the classic appeal to authority that says Roger Goodell knows what he's doing and he's damn sure not going to make a mistake when it comes to making money for the owners.

As Commissioner, Goodell has to negotiate multi-billion dollar broadcast deals; hire and fire subordinates; dodge bureaucratic red tape; and manage the competing interests and considerable egos of thirty-two different owners. He needs to be a salesman, market analyst, manager, lawyer, and politician.

That's more than I could handle. If I were Commisioner for a week, I imagine I'd botch the network contracts, piss off 30 of the owners, and get run out of office after a scandal involving a 19-year-old intern named "Miku" (sorry, active imagination there).

Goodell got skillz. More precisely, Goodell has enough skills to carry out these typical job duties. But has he ever designed a tournament bracketing system? Does he have a proven track record of doing so?

Does anyone?

This is the purview of gaming nerds, not CEO types. Goodell may have the ability to sell and implement such a system, but he has no qualification for picking the system that produces the most intriguing per-game stakes. That's like asking a world-class surgeon to perform a phlogemectomy on an extra-terrestrial life form. Sure, he can handle the scalpel and forceps better than the average Joe, but he has no clue what a plogem is or whether or not the organ should even be removed.

Big companies, big governments, and big shots can make bad decisions. That's why we have bankruptcies, war, and reality T.V. shows with Donald Trump. And for a massive operation like the NFL with tremendous historical momentum, it's possible-- I say likely-- that they could make a bad financial decision regarding playoff expansion and never even realize it. The marginal damage to the bottom line is likely to go unnoticed amidst broader trends.

What's more, we have reason to question Goodell's motives. He's already made a boatload of money during his tenure as Commissioner, a situation which leaves such a man pondering his legacy. Right now, his administration would be remembered for Bountygate, a 5-month lockout and new CBA, a referee strike, and increased concussion awareness. While there's nothing necessarily at fault with those accomplishments, it's not a list that will etch his name in history as a legend who shaped the NFL.

Note that Goodell's other cockamamie scheme is to create a franchise overseas. Like expanded playoffs, a London-based team is a paradigm shift that futures generations will see every time they look at the league standings.

I'm not suggesting that Goodell is deliberately sacrificing the league's future to stoke his own ego. Rather, the desire to make history makes it very, very easy for him to imagine competence in an area where he actually has none.

The Crux of the Matter: Why Six per Conference is Ideal

A few months back, I got into an argument with some jackass who thought he was as smart as me. Said jackass insisted that if six teams were better than seven, then five must be better than six. Why is six ideal? Are we just being dogged traditionalists?

To wit, my answer:

Isn't it obvious?

With six playoff teams per conference, there are four distinct tiers:

Two teams earn a bye.

Two teams get a home game.

Two teams barely make the playoffs.

Ten teams go home.

At any given point in the season, every team in the top half of a conference is just one seed slot away from either moving up or moving down a tier. The playoff stakes in regular-season games are high.

The format provides the necessary margin for error. It is very easy for the best team in a conference to get the #2 seed through some combination of fluke plays, bad calls, inane tie-breakers, and (most likely) a difficult schedule. Likewise, the third-best team can end out with the #4 seed, and the fifth-best team can end out with the #6 seed. By having exactly two teams in each tier, you greatly improve the odds that each of the top five teams per conference will get at least the seeding tier they deserve.

This cannot be improved. Cutting back to five teams per conference means there will be half as many Wild Card teams and half as many teams with a true bye-week advantage (the #2 seed will play an opponent who also had a bye).

Expanding to seven teams per conference means there will be half as many bye weeks. The importance of this cannot be understated, because this impacts the truly elite Super Bowl contenders. Consider, too, that in nearly half of all seasons the two best teams will be in the same conference.

Hypothetically, we could (I know I could) design a good playoff system with a different number of teams. But such would require an elimination of the current Divisional and Conference structure. Holding that constant, six per conference is the only good option.

Simulation Time

The current format is better and I'm going to prove it.

We're going to simulate a number seasons in each playoff format and compare the outcomes to show how often the objectively better teams make the playoffs and win the Super Bowl.

First, we need a set of numbers that represents a typical distribution of team quality. Thankfully, Brian Burke of advancedfootballanalytics has already laid the groundwork with his Team Efficiency calculations. His model is based on the actual correlation between performance statistics and wins, and is very good at predicting future outcomes. Burke's own predictions use a mixed soup of efficiency stats, but these can be merged into a Generic Win Probability: The likelihood that a team would beat a league-average opponent at a neutral site.

So I simply copied the final Generic Win Probabilities over the last five years and averaged them together. For example, the #3 team in the league finished with GWP's of 75%, 68%, 66%, 72%, and 75%. So the #3 team in our simulated seasons will always have a GWP of 70.8%. Piece of cake.

Home Field Advantage

Burke calculates the Home Field Advantage at around 59% (technically it's a logit, or change-in-log-of-odds-ratio, of -0.36 + 0.72, blah blah hard math stuff). I used this for all regular-season games, but Doug from Pro-Football-Reference found that home field advantage disappeared almost entirely in the playoffs when teams had the same record.

Aside from the possibility of this being statistical noise, playoff matches are the only time when an away team has the chance of playing at a venue where they played earlier in the season (if facing a division rival); and other, more detailed studies have shown that this familiarity reduces or eliminates home field advantage.

Since realignment, the winning percentage of home playoff teams looks like this:

Wild Card Round: 27-21 (56.25%)
Divisional Round: 31-17 (64.6%)
Conference Championship: 15-9 (62.5%)

I wanted to account for this change, but cautiously, so as not to overstate the effect. So I used a logit of .28 (equivalent to a 57% home field advantage if evenly-matched) for the Wild Card Round and for Divisional Round games when neither team had a bye; I used a logit of .32 (equivalent to 58%) for the Conference Championship, and the standard logit of .36 (equivalent to 59%) when the home team has a bye and their opponent did not.

Note that I am taking care not to overvalue the bye week!

Fifty thousand years of NFL football

With all that in place, I randomly assigned the teams for each of 50,000 seasons. So while the #1 team in the league always had a GWP (Generic Win Probability) of 76.2%, they were scrambled about so that in any given year their divisional race and schedule could be relatively difficult or easy.

The Simulants played a normal NFL schedule (six home-and-away division games, four interconference games against the same opponents as division rivals, etc.). The only differences were that the last two conference games on each schedule were not based on previous season's results, and ties for playoff spots were randomly determined (tie-breakers that exist are not based on team quality, but random variance in scheduling).

(Full program available on request. Please include notarized indemnification against damage to your sanity.)

Bear in mind that, even though the distribution of team quality is derived from a predictive model, there is no "evaluation error" in the simulation. A team with a 70% chance of winning a game will win that game exactly 70% of the time.

Results: Exactly what I expected:

With just five teams admitted per conference, a top-two team (from among the entire league) will miss the playoffs nearly 15% of the time. That drops dramatically with the inclusion of a second Wild Card contender (6-team format) to 8.6%.

Expanding to a 7-team format is where it gets interesting. Good teams miss the playoffs rarely, but at a cost of adding significantly more crap to the playoff pool (21% are in the bottom half of the league). And, although a top-two or top-four team gets a slight boost to their chance of being in the playoffs, they nonetheless become less likely to win a Conference Championship or Super Bowl!

It's the bye week, stupid.

	format:	5 teams per conference	6 teams per conference	7 teams per conference
chance of MISSING playoffs if your team is...	top 8	31.00%	22.10%	15.60%
	top 4	19.80%	12.40%	7.90%
	top 2	14.60%	8.60%	5.90%
composition percentage of playoff teams from...	top 8	55.17%	51.93%	48.23%
	top 4	32.09%	29.20%	26.30%
	bottom 16	16.70%	18.68%	21.02%
percentage of Conference Champions from…	top 4	56.03%	57.52%	55.70%
	top 2	33.67%	34.63%	33.65%
	NOT in the top 10	29.76%	28.17%	30.32%
percentage of Super Bowl Champions from…	top 4	64.23%	65.24%	64.58%
	top 2	40.39%	41.39%	41.14%
	top 1	23.08%	23.91%	23.56%

The 6-team format produces a better quality of Conference Champions and Super Bowl winners than either alternative.

Of particular interest for the Super Bowl, compare these differences between a 7- and 6-team format:

#1 Team Wins Conference Championship: 37.33% (seven teams) - 38.16% (six teams) = -0.83%
#1 Team Wins the Super Bowl: 23.56% - 23.91% = -0.35%

#2 Team Wins Conference Championship: 29.98% - 31.09% = -1.11%
#2 Team Wins the Super Bowl: 17.58% - 17.48% = +0.10%

That's not a typo and it's not an error. For the #2 team in the league, a 7-team playoff format causes a huge drop in their ability to make it to the big game, but a tiny increase in their chance to actually win it. Why? Because the same degradation is happening in the other conference. A 7-team playoff format will produce more bad Super Bowls.

Think those numbers are too small to matter? Hold that though, bucko. Perspective will be added in Part Two.

Addendum: Win buckets for Champs

The numbers above show decreased playoff success for objectively better teams, regardless of their regular season records. That's the reality.

But what about perception? If good teams have mediocre records (low seed) and medicore teams have good records (high seed), casual fans may not care so much about the underlying reality. So I added a counter to my simulation to show how often the Conference Championship participants, Super Bowl participants, and NFL Champion had each given regular-season win total. Per 100 seasons:

5-team format	Conf Champ Losers	Super Bowl Loser	Super Bowl Winner	Final Four Total
9 or fewer wins	16.7	5.9	4.1	26.8
10 wins	36.2	14.5	11.1	61.8
11 wins	53.5	23.7	21.7	98.9
12 wins	48.3	25.3	26.0	99.7
13 or more wins	45.2	30.6	37.0	112.9
6-team format	Conf Champ Losers	Super Bowl Loser	Super Bowl Winner	Final Four Total
9 or fewer wins	22.4	8.0	5.6	36.0
10 wins	36.2	14.4	12.1	62.8
11 wins	48.8	21.9	20.6	91.3
12 wins	46.4	24.9	25.2	96.5
13 or more wins	46.1	30.9	36.5	113.5
7-team format	Conf Champ Losers	Super Bowl Loser	Super Bowl Winner	Final Four Total
9 or fewer wins	34.9	12.4	8.7	56.0
10 wins	40.5	16.0	13.9	70.3
11 wins	43.8	20.1	19.0	83.0
12 wins	38.8	21.8	22.8	83.4
13 or more wins	42.0	29.7	35.6	107.3

With 7 teams admitted per conference, there will be far fewer 11- and 12-win teams playing in the Conference Championship and winning the Super Bowl. The 7-team format also increases the number of 9-win teams who lose a Conference Championship by a whopping 59%, going from (approximately) one such team every 5 seasons to one such team every three seasons.