Does Payroll Matter?

Editors note: As a byproduct of dealing with some really useless SBN publishing issues this morning, I managed to lose the original piece. This is just a re-posting  with some minor changes as some of the content was lost. 

On Friday, we went through a look at 2008 NFL team efficiency with respect to cap management. Football Outsiders devised a quick metric to evaluate teams on their ability to win with a lower payroll as well as their ability to roll over cap room for next year. That lead to a brief discussion in the comments as to whether or not salaries mean very much with regards to success/talent. Because there is a salary cap, major spending discrepancies are, in theory, wiped out. 

For salary data, the handy USA Today NFL Salaries database.

For standings and point differential data, Pro Football Reference.

For Team Efficiency DVOA data, Football Outsiders.

 

Win % and Point Differential are both flawed in the effort to determine true team talent. DVOA is about the best metric available and should be the more reasonable evaluation of performance.

Because I wanted to compare data from 2002-2008 (the Texans came in for the 2002 season), a straight comparison of winning percentages, point differential, or DVOA  to salary would be skewed by upward trending average payroll. As such, I normalized season salaries with something akin to Salary+ (if you are familiar with baseball statistics): Adjusted Salary is then each individual team salary in any given year divided by the average salary for that respective year. Simple enough. I then dropped all the data into Minitab and ran three separate regressions.

Adjusted Salary Regression Results

P values
Adj. R Sq.
Pearson Correlation
Win Percentage to Adjusted Salary
0.039
1.50%
0.138
Point Differential to Adjusted Salary
0.040
1.40%
0.137
DVOA to Adjusted Salary
0.300
0
0.070

 

Both Win % and Point Differential are statistically significant with respect to Adjusted Salary. While they are significant, the correlation is dreadful: check the scatterplots below.  For the two flawed statistics (Win% and Point Diff), there is a minor amount of correlation to go along with the significant p-value and minuscule adjusted r-squared. DVOA is just not affected by Adjusted Salary.

How can we justify the fact that the outcomes measures show significance while the more accurate team talent measure has none? We have a fairly large sample size, with n=224. Even with low correlation, large enough data pools can prove statistical significance in weak relationships. The rest is just noise.  In this case, it's safer to look at the adjusted r-squared values to know that only a fraction of our variation is explained by the model. This is a case of statistical significance without practical significance.

There may be a relationship between success and payroll, but that relationship is hazy at best. As long as a GM is spending money at a level near the salary cap, the rest has to do with how that money has been spent (surprise!).  Does the result make logical sense? Yup. There are enough bad contracts, injuries, and inefficiently run teams to negate the effects of  slightly differing payrolls. I'm sure you can think of enough individual data points (2008 Seahawks, the most recent Redskins teams, Oakland, etc.) to understand the concept. The data backs it up.

Scatterplots of the data (you can click on any of the scatterplots for the full image):

Winning_medium Pointdiff_medium

Dvoa_medium

 A few other notes on salary differential that I came upon:

  • The largest salary disparity in any single season came in 2002, where the Atlanta Falcons had 196% the payroll of the Vikings ( $85.1 million to $43.5 million)
  • The largest single Adjusted Salary? The 2003 Redskins, at 140.1
  • The 2002 Vikings had the lowest Adjusted Salary at 66.0
  • For recent reference, the Raiders had the highest Adjusted Salary at 134.5 last year while the Chiefs brought up the bottom at 73.8

 


X
Log In Sign Up

forgot?
Log In Sign Up

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior users will need to choose a permanent username, along with a new password.

Your username will be used to login to SB Nation going forward.

I already have a Vox Media account!

Verify Vox Media account

Please login to your Vox Media account. This account will be linked to your previously existing Eater account.

Please choose a new SB Nation username and password

As part of the new SB Nation launch, prior MT authors will need to choose a new username and password.

Your username will be used to login to SB Nation going forward.

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join Field Gulls

You must be a member of Field Gulls to participate.

We have our own Community Guidelines at Field Gulls. You should read them.

Join Field Gulls

You must be a member of Field Gulls to participate.

We have our own Community Guidelines at Field Gulls. You should read them.

Spinner.vc97ec6e

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.

tracking_pixel_9341_tracker