Wednesday, April 13, 2011

Death to a Playoff: Chapter 11

The BCS uses computer rankings as a way to objectively rank teams according to their performance, without factoring in bias that is inherent in human rankings. This chapter focuses on the computer rankings that the BCS uses, and the authors do a fair job of describing the roots of the computer system, but do miss some key points. The chapter also unfortunately harps on the fact that the BCS has changed its formula 3 times, which should be viewed positively because it shows that they aren't blindly sticking with a formula if it is making mistakes.



I was under the impression that the BCS computer ranking system was a bit of a mystery, and certainly after reading this chapter, you are left believing that there is no way to figure out how the included computer rankings are calculated. However, I just went to the BCS site and it has links to the websites of each computer ranking, and each website has details about their rankings and how they are computed. While I don't think this has always been the case, and the information may not have been available to the authors at the time of their writing, the BCS is advancing their cause by publishing this data. With the computer polls being kept secret, there is an accountability problem, and this can lead to accusations of secrecy and conspiracy. The authors would likely view the transparency of the computer ranking system as a positive step to address this problem. The revealing of how the computer rankings are calculated makes much of this chapter simply out of date.

One area that the BCS has not made transparent is why they picked these particular computer ranking methods. Maybe they have their own selection process, but their reasons for picking these systems haven't been made public and their sources seem to be randomly selected people.

Another controversial aspect of computer rankings is the margin of victory issue, which is not currently a factor in the computerized rankings. The authors argue for it to be included in the rankings; I will give reasons why it should be left out. First, scores that happen when the game is already essentially decided, and when both teams may be playing backups, will then affect the standings. A team may be up 21 points with 1 minute left in the game, and someone scores a meaningless touchdown with 2nd and 3rd stringers in the game. This late touchdown should obviously not be factored in when considering the overall strength of a team, but if the computers factored in margin of victory that late, meaningless touchdown would then matter in those teams rankings. To take the point even further (and I am fairly certain something like this affected the standings one season, but I can't find the link or the concrete example), there is the possibility of that meaningless touchdown also occurring in an essentially meaningless game, and thus it affecting the outcome of the BCS standings. To give an example, since I can't remember the real example of this happening, let's say the #2 team in the BCS (Oregon, for example) played and beat a team that wasn't going to a bowl (Washington State, for example). Oregon is judged by who they play, who they beat, and by how much they beat that team. Washington State is judged by the same factors, and their ranking will obviously affect Oregon's ranking. The problem arises when Washington State is playing another unranked team late in the season with nothing on the line (we'll use Washington since they always play each other in the last game). Now, a meaningless score in this game will directly affect Oregon's ranking. What happens when Washington is lining up for a field goal to prevent a shutout and make the game 20-3 instead of 20-0? The field goal score would lower Washington State's ranking enough to affect Oregon's ranking and drop them to #3 in the polls. Doesn't that seem wrong? I know it was a roundabout way to prove my point, but those types of scenarios would literally happen every season. Games and scores that have no impact on the overall season become paramount to the ratings systems. Teams should be judged on whether or not they win games, not how close the score is.

Something else that the BCS has done correctly with the computerized rankings system is to continually make improvements. The BCS changing its formula is not a negative thing. They recognized flaws in their formula/process and made corrections to ensure that the two most qualified teams have a chance to play for the national title. When a system allows a team that doesn't win their division or conference into the title game, that is a problem. When a system prevents a team ranked #1 in both polls from making the championship game, that error needs to be corrected. The computers are not always the “fall guys,” as the authors claim;  the human polls' votes have been devalued as well throughout this process of figuring out the best system. The BCS realized that they would not create the perfect formula on the first try. The ranking system that exists today is superior to any previous ranking system in college football.

3 comments:

  1. If you don't take the score of the game into account, you by default rely on strength of schedule in your rankings. You're saying Oregon is better than TCU because Oregon played better teams. That doesn't make them better in any way. All it means is they beat better teams, teams that TCU probably could have beat as well. Who you play should never determine your ranking, how you play should.

    Instead of margin of victory, scoring defense should be used. That way you don't have to run up a score to get better rankings. You could also look at metrics like yards, yards allowed, 1st downs and other stats. Typically if a team has more 1st downs and more yards in a game, they win. You could then compare a teams performance in a game to their season averages. Maybe Oregon won all their games, but they may have given up more yards than their opponents averaged coming in. You can look at amount of time in a game someone has the lead to determine how dominant they were.

    There's lots of things you could look at that the computer rankings are not all doing. The computer rankings are designed to reward teams with harder schedules. If a good team played a bad team they should destroy them. If a good team plays another good team it should be a close game. If you barely beat a bad team, you aren't that good. If you blowout a good team, you're amazing. I think how you play each team on your schedule should figure into the computer rankings.

    ReplyDelete
  2. thanks for replying. i do think that incorporating the score can be good, but it is a slippery slope. computers should be used for simpler, more straightforward tasks like comparing wins and losses. let the human voters sort out the score of the games and how teams should be judged on that. granted, as i write in my newest post, there are plenty of things to be improved upon when it comes to human voting, but i dont think that is an area for the computer systems.

    yards, 1st downs and other metrics shouldn't be used because it simply looks at facts taken completely out of context. until Texas played OK State last year, they had outgained all of their opponents, yet they lost most of those games because of turnovers. Oregon probably gave up a lot of yards, but that is mainly because their offense is so fast-paced. a team shouldn't be penalized because of their style of play. simply because Oregon and Auburn's offenses are of the no-huddle variety, and TCU and Wisconsin apply a time-draining run-first strategy doesn't mean Auburn and Oregon should be punished.

    ReplyDelete
  3. well obviously you would put turnovers into the list of metrics too. Teams that win games tend to turn the ball over less. All I'm saying is that there are a lot of stats to a football game. Winning teams and good, dominant teams have stats that trend a certain way while poor teams have stats that trend a certain way. Teams that rarely turn the ball over, allow few points and have the lead most of the game are going to be pretty dominant.

    Auburn and Oregon won all their regular season games last year, but neither was a very dominant team. Auburn beat Oregon, but it was not a dominant performance. Alabama beat Michigan State and it was a dominant performance. If you compare stats from those and all other games, you start to see the "dominant stats" if you will. Any intelligent statistician, programmer or engineer could use that data to then rank teams based on how dominantly they play...and of course wins and losses take center stage in that debate.

    ReplyDelete