Week 21 is here. Logjam in the middle has started to clear a bit and hopefully once we get back to a somewhat regular schedule as teams exit the Gold Cup things will get less loopy. Only a few screwy results this week (San Jose slight drop despite the win, LA and Chicago slight rise despite the loss) but I'm gonna throw up the "explanation table" as well as add a "snapshot" stat that captures a more immediate sense of where a team is at. First, overall rankings. Or, if youre a Dallas fan, tankings... yeesh. Wheels are officially off. The graph And what I guess you could call an explanation table? Average season long and last 5 games are what's been used to generate team rankings. "Snapshot" is a one game value that maybe offers some consolation when the spreadsheet seemingly rewards teams that lose or punishes those who win. LA and Chicago turn in #11 and #14 performances last week, San Jose at #8. Dallas bottoms out at #19...
I dunno, I think the weighting of the last five games is generating enormous amounts of noise that translates as movement in the rankings, but doesn't result in a great deal of difference in the relative tiering of teams. This system seems to overdramatize the actual changes.
Jough Does your ranking system take into account the strength schedule? For instance, if one team gets only 3 points from 5 games against tough teams and another gets 3 points from 5 games against weak teams, does the system know that the first team did better than the second? If there isn't some correction for strength of opponents, than the noise added by the last 5 games could be even more dramatic because one team's schedule could be far tougher than another teams during a 5 game stretch.
It does not, and I've gone back and forth on whether or not I want to put the effort into correcting this. It's something I'm looking at, but I definitely, defintely agree and wish I had taken the time to fully reflect on it. At the time I started to incoporate last 5 into the calculation it made everything look really really good, but now in hindsight simply averaging the last-5 and season-long has added more noise and movement than I'd like. At this point though, I feel like I'm all in and don't want to make another change, for consistencies sake. Next season I feel like I'll dial back the last-5 component to perhaps a 33% weight instead of the current 50%. Good comments, I really appreciate it. Of course on the flip side, I could argue that there actually is that much weekly variability and parity in MLS, especially during International tournement weeks...
It really depends on what "Power Rankings" means to you. Does it mean "which team is actually the best" or does it mean "which team has played best recently" or something else? The rankings I have presented in this thread are aimed more at the first of these which might make it hard to compare to other people's rankings.
Totally back your thinking here. You've got to run the data in a consistent way to get this sort of feedback. I think your approach is capturing the top and bottom well, but I'm having a hard time wrapping my head around the differences in the middle, and altering the weighting formula might clarify that a bit better.
This is what I've struggled with for two seasons now. It's pretty clear that the "best" team in MLS doesn't always (maybe rarely does?) win what the league considers the ultimate trophy, i.e. the MLS Cup. Run of form seems like it's more geared towards what people mean with power rankings, but is that 3 game form? 10 game? I've chosen 5, but that's pretty arbitrary, and even then you need a factor to consider season long performance less you start running into real head scratchers (Chicago being #5 would be one). I think ELO ratings are a really solid way of ranking teams to be honest, but even then there's so much that can be tweeked. Ultimatly I thnk Power Rankings should be a jumping off point for discussion on who's playing well and who's not, and why that is. The issue I've always had with what MLS, ESPN, etc. put out is their rankings and justifications can just be silly. With the numbers I've come up with I can at least point to simple inputs (results and goals) and explain why a team is ranked where they are, even if the ranking itself might seem (and probably is) flawed...
I love occasionally looking at different ranking methodologies, even if they don't necessarily apply that well to soccer/MLS. One such thing from baseball is the RPI (Relative Power Index). It was devised as a method of determining which team is the strongest by accounting for strength of schedule. For baseball, it is calculated as follows: 25% Own winning percentage 50% Opponent's winning percentage 25% Opponent's opponent's winning percentage For the Opponent's winning percentage, it is calculated as follows: For each game, calculate the opponent's winning percentage for all games which don't involve the opponent and the team. Meaning, find out how that team is doing against all other teams in the league. Average the results for all of the games. For the Opponent's opponent's winning percentage, it is calculated as follows: For each game, take the opponent's calculated value for opponent's winning percentage. Average the results for all of the games. To adapt this to soccer/MLS, I replaced winning percentage with PPG, but left the rest of the setup alone. I decided to post them here, because some things which I thought were interesting came out: Code: Team PPG OPPG OOPPG RPI POR 1.74 1.40 1.37 1.48 RSL 1.85 1.33 1.39 1.47 MON 1.72 1.35 1.36 1.45 VAN 1.68 1.33 1.38 1.43 FCD 1.55 1.40 1.37 1.43 LAG 1.50 1.41 1.37 1.42 NYRB 1.55 1.37 1.35 1.41 SKC 1.65 1.30 1.37 1.40 SEA 1.41 1.39 1.36 1.39 HOU 1.53 1.30 1.37 1.37 PHI 1.50 1.30 1.37 1.37 NER 1.33 1.36 1.37 1.36 COL 1.35 1.38 1.38 1.36 SJE 1.14 1.39 1.37 1.33 CHI 1.17 1.39 1.35 1.32 CLB 1.21 1.30 1.37 1.29 CHV 0.74 1.47 1.36 1.26 TOR 0.72 1.46 1.34 1.24 DCU 0.53 1.36 1.35 1.15 First of all, I was surprised that RSL didn't top Portland for #1. Personally, I think RSL is likely be the better team, but Portland has apparently played against tougher opposition so far. If this holds, August is going to be a very entertaining month, since they face each other twice in league play, and once for the USOC. Second, I expected Chivas to be #18, but it seems that Toronto is doing worse than I had noticed. I probably just ignored it because I expect Toronto to do that badly, and Chivas has occasionally been at least OK (on the field). Finally, there is only 1 team from the East in the top 5 (Montreal), and only one team from the West in the bottom 5 (Chivas). Based on the EvW thread, I expected the West to come up better, but not THAT much better.
It's been awhile since I updated. Still using a monthly change in parentheses here. Can't argue with RSL at #1 too much. SKC is at number 2 in the rankings, but 1596 is a fairly weak rating for #2, meaning there is a lot of parity between teams at this point with no real clear powerhouse. Vancouver has had a hot month, Montreal and Dallas are sinking like stones, and the bottom dwellers aren't changing much. The End of Season prediction is showing RSL winning the SS now, with a four point projected lead over SKC.
I appreciate you keeping track of this. It is fascinating to see. However, its at this point in the season that my reasons for dislike of ELO ratings for leagues show up plainly. We are more than halfway through the season, and teams which have made big turnarounds this season (plus or minus) still don't reflect that. I'd put Seattle, even with their potential and games in hand, no more than middle of the pack. Yes they've been hit by injuries and stuff, but they've also been very inconsistent. San Jose shouldn't be as high as mid-pack, especially since most other teams have games in hand on them. FCD should maybe a bit higher than they are, because of the results they have gotten. I also think Montreal should be higher than mid-pack; their record also justifies that. My best suggestion would be to increase the weighting used, so that larger adjustments are made. It will make the results jump around a bit more on strange match results, but the system should theoretically correct for that. As they stand, they are more of a measure of past worth stretching back for over a year than they are a measure of how good of a team they are now.
This is a true statement, and it describes what vividox effectively intends his Elo ratings to be. My ELO+ ratings only take this season's results into account and are probably closer to what you are seeking. Here are the 2013 MLS ELO+ ratings and points predictions for 07/18. Code: Rank Team ELO+ Rank Team Points 1 RSL 125 1 RSL 60.0 2 POR 121 2 POR 57.6 3 VAN 119 3 MTL 55.1 4 SKC 112 4 VAN 55.1 5 NYR 109 5 SKC 54.2 6 HOU 107 6 NYR 53.0 7 MTL 106 7 HOU 51.5 8 PHI 105 8 PHI 50.4 9 LAG 105 9 LAG 50.3 10 COL 105 10 DAL 48.3 11 SEA 102 11 SEA 48.1 12 DAL 99 12 COL 46.6 13 CLB 93 13 NER 43.5 14 CHI 92 14 CLB 42.2 15 SJE 92 15 CHI 41.3 16 NER 91 16 SJE 39.3 17 CHV 78 17 CHV 30.5 18 TOR 73 18 TOR 28.4 19 DCU 66 19 DCU 23.2 The average number of games played per team is now 19.4 (K=10.3), and so far home teams have a winning percentage of 0.628 (H=26).
alethist is correct. I don't really intend for my ratings to be a short term form. I do like showing a month's worth of change in the parentheses now, but I truly see my Elo rating as a POWER rating, not a FORM rating. Power isn't something you have because of a handful of results, power is something you have to build up for a long period of time. And it doesn't just go away because you lose three games in a row, either. If defending champs the Los Angeles Galaxy lose three games in a row, we don't think, "oh, their season is shot", we think, "they've had some bad results, but we know they'll probably figure out how to turn it around." And that is how I look at my Elo model as well. So many results - especially in MLS where parity reigns - can be written up as simple variation. The actual power of each team is, to me, something that is much slower moving and less short term based. I don't really care about making a model that shows how teams have done in their last five or ten games, we can look at game results and table standings to see that - I don't need to build a model for it. I want my model to predict the rest of the season. When I look at my model, I ask questions like, "Okay, FC Dallas got off to a really good start, but all their wins were at home and they really aren't that 'powerful' of a team, how do I think they are going to finish out this season, despite their success in their first eleven games of the season?" Right now, despite the great start FC Dallas had, despite Dallas sitting on 31 points, my model predicts they will miss the playoffs. And I totally agree with that. Similarly, Seattle hasn't had great results, but they are currently predicted to make the playoffs. I totally agree with that too. I don't count Seattle down and out right now, and - despite some results they've had - I think they can still get results the rest of the season and make the playoffs. So, that is what I try to show in my model.
As an aside, don't let my Elo ratings discourage you from Elo ratings in general. They are all different, and almost entirely up to the author's subjectivity. Especially when you get into more complex Elo ratings that deal with a lot of different elements (home vs. away, goal differential, level of competition, season to season form, etc) like these do.
Oh, I know that Elo ratings are good for some things. I maintain my own set of Elo ratings for the differences between leagues in the US Open Cup, and I find them to be a relatively reliable indicator there (An NASL team hosting an MLS team should be almost a toss-up for result based on my ratings there, which is largely what we see). They also seem to do well for national team rankings. My opinion (and it is just that, an opinion) is that when we are more than halfway through a season, the ratings shouldn't be so heavily influenced by prior seasons. In a league with such huge turnover and salary caps, the prior season isn't quite as important in determining the strength of a team. I don't think you should completely ignore it, but based on your raw Elo ratings (not the season predictions), Seattle is still considered to be a stronger team than Portland, which I'm not sure should be the case right now. I agree that Seattle can still go on a good run and get into the playoffs, but I think they should be shown on the outside looking in. They have a history of success, but have been very inconsistent and are 7th in PPG in the west so far, and with all of their injuries, that looks unlikely to change soon. Likewise with San Jose, I don't think they deserve to be shown as a midling team in the raw ratings. These are just my qualms with using Elo ratings for club seasons, and I don't expect any changes to your system, and I do thank you for maintaining them. I'm just trying to have some friendly discourse. =D
I totally understand where you are coming from. I guess I just have this innate feeling that power is more on a team level than a personnel level. A few examples: Toronto perennially sucks even though they are a revolving door, RSL completely flipped house and yet are still seeing success, LAG are always in the playoff discussion no matter how many games they lose in the front half of the season. Now of course, for every example I just listed, there is a counter-example, but I just tend to think that as much as personnel plays into results, the FO's mentality and club expectations almost have more to play into it when you look at it from a season-to-season standpoint. And if a club does turn a corner, my Elo ratings will certainly show that (though, admittedly, slower than some would like). I think it is also worth mentioning that the actual top down ranking doesn't mean as much as the relativities of the ratings themselves. San Jose is in "tenth place", but they are only 17 points from 15th place. As far as the ratings are concerned, San Jose, Philly, NE, Colorado, Columbus, and Dallas are clustered very tightly, and might as well be considered on equal ground.
Hrm, that gives me an idea, which maybe I'll investigate tonight. I think it'd be worthwhile to see how much of a correlation there is between results one year, and results in the previous/next. If nothing else, it should give a baseline for how important "dynasty" is in MLS. I agree that there are some teams which seem to always be floating to the top, no matter how much they change (LA, RSL, HOU, DC of old), and some teams which always are sinking to the bottom (TOR, CHV, DC of late). It'd just be interesting to see how similar one table is to the next. I'll post a summary of my findings in case anyone else is interested. =D
Here are my updated rankings. As a reminder, this method gives equal weight to every game; no extra weight for recent games. Also, the numbers below do not incorporate the results of games that have been played. They reflect simulated results assuming a hypothetical double round-robin season. Things I noticed: New England is still ranked pretty high. This is because the model only concerns itself with goals since it is assumed the ability of a team to get results can be measured by the goals it scores and allows. New England is tied for second best defense (measured in goals allowed per game) and good defenses tend to be rewarded more than good offenses. Portland is further ahead of RSL than I expected. A six point gap is very large, especially since it is undecided which team is better in my opinion. Positions 5 and 14 are separated by just 6.5 points. This really demonstrates the parity in this league and forces me to point out that the just because a team is ranked ahead of another here doesn't mean there is a measurable difference between them. This makes me feel better since my Rapids are in 12th, but really they are just tied for "middle". The gap between Chivas USA and DC United is probably not 14 points in reality. If DC find their shooting boots, that gap would narrow fairly fast. I am going to try to get in the habit of posting some extra information about the model when I post my rankings. The average number of goals per game came out to 2.59 (2.64 was the average last season). The percentage of games ending in ties was 25.4% (24% last year). The home advantage parameter that was derived indicates that the home team is on average 1.53 times more likely to score than the away team if the strength of the teams were equal. Another aspect of the model is the effect of the score on the probability of future goals. I am finding a significant decrease in scoring probability when a team takes the lead and a marginal increase in probability when a team falls behind.
The thing I wonder about your model, Zona, is the weighting. If you notice, Portland ALREADY has more ties than the model would predict given the results up to now. Also take into account the GF projected vs current. I think what is happening is that Portland ties a majority of their games, but when the do win they win BIG (for the most part). This kind of screws with the numbers when you try to use weighting for GF and GA to help project results. On paper a big positive GD should indicate a team winning a majority of their games, but in Portland's reality the big GD is a result of big wins in a few games. Just another variable in the schematic for trying to put together a "power ranking" Oh and I still think some marginal weight should be given to recent results. How that plays into projected outcomes...I have no idea. I think over the course of a season small sample trends (say 5 games) is totally meaningless, teams will have ups and downs but a regression towards the mean is the most probable event. The tough part is trying to figure out how to project what exactly that mean is. Also, why does your model project more games (W+L+T) than is possible? EDIT: i'm an idiot, totally glossed over the double round robin format resulting in what would be 36 games.
This might come in handy: A few standouts: - Kansas City from 1999 to 2000 was near worst to first - Chicago from 2003 to 2004 was first to worst - Los Angeles from 2008 to 2009 was second worst to second best - DC went from back-to-back Supporter's Shields in 2006 and 2007 to missing the playoffs in 2008 - DC went from 1999 MLS Cup Champion to missing the playoffs in 2000 - Los Angeles went from 2005 MLS Champion to missing the playoffs in 2006
UPinSLC, I see what you are saying. I think the fact that Portland has tied so many games while winning big in the games they do win indicates that they probably are better than their point total would imply. Now whether they are that much better, I am skeptical. In general, it is tough to predict ties. Wins and losses aren't so tough. Good teams win a lot, bad teams lose a lot, but there isn't such a thing as a very medium team that ties a lot. I tend to believe that a team with a lot of ties like Portland probably doesn't have any true predisposition to tying games; it is probably just coincidence. As for your comment about weighting recent games, I have actually been thinking an alternative method would be to add a slowly varying time dependence to the model and I could report the current state of each team. However, it would be difficult to do and I am still contemplating how to incorporate it or whether it would be beneficial.
Using a Google spreadsheet, I calculated the correlation coefficient for the end of season rankings and PPG between a given season and the previous. I did this for 2003-04 through 2012-13. For 2013, I used the current standings as the mark, and they don't fall too far out of line with the rest. For a quick visual reference as to what these numbers mean, this wiki page has descriptions, and a handy picture: http://en.wikipedia.org/wiki/Correlation_and_dependence Also, with a relatively small sample size (there are some seasons with only 10 teams, and even 20 teams is too few), I don't know how much you can read into these numbers. The Aver line is the average over the whole period, the AAver is the average without the first two "years" in the list. The year column represents the latter of the two years. Code: Year PPG_ Rank 2004 -0.23 -0.49 2005 -0.61 -0.71 2006 0.46 0.68 2007 0.65 0.73 2008 0.25 0.34 2009 0.47 0.54 2010 -0.12 0.04 2011 0.66 0.66 2012 0.41 0.34 2013 0.15 0.06 Aver 0.21 0.22 AAve 0.37 0.42 With the overall Average, I'd make the conclusion that there is no significant correlation between one year and the next. With the 2005-2006 onwards data (basically the data which covers the latest expansion era), it implies that there might be a slight correlation between one year and the next. However, if you look at the picture on the wiki page linked above, that correlation would be weak, if it is even present. This doesn't tell us how that data actually sits, though. It could be that the ones at the top very consistently stay there, or that the bottom ones stay there, or the middling ones stay there. It could also be any combination of the above in a slightly weaker manner. Anyways, it does seem to imply that keeping around some of history is fine. I still think that your (vividox) rankings may hold onto a bit much of it. But that is good contrast to basically everyone else's in here, which don't hold onto any history from prior seasons.
Here are some other rankings I occasionally keep up to date. The both are Poisson models, the one labeled Pois accounts for home field advantage, past results, and future strength of schedule. The one labeled EAP is how the team would be expected to perform over a season of playing "average" opponents. Code: Team Pois EAP RSL 58.8 63.4 POR 57.0 61.7 SKC 54.2 58.7 MON 54.2 49.5 VAN 53.4 54.0 NYR 52.0 52.5 LAG 51.7 53.7 HOU 50.9 50.1 PHI 50.1 49.2 FCD 49.5 46.6 SEA 48.4 47.8 COL 47.3 48.6 NER 46.7 52.8 CLB 43.2 46.3 CHI 40.5 36.3 SJE 39.4 34.1 CHV 31.9 27.5 TOR 31.1 32.3 DCU 24.6 18.6
Just for the record, from a purely theoretical standpoint, the weighting factor K that is applied to every new outcome in an Elo rating system is inversely proportional to the number of previous games that are assumed to have contributed to the incoming rating of the two teams involved. Strictly speaking, it also varies with the difference in ratings between the two teams, but Elo himself advocated the simplification of using a constant for that part of the equation, approximating the average over the "usual" range. Following this approach, the value of K=30 that vividox uses during the regular season means that each result is treated as though the rating of the two teams coming into that match was derived from their actual performance in roughly 26 previous games. For teams with equal ratings (after the 90-point adjustment for home-field advantage), the more accurate number is about 22, while for teams separated by 393 rating points--e.g., if Real Salt Lake played Chivas USA at Rio Tinto right now--it is a whopping 67. In my ELO+ ratings, the difference between teams really is irrelevant, and K is simply adjusted to correspond to the average number of games played by each team to that point of the season (currently 19.4). This means that ELO+ ratings are quite "bouncy" early in the year, but gradually stabilize over time. There is no right or wrong here, just different methodologies that are intended to measure different things.
I'll be interested to see what everyone else's ratings are saying, but SKC's big road win in SLC threw them to the top of my standings. New England moves up 28 points with their big road win over Columbus in stoppage time, and Chicago's 4-1 drubbing over the cellar dweller moves them only 13 points. Not a lot of movement other than that with all the draws we saw this weekend.