Which Ranking Will Win Out in Brazil: FIFA, ELO, or SPI?

Yes, the draw against Thailand in Bangkok proved costly, but no the draw against Saudi Arabia in Saudi Arabia (we beat them in Tehran 2:0) wasn't controversial. We didn't qualify in 2002 ultimately because we didn't earn the points. The main culprit is definitely Iran. But sometimes the math and how the points add up doesn't really tell you everything either. Just because a team slips in one match, you can't ignore its entire record for the preceding 4 years.

 

Since World Cup 2014 is pretty much over, we an revisit this thread to see which ranking system did better to predict the results in this tournament?

When it came to Group A, ELO and SPI got the order right; FIFA didn't even if they all were right that Cameroon was the weakest in that group.

In Group B, all got Spain wrong! ELO and FIFA would have picked Holland while SPI did not. All got Australia correctly as the team to finish last.

In Group C, ELO and FIFA got it right. While all were right that Japan would finish last, SPI had rated CIV above Greece.

In Group D, all were wrong about Costa Rica. Completely wrong: instead of finishing last, Costa Rica finished 1st in the group! The other team that ultimately advanced from this group was Uruguay which gives the advantage to FIFA and SPI.

In Group E, ELO got it all right. FIFA got the order between France and Switzerland wrong. SPI had Ecuador advancing which was wrong. They were all right about Honduras finishing last.

In Group F, all of the rankings were kind of wrong picking Bosnia as the team to join Argentina in the next round although referee issues certainly played a role in that error. FIFA had Nigeria last while ELO had them last at the time this thread was stated but I believe had moved to right above Iran. All in all, SPI and ELO did better than FIFA, although frankly ELO IMO was the one that gave the most accurate impression of this group.

In Group G, deserving Ghana was correctly picked by all to finish last. They all correctly picked Germany to finish first. All got Portugal over the US, whcih was wrong.

In Group H, FIFA was closest to how things turned out. They had South Korea last (correct) and Belgium first (correct), although they had Russia second and Algeria third while Algeria actually finished 2nd and Russia finished third. SPI correctly picked Belgium first but totally missed Algeria, having them finish last. ELO was the worse in this group as it almost got it all wrong, picking Russia first, Belgium 2nd, S.Korea 3rd and Algeria last.

All in all, the results IMO actually vindicated FIFA and ELO but didn't look good for SPI. All of them of course got some way wrong (Spain and Costa Rica), but in the light of all we know, FIFA had actually done alright. Their ranking of Switzerland and Portugal and Nigeria their biggest failing otherwise. The rest (including some they were all alone such as picking Brazil as barely top 10 and Algeria as in the 20s as opposed to much lower according to ELO and SPI) were more or less within range of what they had said. ELO did alright too but SPI showed that fiddling with what you get from actual results by factoring in subjective criteria such as "talent" and the rest actually weakens the rankings.

 

Interesting, since while I got the impression that FIFA did best, I felt ELO did better than SPI. I will try to use a point system to see how it works out for me, but in the meantime, can you share the point system you used? I like to see how they did in your point system.

 

There are two issues with regard to the rankings. First, whether they did a reasonable job translating the results of the various sides? I agree in that sense no one got Spain wrong. Spain's results justified its ranking. The other issue, however, is to what extent these rankings accurately predict the future? In that sense, with respect to Spain, all got it wrong. But your overall point is still quite valid; there was no basis to rank Spain much lower than they had been ranked before World Cup 2014 got underway.

 

Thanks. I don't think there is a perfect point system, but here is one that I suggest would be helpful for this exercise:

2 points for getting a team's position in the standings right;
1 additional points for correctly predicting a team to either advance or not advance from its group.

 

I have to actually think more about the issue. You might have a valid point about not double counting whether a ranking correctly predicted a team advancing or not.

 

True, but I had mentioned giving 2 points for correctly predicting whether a side would advance (i.e., finish 1st or 2nd), or alternatively not advance (i.e., finish 3rd or 4th). It was not points for just predicting those who advanced. The difference here is basically treating correctly predicting a team finishing 3rd as opposed to 4th less important than correctly predicting that team finishing say 2nd as opposed to 3rd.

 

I think the way teams play makes advancing or not advancing more significant than finishing 3rd or 4th. That affects their tactics and is particularly significant in how the games are played and how much meaning is attached to them when it comes to the last group stage games. A team which must win a match to advance might play differently against an already eliminated team if its objective is to advance as opposed to if its objective was place 3rd or 4th in the group.

However, as I tried to figure out the numbers, I realized that it actually is very important to award negative points as you had suggested. Otherwise, a ranking like SPI would get a lot more points for predicting Group B than either FIFA or ELO in my point system despite the fact that SPI had the group winner (Holland) finish 3rd while ELO rand FIFA had Holland 2nd and Chile 3rd. (They all had Spain 1st). Without negative point, the fact that Chile finished second would give SPI a lot of points while the fact that FIFA and ELO picked Holland 2nd would get them no points in comparison to SPI who had them 3rd.

Put differently, intuitively, I would say FIFA and ELO were more accurate predicting 1- Spain 2- Holland 3- Chile 4- Australia compared to SPI which predicted 1-Spain 2-Chile 3- Holland 4- Australia. But SPI actually gets more points without negative points being awarded for getting a team very wrong.

 

Well according to this site, FIFA's rankings did very well. Certainly, much better than relying on transfermarket values, better than ELO, and better than many other sources. Andrew Yaun, who I am not familiar with, is the only source that beat FIFA in this analysis and even then, by only one better prediction.

http://www.sportingintelligence.com...d-cup-winners-almost-nobody-predicted-160701/