One is silver and the other’s gold
Posted on December 16, 2013 9:35 am
Almost no sooner did Bill favorably namecheck me in comparison to Nate Silver than his formula over at ESPN is used to cheer up American soccer fans everywhere. Apparently the United States has just shy of a 40% chance to escape from its group – odds of the USMNT being subsequently pounded into hamburger in the second round have not yet been released.
Absent the formula, the reasoning boils down to “Without Cristiano Ronaldo, Portugal ain’t that sexy.” It’s an extremely tempting line of thinking for us to take, but it omits a few things.
One is obvious – what doth it profit a team to beat Portugal but lose to the others? Ghana is also in the group, and the US Soccer Federation has been a wholly owned subsidiary of Ghana since 2006. Getting a point against Germany would be a significant upset all by itself – this group is asking for three separate surprising results.
Ah, but according to the ESPN International Power Rankings!, those would not be upsets, would they? The United States is in better form than, well, all but a half dozen teams in the world.
Which brings us to the real problem – the entire premise of Power Rankings!, especially in the World Cup. Nate Silver’s political predictions are based on aggregations of polls, and are usually revealed gospel. Not coincidentally, for his political predictions he has forests’ worth of information. This isn’t the place to discuss his methodology, but the keys are the sheer amount of data available, the ability to check the sources of that information against each other for accuracy, and the comparative safety of that profession’s historical assumptions. Silver goes against conventional wisdom a lot of the time, but he never strays far from statistical margin of error to do so.
Meanwhile, there’s this sport called baseball, which also produces mountains of data, and people started seriously analyzing said data to that sport. This captured the popular imagination, and before you knew it people were trying to apply these tactics to other sports. One thing leads to another, and all of a sudden we’re favored over Portugal and Ghana.
Except that’s not based on anything nearly as reliable as polls, or even on-base percentage. It can’t be. The relevant numbers don’t exist, and can’t exist. The numbers you do put in are guesswork.
I don’t know ESPN’s formula, and I don’t know FIFA’s coefficient for its team rankings. I’m not saying I don’t need to, because that makes me sound like an innumerate troglodyte. But let’s say we were designing a system to rank national teams and predict the outcomes of World Cup groups.
What information would we use? Recent World Cup results? Record in qualification? Recent form in friendlies? Club form of important players? Highest-paid players? Of course all of these are flawed in some way, so we weigh them…but in what proportions?
The last time the US played Portugal, it was such a mismatch that even Jeff Agoos wasn’t able to affect the outcome. This is the sort of thing you’d think would be extremely useful as an object lesson – the United States had plenty of World Cup experience at the time, certainly more than the 2002 Portugal team, which had not recently qualified for the finals.
So, recent World Cup experience trumps nearly everything else. In qualifying, Portugal didn’t lose a game, while the US, among other missteps, botched a freaking home game to Honduras that I’m still mad about. The Portuguese players were (and are) vastly better paid than the US indentured servants, and Luis Figo used to be a pretty big deal.
Except all that recent World Cup experienced availed us nothing against Ghana and the Czech Republic four years later.
Too small a sample size? Well, let’s see how the teams in the best confederation with the best record actually did, aside from Portugal. The group winners for UEFA in 2001 qualification were: Russia, Poland, Sweden, Denmark, England, Portugal, Croatia, Spain and Italy. Five of those teams made it to the second round in 2002. Two made it to the quarterfinals.
The group winners for UEFA in 2005 qualification were: Holland, Ukraine, Portugal, France, Italy, England, Serbia, and Croatia, plus Germany as hosts. Six of those teams were quarterfinalists in 2006, and four of them were semifinalists.
The group winners for UEFA in 2009 qualification were Denmark, Slovakia, Switzerland, Germany, Spain, England, Serbia, Italy, and Holland. Only five of them would be alive for the quarterfinals…but three of them made it the semifinals, and two of them were finalists. Except the two finalists had never won the tournament before, and neither had even been to the final game since 1978.
Meanwhile, in South America…Argentina won CONMEBOL in 2001, Brazil won in 2005 and 2009. Guess which World Cup Brazil won in that sequence, and guess which tournaments they crashed out of. Uruguay finished fifth in CONMEBOL all three times in that sequence, and had to qualify through playoffs. They made no impact on the finals…until all of a sudden they did in 2010.
All this is still a meager sample size, so you can’t use that to conclude anything. Except (1) all this really happened and (2) these are the sort of results you use to put in a computer program to predict future results. Silver and ESPN are using these formulae to predict group results in the course of six games, as opposed to say, voting trends of tens of millions of people. I’d be interested to see what happens when you plug in Silver’s formula to “predict” the 2002 World Cup. Bet we don’t have a 40% chance to get out of the group.
I haven’t even mentioned intangibles like injuries and coaching. The nearest equivalents in politics would be campaign managers and gaffes, but because we’re talking about groups of 23 people, and not tens of millions, their impact on events is probably greater, certainly a lot more difficult to chart. You would THINK dropping John Harkes wouldn’t have a greater impact than dropping Eric Cantona, but there you go.
Most of us are familiar with what happens when numbers are entered into a computer to rank teams based on a sport with few games and a high turnover of players. We know it as the college football Bowl Championship Series, and it’s being junked as of a year from now.
You can be as optimistic or as pessimistic as you want – but please don’t tell me we have any specific percentage of getting out of the group.
(Edited because I am also an illiterate troglodyte and I forgot a couple of paragraphs.)