Thread for Developing a Statistical System for Determining WC Allocations

[photar74]

Everyone posts about it, everyone writes about it. In fact, the notion that confederation X has “earned” Y number of allocations through play on the field is one of the bigger clichés of this entire board. However, usually whenever someone makes a claim about “deserved” or “earned” allocations, they do one of three things: fail to produce their rationale; fail to follow their rationale all the way through; fail to offer a convincing rationale.

What I hope can happen with this thread is that we can collectively develop a statistical system that would at least come close to determining how many allocations each confederation actually does “deserve.” It is my hope that this system would be transparent to the casual observer, that it would be thorough, and that in so doing it would still contain a minimum number of assumptions.

In order to develop such a system, we would need to answer the following preliminary questions:

1. What data should be included, in terms of time span?
2. What data should be included, in terms of competition?
3. How much, if any, should various portions of the data be weighted?

Since this system seeks only to determine how many spaces each confederation has come to “deserve” strictly through play upon the field, considerations such as population, money, and FIFA politics will not be taken into account.

[photar74]

My preliminary answers to the three starter questions:

The only competitions that should be included in the data are the qualifying and group stages for the ’98 and ’02 WCs. There should be no weighting to the data whatsoever. My rationale is as follows:

-Only WC matches should be included, since the WC is the only competition where every nation in FIFA brings its best team and gives its best effort.
-The qualifying stages of the WC should be used in order to determine confederation depth.
-The WC Finals group stage should be included in measure the relative strength of the confederations when in competition with each other. However, only inter-confederation results from the group stage should be considered (i.e., England vs. Sweden doesn’t factor in).
-Results from only the past two WCs should be used, since they were the only WCs with 32 teams and the same system all around (they were even the same number of teams from every confederation both times). In the future, only results from the previous two WCs should be used so as not to value the results from one WC more than another.

[photar74]

Since it is inevitably brought up by various camps (its AFC’s entire “rationale” in their campaign for 5.0), I wish to make the case for why the knockout stages of the WC should not be included in the data for measuring confederations:

-The WC finals knockout stages should not be used since there is no possibility for a draw, and therefore is a statistically incompatible with other results.
-Results from the WC finals knockout stage should also not be used because, if results are measured based upon the result after regulation thereby allowing for draws, advancement can actually reduce a team’s relative value to its confederation. For example, in ’02 South Korea reached the semifinals, though went into extra time in both the round of sixteen and the quarterfinals before losing to Germany in regulation. Thus, South Korea managed 0.67 points per match in the knockout stages if results are measured after regulation. Both England and the USA, however, won their round of sixteen matches in regulation and lost their quarterfinal matches in regulation, for an average of 1.50 points per match, despite not advancing as far in the competition as South Korea.
-Results from the WC finals knockout stage should further not be used because of the preponderance of intra-confederation matches in the round of 16. What this means is, for example, that for Spain, England, and the USA their quarterfinal match was the first match that counted for their confederation. This is obviously unfair for the confederations involved, as it actually punishes a confederation for having two of its teams play each other in the knockout stages.
-Yet still further, the clear advantage that host nations have always drawn in the WC inevitably skews results from the knockout stage in favor of the hosting confederation.
-Lastly, results from the knockout stages should not be used because of the “Brazil factor.” Just because Brazil runs, or nearly runs the table in the WC does not make Bolivia any better of a team. The confederation with the champion will inevitably have results wildly skewed in its favor, especially if that confederation is small like CONMEBOL (10 teams).

Otherwise, at this point, I’ll wait for other responses.

[Sachin]

Actually, if a match goes to a penalty kick shootout to determine a winner, it is recorded as a draw. For example, Brasil and Italy drew in the 1994 World Cup final. Brasil won the Cup on the basis of the PK shootout.

Intra-confederational friendlies do have a place in this ranking system. Granted not every country brings their A side to every match, but over time it evens out so the average is maybe a B or B+ side.

But any statisical system fails to take into account the political horse-trading that goes on within FIFA. And if you think that any system will get rid of that, then you are being naive.

Sachin

[photar74]

I don't in any way think that there would be a system to somehow supplant or circumvent FIFA politcs. On the CONCACAF board and elsewhere, I have repeatedly urged people to stop discussing performance on the field as reasons why one confederation or another will recieve more or less allocations for '06. I am perfectly aware that politics and money has almost everything to with it, and that on the field performance has relatively little.

This thread is meant to develop a hypothetical system that would allow us to discuss what confederations do actually "deserve" what. There is nothing naive about my position on allocations. The thread is meant to be hypothetical (and wishful), nothing more.

[RichardL]

The big problem is that meetings between different confederations happen so rarely (i.e. every 4 years) that any statistic can get skewed by just one or two odd results. Would it be right to increase the South American allocation just because there was a Brazil v Argentina final, even if the other South American teams were mediocre? What if Australia came in and did well. The stats could show that Oceania, based on some statistical weighting, now deserve three spots.

It'll never happen, but the only fair way is to have a global qualification tournament, perhaps 16 groups of 6 or something like that. Of course you'd then have to argue how many from each confederation get to be among these 96 teams in the final qualifying phase..........

[Fender playa]

Originally posted by RichardL
The big problem is that meetings between different confederations happen so rarely (i.e. every 4 years) that any statistic can get skewed by just one or two odd results. Would it be right to increase the South American allocation just because there was a Brazil v Argentina final, even if the other South American teams were mediocre? What if Australia came in and did well. The stats could show that Oceania, based on some statistical weighting, now deserve three spots.

It'll never happen, but the only fair way is to have a global qualification tournament, perhaps 16 groups of 6 or something like that. Of course you'd then have to argue how many from each confederation get to be among these 96 teams in the final qualifying phase..........

Thats easy..........pick the best 96 teams in the world..........:D

Yah........yah..........i know its not that easy;)

[photar74]

Originally posted by RichardL
The big problem is that meetings between different confederations happen so rarely (i.e. every 4 years) that any statistic can get skewed by just one or two odd results. Would it be right to increase the South American allocation just because there was a Brazil v Argentina final, even if the other South American teams were mediocre? What if Australia came in and did well. The stats could show that Oceania, based on some statistical weighting, now deserve three spots.

It'll never happen, but the only fair way is to have a global qualification tournament, perhaps 16 groups of 6 or something like that. Of course you'd then have to argue how many from each confederation get to be among these 96 teams in the final qualifying phase..........

Yeah--but can't we TRY to develop a statistical system, at least for our own use?

Anyone? Anyone?

[Nobby]

photar74 for FIFA president!

How does this sound? I've seen versions of this system posted from time to time here at BigSoccer.

After each WC: find the six lowest six ranked teams. Count the numbers from each confederation. For each number subtract one. Subtract result from current allotment to get the new allotment.

Here are two examples using five instead of six since an OFC team didn't reach the finals.

Lowest five in FRANCE 1998 were:

Saudi Arabia
Bulgaria
South Korea
Japan
USA

by confederation that's:
afc 3
uefa 1
concacaf 1
conmebol 0
caf 0

which means:
AFC loses two. 4 - 2 = 2 next time.
UEFA, stays the same. 14 - 0 = 14.
CONCACAF, stays the same. 3 - 0 = 3.
CONMEBOL, 5 + 1 = 6.
CAF 5 + 1 = 6.

These numbers mean we would've seen Columbia and another African team like Liberia or Morocco instead of say, China and Slovenia at the 2002 world cup. This assumes Europe surrenders one to Asia so it can have a qualifying round.

Lowest five in KOREA/JAPAN 2002 were:

France
Tunisia
Slovenia
China
Saudi Arabia

UEFA 14 - 1 = 13 in 2006.
CAF 5 - 0 = 5
CONMEBOL 5 + 1 = 6
CONCACAF 3 + 1 = 4
AFC 4 - 1 = 3

With this system its feasible that if Australia avoids finishing in the lowest six then two Oceania teams will reach the finals next time. Go Fiji! It also seems like it could erode Europe's large number. It will also be harsh on Asia which has a history of sending poor teams to the World Cup. Theoretically, all ten South American teams could make the finals!

I like this system because its simple, self-correcting and severe. It only uses whole numbers. It is promotion and relegation and it'll make some of those late group games more meaningful and fun to watch. Teams eliminated after two games will have something to play for.

[photar74]

Originally posted by Nobby
photar74 for FIFA president!

How does this sound? I've seen versions of this system posted from time to time here at BigSoccer.

After each WC: find the six lowest six ranked teams. Count the numbers from each confederation. For each number subtract one. Subtract result from current allotment to get the new allotment.

Here are two examples using five instead of six since an OFC team didn't reach the finals.

Lowest five in FRANCE 1998 were:

Saudi Arabia
Bulgaria
South Korea
Japan
USA

by confederation that's:
afc 3
uefa 1
concacaf 1
conmebol 0
caf 0

which means:
AFC loses two. 4 - 2 = 2 next time.
UEFA, stays the same. 14 - 0 = 14.
CONCACAF, stays the same. 3 - 0 = 3.
CONMEBOL, 5 + 1 = 6.
CAF 5 + 1 = 6.

These numbers mean we would've seen Columbia and another African team like Liberia or Morocco instead of say, China and Slovenia at the 2002 world cup. This assumes Europe surrenders one to Asia so it can have a qualifying round.

Lowest five in KOREA/JAPAN 2002 were:

France
Tunisia
Slovenia
China
Saudi Arabia

UEFA 14 - 1 = 13 in 2006.
CAF 5 - 0 = 5
CONMEBOL 5 + 1 = 6
CONCACAF 3 + 1 = 4
AFC 4 - 1 = 3

With this system its feasible that if Australia avoids finishing in the lowest six then two Oceania teams will reach the finals next time. Go Fiji! It also seems like it could erode Europe's large number. It will also be harsh on Asia which has a history of sending poor teams to the World Cup. Theoretically, all ten South American teams could make the finals!

I like this system because its simple, self-correcting and severe. It only uses whole numbers. It is promotion and relegation and it'll make some of those late group games more meaningful and fun to watch. Teams eliminated after two games will have something to play for.

Thank you for the wonderful comments nobby!

I enjoyed your system, although I feel that measuring confederation strength only by its weaker teams in WC group play focuses too much on one aspect of a confederation. As perhaps the most glaring example, why should UEFA lose a spot for France finishing 28th, and not gain any when they won the title?

Sachin, I'm not sure if you are still reading this thread, but I do not feel that you need to include inter-confederation (intra would mean two teams from the same confederation). There is still a reasonable amount of data, even if you only use group stages from the previous two WCs:

-CONCACAF would have 18 matches to determine its strength
-AFC would have 24 matches to determine its strength
-CAF and CONMEBOL each would have 30 matches to measure their strength
-UEFA would have 62 matches to measure its strength

If you were using any of these totals to measure the strength of one team, certainly it would be enough.

Perhaps, however, if there was a generally agreed upon need to include more data, only certain friendlies could be considered. For example:

--WC warmups, where both teams have qualified. These would be acceptable, since players would be performing at their highest level, and otherwise competing for starting time.
--The Confed Cup. Hey, at least the teams are competing for something.
-Round robin friendly tournaments, such as the US Cup. Again, at least they are competing for something.

I'd like to hear other thoughts on whether more data than just WC finals and qualifying should be considered (oh yeah--inter-confederation playoffs should definately be included, I think). For me, however, friendlies just don't seem right. Its too random.

[photar74]

In case anyone is interested, here are the records of all six confederations in inter-confederation WC qualifying / finals play over the past two WCs. I am ranking by points per match. I am including playoff qualifiers, although I am not including knockout stage results:

UEFA: 28-20-16, 104 points, 1.625 per match (21 teams)
CONMEBOL: 13-10-9, 49 points, 1.53125 per match (7 teams)
CONCACAF: 6-5-7, 23 points, 1.2777 (repeating) per match (4 teams)
Austra—I mean OFC: 1-2-1, 5 points, 1.25 per match (1 team)
CAF: 6-12-12, 30 points, 1.00 per match (6 teams)
AFC: 6-6-16, 24 points, 0.857142857142 (repating) per match (5 teams)

How does one move from these stats—which are the core of what I would like to use as I feel they are most accurate measure of confederation strength—to figuring out how many spaces each confederation “deserves?” I really don’t know.

[photar74]

Using only the above figures, simple cross multiplication leads to the following numbers of deserved spaces for each confederation (32 teams host withdrawn from host confederation, 31 teams host added to host confederation)

UEFA: 11.375 (17.76, 17.20)
CONMEBOL: (5.58, 5.40)
CAF: (3.13, 3.03)
CONCACAF: (2.66, 2.58)
AFC: (2.23, 2.16)
OFC: (0.65, 0.63)

Yuk. What does this tell us? For one, that OFC deserves 1.0 far more than AFC deserves 5.0, and even more than CONCACAF deserves 3.5. For another, it tells us that outside of UEFA, CONMEBOL and OFC, all confederations are yet to “earn” their current allocations (although CONCACAF and OFC are closer than CAF and AFC). However, surely, a UEFA with Germany + (16.76 or 17.20) spaces is nauseating to everyone, as is a CONMEBOL with 5.58 spaces. So, since we’re being hypothetical, let’s assume that UEFA is capped at 16.0 spaces including Germany (as many teams as the Euros, and never more than half or the overall total), and CONMEBOL is capped at 5.0 spaces (only half of the teams from the confederation advance to the finals). Now, let’s see, based on the figures from the previous post, how many of the 11.0 spaces remaining the other four confederations each deserve, once again using cross multiplication:

CAF: 2.00 (3.97)
CONCACAF: 1.703704 (3.38)
AFC: 1.426572 (2.83)
OFC: 0.41667 (0.83)

In such a scenario, if all allocations were rounded off to the nearest 0.5, there would be a total of 11.5. Rather than OFC, CONCACAF, and AFC fighting amongst themselves for the 0.5, they would team up and strip it from CONMEBOL.

[photar74]

With all of this in mind, here are three sets of allocations:

Allocations One: Perhaps the truly deserved allocations for ‘06
UEFA: Germany + 17.0
CONMEBOL: 5.5 (playoff with Oceania champion)
CAF: 3.0
CONCACAF: 2.5 (playoff with AFC third place)
AFC: 2.5 (playoff with CONCACAF third place)
OFC: 0.5 (playoff with CONMEBOL sixth place)

Allocations Two: Minimum politics
UEFA: 15.0 + Germany
CONMEBOL: 4.5 (playoff with CONCACAF 4th place)
CAF: 4.0
CONCACAF: 3.5 (playoff with CONMEBOL fifth place)
AFC: 3.0
OFC: 1.0

Allocations Three: What will happen for ‘06
UEFA: 13.0-13.5 + Germany
CAF: 5.0
AFC: 4.5-5.0
CONMEBOL: 4.0-4.5
CONMEBOL: 3.0-3.5
OFC: 0.5-1.0

Maybe this just tells us what we already knew. Every confederation, except CAF, is getting the screw job by AFC.

However, maybe it doesn’t tell us enough. Maybe more data needs to be included in the analysis, such as friendlies, the confed cup, and even intra-confederation WC qualifying (to determine confederation depth better than the above model).

Anyway, I think this is a start.

[RichardL]

I think if you are going to do something like this then you need to remove the top teams (who pretty much always qualify anyway) from the equation. There's no logical reason why success or failure for Brazil & Argentina has any relevence to South America having 4,5 or 6 spots in the World Cup. What the Argument really is are the likes of Paraguay, Ecuador, Uruguay & Bolivia more deserving than Iran, China, Saudi Arabia, Australia, Costa Rica, Honduras, Australia, Bulgaria, Czech, Austria, Egypt, Tunisia etc etc. Really you need to be looking at the teams who just qualified, say the 4 UEFA play-off winners, 2 or 3 from South America and 1 or 2 from the other confederations.

[cosmosRIP]

This is where the teams eliminated in the 1st rd. of the last 2 WCs where from and the order in which they finished:

*2002*
17. Africa(2)
18. S.America(3)
19. CONCACAF(3)
20. Africa(3)
21. Europe(10)
22. Europe(11)
23. Europe(12)
24. S.America(4)
25. S.America(5)
26. Europe(13)
27. Africa(4)
28. Europe(14)
29. Africa(5)
30. Europe(15)
31. Asia(3)
32. Asia(4)

*1998*
17. Europe(11)
18. Africa(2)
19. Europe(12)
20. Asia(1)
21. S.America(5)
22. CONCACAF(2)
23. Europe(13)
24. Africa(3)
25. Africa(4)
26. Africa(5)
27. Europe(14)
28. Asia(2)
29. Europe(15)
30. Asia(3)
31. Asia(4)
32. CONCACAF(3)


Clearly Asia is over represented with 4 teams. Looking at how well S.America(5) has done, shouldn't there be a S.America(6)?

[photar74]

Originally posted by RichardL
I think if you are going to do something like this then you need to remove the top teams (who pretty much always qualify anyway) from the equation. There's no logical reason why success or failure for Brazil & Argentina has any relevence to South America having 4,5 or 6 spots in the World Cup. What the Argument really is are the likes of Paraguay, Ecuador, Uruguay & Bolivia more deserving than Iran, China, Saudi Arabia, Australia, Costa Rica, Honduras, Australia, Bulgaria, Czech, Austria, Egypt, Tunisia etc etc. Really you need to be looking at the teams who just qualified, say the 4 UEFA play-off winners, 2 or 3 from South America and 1 or 2 from the other confederations.

The problem I have with this is that, if you were to only focus upon the teams who "just qualified," then thepoor performance of teams such as Ecuador or Uruguay would reduce the chances of a team like Brazil or Argentina qualifying.

Anyway, how much worse did the four teams you listed from SA really do compared to the other teams you listed, even over the past three WCs? (Again, I'm using group play and inter-confed qualifiers, byut not knockout stages):

Paraguay, Uruguay, Ecuador, Bolivia: 3-5-6, 14 points, 1.00 points per match, (4 teams, 5.0 spaces over three WCs)

Iran, China, Saudi Arabia, Australia, Costa Rica, Honduras, Bulgaria, Czech, Egypt, Austria, Tunisia 8-13-24: 37 points, 0.82 per match, (11 teams, 12.75 spaces over the past three WCs)

Its not clear in any way, even by only looking at the teams you mentioned, that CONMEBOL deserves fewer spaces.

I still don't feel that examining only the weaker teams in a confederation is accurate, however. I especially do not feel that examining teams that "just qualified" is accurate. Germany, Turkey, Belgium and Ireland all qualified through playoffs, just like Slovenia. It certainly doesn't tell us much about their performacne in the WC, however, as the first four all finished ahead of group winners such as Russia, Portugal, Poland and Croatia.

Heck, Ecuador finished ahead of Brazil in the most recent round of qualifying.

[RichardL]

The teams I listed I just plucked out of the air to be honest. Brazil & Argentina always qualify. They may not always come first and second in qualifying, but they are probably so complacent about qualifying that the don't give 100% all the time. The point I was trying to make was that theoretically Brazil & Argentina could be amazing, third and fourth south americans team could be average and the rest absolute garbage, but because the top teams are doing so well, middle ranked (and garbage in this case) teams would be getting qualification slots they don't deserve.

Basically the top teams may improve/decline but there's no reason for this change to be reflected in the average teams in each confederation.

[photar74]

Originally posted by RichardL
The teams I listed I just plucked out of the air to be honest. Brazil & Argentina always qualify. They may not always come first and second in qualifying, but they are probably so complacent about qualifying that the don't give 100% all the time. The point I was trying to make was that theoretically Brazil & Argentina could be amazing, third and fourth south americans team could be average and the rest absolute garbage, but because the top teams are doing so well, middle ranked (and garbage in this case) teams would be getting qualification slots they don't deserve.

Basically the top teams may improve/decline but there's no reason for this change to be reflected in the average teams in each confederation.

This is entirely possible. Certainly, it even happened to a certain extent during this past WC, when the gap in AFc performance was vast and gaping.

Overall, I think I understand you complaint, but identifying two teams, Argentina and Brazil, as somehow above any qualifying system is a mistake. Sure, those two teams, as well as Italy and Germany, may appear to be locks to qualify, but no one's space is really assured (Netherlands in '02, England and France in '94). This is why I believe that the performance of every single team from a confederation in the WC should be euqally weighted, no exceptions. Sure, Brazil and Argentina almost always perform better than every team in CONMEBOL, but they are still in CONMEBOL, and I think that the confederation's measured performance should reflect that.

On a side note, I should add that my previous calculations can only be viewed as accurate relative to current allocation spaces. I'll try to think of a way to remove that bias.

[Nobby]

I enjoyed your system, although I feel that measuring confederation strength only by its weaker teams in WC group play focuses too much on one aspect of a confederation. As perhaps the most glaring example, why should UEFA lose a spot for France finishing 28th, and not gain any when they won the title?

Thanks for reading. I was thinking the same thing about France. My best answer is this: UEFA does in fact gain since if France is #1 then France is not #31. In the example, UEFA broke even because Bulgaria finished in the low five. The intent of the system is to distribute failure evenly to all participating confederations.

[condor11]

Originally posted by RichardL
The teams I listed I just plucked out of the air to be honest. Brazil & Argentina always qualify.
actually only brazil always qualifies
argentina has missed some wc