1st Time Ever: CONCACAF Club Ratings

[voros]

Well I finally sucked it up and did the work on it, though the results are mixed. The results are through the Apertura in the Latin American leagues and through the 2005 MLS Cup in the USA. The leagues included are the top divisions in Costa Rica, Mexico, USA, Honduras, Guatemala and El Salvador, and results are taken over the last three years with more recent results weighted more heavily than early results.

The main problem with the system is that it uses the CONCACAF Champions cup as the means to integrate the leagues with one another. We know that not everybody takes that tournament as seriously as others do, and a bigger problem is the lack of matches between the teams in those leagues leading to sample size problems. One way I addressed that is by going back 5 years in the CCC, with the 4th and 5th years being weighted much much less, but being enough to help with rating the leagues against one another. Anyway, because of these weaknesses, the system probably overrates a few of the Central American powerhouses relative to the teams in Mexico and the US who don't necessarily care as much about it (or at least didn't in the past).

The rankings:

1. Club America, Mexico
2. Pachuca, Mexico
3. Toluca, Mexico
4. Olimpia, Honduras
5. Monterrey, Mexico
6. Tigres, Mexico
7. Alajuelense, Costa Rica
8. Saprissa, Costa Rica
9. Cruz Azul, Mexico
10. Herediano, Costa Rica
11. Jaguares, Mexico
12. Chivas, Mexico
13. Necaxa, Mexico
14. Santos Laguna, Mexico
15. Atlas, Mexico
16. Morelia, Mexico
17. Atlante, Mexico
18. Pumas, Mexico
19. Tecos, Mexico
20. Marathon, Honduras
21. Dorados, Mexico
22. Real Espana, Honduras
23. Veracruz, Mexico
24. Perez Zeledon, Costa Rica
25. Puntarenas, Costa Rica
26. Cartagines, Costa Rica
27. Houston 1836, United States
28. Brujas, Costa Rica
29. New England, United States
30. Victoria, Honduras
31. D.C. United, United States
32. Kansas City, United States
33. Santacrucena, Costa Rica
34. Santos de Guapiles, Costa Rica
35. Municipal, Guatemala
36. San Luis, Mexico
37. Chicago, United States
38. Platense, Honduras
39. Hispano, Honduras
40. MetroStars, United States
41. Los Angeles, United States
42. Municipal Valencia, Honduras
43. Columbus, United States
44. Vida, Honduras
45. Universidad, Honduras
46. Colorado, United States
47. Motagua, Honduras
48. Dallas, United States
49. Carmelita, Costa Rica
50. FAS, El Salvador
51. Firpo, El Salvador
52. Comunicaciones, Guatemala
53. Vista Hermosa, El Salvador
54. Isidro Metapan, El Salvador
55. Liberia, Costa Rica
56. Xelaju, Guatemala
57. Aguila, El Salvador
58. Ramonense, Costa Rica
59. Marquense, Guatemala
60. Balboa, El Salvador
61. Suchitepequez, Guatemala
62. Alianza, El Salvador
63. Once Municipal, El Salvador
64. San Salvador, El Salvador
65. Coban, Guatemala
66. Heredia, Guatemala
67. Chalatenango, El Salvador
68. Jalapa, Guatemala
69. Antigua, Guatemala
70. Real Salt Lake, United States
71. Chivas USA, United States
72. Petapa, Guatemala

In terms of average team quality they rank:
1. Mexico
2. Costa Rica
3. Honduras
4. United States
5. El Salvador
6. Guatemala

[scaryice]

For the USA and Mexico, are you counting the postition the teams finished in the table or in the playoffs?

Also, a minor thing, but officially Houston is a different team than San Jose (like the Baltimore Ravens).

[voros]

For the USA and Mexico, are you counting the postition the teams finished in the table or in the playoffs?
All games, whether in season or post-season, count the same.

[voros]

Also, a minor thing, but officially Houston is a different team than San Jose (like the Baltimore Ravens).
It's the same players, the same coach and the same franchise. Considering that, there's no point in removing them from the list.

[numerista]

The main problem with the system is that it uses the CONCACAF Champions cup as the means to integrate the leagues with one another. We know that not everybody takes that tournament as seriously as others do, and a bigger problem is the lack of matches between the teams in those leagues leading to sample size problems.

Good stuff. I'm sure Chivas will be comforted that they avoided last place in something. ;)

In addition to "taking it seriously," there may also be the following ...

a) Different countries have different home/road swings in performance
b) MLS teams suffer due to their seasonal cycle (longer off-season, often with heavy turnover, games played during pre-season, or at the very start of the season).

Have you looked into adding parameters for these effects? I suspect that "a" would be useful in national team ratings, as well, although I can't remember ever implementing it.

[voros]

Good stuff. I'm sure Chivas will be comforted that they avoided last place in something. ;)

In addition to "taking it seriously," there may also be the following ...

a) Different countries have different home/road swings in performance
b) MLS teams suffer due to their seasonal cycle (longer off-season, often with heavy turnover, games played during pre-season, or at the very start of the season).

Have you looked into adding parameters for these effects? I suspect that "a" would be useful in national team ratings, as well, although I can't remember ever implementing it.
I really haven't found too much with applying standard home roads for everybody. It's easy to measure the problems if this were off, but it doesn't seem to be off to any noticeable degree. The only difficulties come in specific individual matches like say Chivas versus the Galaxy and equivalents in other countries.

b) is, in my opinion, impossible to do without simply rigging the system to give you the results you think ought to be the case. Unless I have enough examples of teams playing in and out of season and those examples demonstrate a clear and quantifiable trend, it's going to be hard to account for in the mathematical system. As such I think it's best letting, for now, the system do its things and then make the mental adjustments from there until I can come up with a better way.

One thing these systems do well is give an expected result for each match, so we can separate out specific types of matches and see if their significant discrepancies between expected and results for those types. I'm always looking out for those sorts of things.

It's also worth noting that due to the sample size problems mentioned, if the Revs and Galaxy both get by their Costa Rican opponents, expect the ranking of _all_ MLS teams to climb dramatically.

[khucke]

Voros,
could you give a little more details about your methods when comparing clubs from different leagues. I ask this, because I did something similar comparing the various European leagues and clubs.
(Search for my method in posts like Innovative Virtual European Football either in this section or in the UEFA section)

[numerista]

I really haven't found too much with applying standard home roads for everybody. It's easy to measure the problems if this were off, but it doesn't seem to be off to any noticeable degree.

FWIW, I checked the home-road swing on MLS teams in the Concacaf Cup, 2002-05, and it came out to an average of 3 goals per series (57 goals in 19 series, which should be close to right if not exactly). If both teams' home fields contributed equally to that advantage, that suggests a home field advantage of 1.5 goals per game.

In the 2005 MLS season, the home field advantage was 61 goals in 192 games, a little over 0.3 goals per game. In the 2005 Costa Rican Apertura, the home field advantage was a virtually identical: 30 goals in 96 games, a little over 0.3 goals per game. But when Costa Rican teams face MLS teams, somebody's home field advantage goes through the roof.

This information doesn't suggest an obvious correction (to me, at least), but the assumption that both teams benefit equally is a very strong one.

[voros]

This information doesn't suggest an obvious correction (to me, at least), but the assumption that both teams benefit equally is a very strong one.
Right but because in almost all situations in the CCC each team plays one at home and one on the road, the overall effect on the ratings will probably be minimal anyway (IE no teams are likely playing an unusually high percentage of road or home games). There just isn't a good easy solution, and ultimately I doubt it effects the rankings much. If I eliminate all home field advantage the ratings come out close to the same.

More important adjustment in international games in my opinion.