Using Elo Ratings to Predit the Hex

Discussion in 'USA Men: News & Analysis' started by Maximum Optimal, Oct 20, 2012.

  1. EvanJ

    EvanJ Member+

    Manchester United
    United States
    Mar 30, 2004
    Club:
    Manchester United FC
    Nat'l Team:
    United States
    That makes sense. I decided to look at the Bet365 odds for some 2012-2013 UEFA Champions League qualifying round games to compare games between close clubs and games where one club was much better:

    One club much better: Anderlecht-Ekranas (Anderlecht won 11-0 on aggregate)
    At Anderlecht, the expected points were 2.67 for Anderlecht and 0.23 for Ekranas (Anderlecht 2.44 higher) and at Ekranas the expected points were 2.33 for Anderlecht and 0.49 for Ekranas (Anderlecht 1.84 higher) which means the expected point differentials were 0.60 off.

    One club much better: Basel-Flora Tallinn (Basel won 5-0 on aggregate)
    At Basel, the expected points were 2.66 for Basel and 0.24 for Flora Tallinn (Basel 2.42 higher) and at Flora Tallinn the expected points were 2.49 for Basel and 0.36 for Flora Tallinn (Basel 2.13 higher) which means the expected point differentials were 0.29 off.

    Clubs about equal: Spartak Moscow-Fenerbahce (Spartak Moscow won 3-2 on aggregate)
    At Spartak Moscow, the expected points were 1.88 for Spartak Moscow and 0.87 for Fenerbahce (Spartak Moscow 1.01 higher) and at Fenerbahce the expected points were 0.98 for Spartak Moscow and 1.75 for Fenerbahce (Spartak Moscow 0.77 lower) which means the expected point differentials were 1.78 off.

    Clubs about equal (according to the UEFA coefficients but the odds though Lille was much better): Lille-Copenhagen (Lille won 2-1 on aggregate)
    At Lille, the expected points were 2.26 for Lille and 0.55 for Copenhagen (Lille 1.71 higher) and at Copenhagen the expected points were 1.69 for Lille and 1.02 for Copenhagen (Lille 0.67 higher) which means the expected points differentials were 1.04 off.

    The mean of 0.60 and 0.29 is 0.445 for the very unequal clubs, which is much small than the mean of 1.78 and 1.04 which is 1.41 for the about equal clubs. I used expected points so that each club would have one value for each game rather than comparing probabilities for wins, draws, and losses.
     
  2. Reccossu

    Reccossu Member+

    Jan 31, 2005
    Birmingham
    Ok. I don't know anything about programming, but I wrote a monte carlo simulation of the hex to see what would happen.

    I found a plot online that shows the expected chance of a given result, given a certain difference in ELO (accounting for a home advantage of 100 points). I painstakatingly plotted it in xcel to get a best fit equation (R-squared over .99). I used that equation in a basic program loop to simulate each hex game.

    I counted up how many times each team finished in 1-3rd, 4th, and 5-6th.

    I determine in the program if there are relevant ties and break them randomly. (Or I believe the program does this).

    Results:

    Team-------------- Direct Q %-------------- Playoff %----------------- Out%

    Mexico: ---------------89----------------------- 2-------------------------- 9
    USA: ------------------64------------------------14----------------------- 22
    Panama:------------- 43-------------------------21------------------------35
    Honduras: ------------40------------------------22------------------------38
    Costa Rica: -----------35-------------------------21------------------------43
    Jamaica: -------------27-------------------------20------------------------53

    Putting aside that I may have screwed this up, it looks very reasonable. I wish the 64 were higher though.
     
  3. Maximum Optimal

    Maximum Optimal Member+

    Jul 10, 2001
    It would be great if you saved your program and updated the odds as we go through each round of the Hex.
     
  4. Reccossu

    Reccossu Member+

    Jan 31, 2005
    Birmingham

    I will do that. And I can run it again if the ELOs change before the hex starts.
     
  5. blacksun

    blacksun Member+

    Mar 30, 2006
    Seoul, Korea
    Club:
    San Jose Earthquakes
    Nat'l Team:
    United States
    Can you post the equation?
     
  6. dlokteff

    dlokteff Member+

    Jan 22, 2002
    San Francisco, CA
    Where did you find the "plot online that shows the expected chance of a given result." I'm curious as to how Wins, Losses, and Draws are distributed since the Elo doesn't explicitly give the outcome.

    It's results are similar to my Sim above, but yours is painting a less rosy picture for the favorites. I think that may be because I assumed more points would come from wins relative to draws. If you assume the "average" case, it gives you a lot of draws (and I think correspondingly a lot of ties to break). For example, a Sim using that assumption yielded this result for USA @ PAN: Win = 19.9%, Draw =53.1%, Loss = 27.0%, compared to a sim with my assumptions of Win = 33.0%, Draw = 27.9%, Loss = 39.1%. Both calculate very closely to the Elo Win Expectancy, but the avg. points is quite different given the 3 for a win and one for a draw. I think I see the liklihood of us winning 1/3 of the time as higher than >50% of the matches ending in a draw, but maybe not.

    Anyway, My Direct Qual %s were:
    MEX = 97.8%
    USA = 73.3%
    PAN = 34.2%
    HON = 32.5%
    CR = 27.5%
    JAM = 21.2%

    I'm choosing to believe mine because I will sleep better with 73% chance vs. 64%:) :unsure:
     
  7. Maximum Optimal

    Maximum Optimal Member+

    Jul 10, 2001
    Could we use the results of the last three or four Hexes to estimate the probability of a draw and proceed from there.
     
  8. EvanJ

    EvanJ Member+

    Manchester United
    United States
    Mar 30, 2004
    Club:
    Manchester United FC
    Nat'l Team:
    United States
    Each Hexagonal has 30 games. Here is how many draws they have had:

    2009: 5
    2005: 4
    2001: 6
    1997: 13

    Average: 7
     
  9. Reccossu

    Reccossu Member+

    Jan 31, 2005
    Birmingham
    Sure. Have to get back to my office.
     
  10. Reccossu

    Reccossu Member+

    Jan 31, 2005
    Birmingham

    I just did a search and found a paper that had a graph.

    For nearly equal strength teams, the equations predicts draws in the 25-28% range.

    Breaking ties on points made a big difference. The US, for example, finished tied for third or better about 72% of the time, but finished in sole third or better only 64%. Ties for third happend about 15% of the time and ties for fourth happened at close to the same rate.

    I would image that higher ELO teams are a little more likely to have a goal differnece advantage, but I did not break ties that way, so my version may be a little pessimistic -- but not a lot.

    Unless, of course, I have a bug in the code.
     
    Editor In Chimp repped this.
  11. Reccossu

    Reccossu Member+

    Jan 31, 2005
    Birmingham
    I will add a line to count up all the draws just to see if the equation I am using is close to this at all.
     
  12. Reccossu

    Reccossu Member+

    Jan 31, 2005
    Birmingham
    My sim predicts an average of 6.13 draws, so pretty well in line with the history.
     
  13. Reccossu

    Reccossu Member+

    Jan 31, 2005
    Birmingham
    Here's the equation I used for an away win.
    0.000000001*elodiff^3 + 0.0000008*elodiff^2 - 0.0011*elodiff+0.2729
    home win
    -0.000000001*elodiff^3-0.00000009*elodiff^2+ 0.0012*elodiff+0.5069
     
    blacksun repped this.
  14. az2004

    az2004 Member

    Jun 5, 2012
    i see the 22% of usa being out as being high....usa needs to be 5 or 6 to be kicked out, a 4 place finish, while not crtainty, i'd give usa a 70% chance tehre

    is the probability of a 5th or 6th place finish possible
     
  15. az2004

    az2004 Member

    Jun 5, 2012
    these odds look far more realistic:

    MEX = 97.8%
    USA = 73.3%
    PAN = 34.2%
    HON = 32.5%
    CR = 27.5%
    JAM = 21.2%

    the original ones for jamaica and costa rica seemed a bit too high

    i remeber the espn odds, being closer to the ones here
     
  16. EvanJ

    EvanJ Member+

    Manchester United
    United States
    Mar 30, 2004
    Club:
    Manchester United FC
    Nat'l Team:
    United States
    Those odds add up to only 2.865 teams finishing in the top three.
     
  17. Reccossu

    Reccossu Member+

    Jan 31, 2005
    Birmingham
    I would like to think so too, but based on those ELO's the US would be an underdog in 5 of it's ten matches. That tells me the chance of being 5 or 6 is higher than we might typically think. This is a strong and even Hex.
     
  18. Reccossu

    Reccossu Member+

    Jan 31, 2005
    Birmingham
    If the ELO's mean anything then it's hard to imagine that since the US is only 80 point ahead of PAN and HON that they would be more than twice as likely to qualify than those teams.

    Maybe the ELO underrrates the US. (Although recent play suggests its not far off to me). I could back calculate an ELO that gives the US a 73% chance of going through. We could look and see what teams in the world have that ELO and then decide if we feel like wthe US is more like that team than the current ELO reflects.
     
  19. az2004

    az2004 Member

    Jun 5, 2012
    garbage in garbage out

    how close was the semis usa to the hex usa

    the real questyion ids WHAT willthe real usa be...with BRADLEY and DONOVAN

    if DONOVAN plays....

    my basic question is why did espn have usa a solid lock, is their model better than the one here..

    clearly it is different...

    agree, odds add up to 286 or so, but i was piscking that prediction from one above

    maybe the current data says something

    but expected performance is something else
     
  20. dlokteff

    dlokteff Member+

    Jan 22, 2002
    San Francisco, CA
    Yep. I must have looked at the wrong line on my spreadsheet from Panama on...

    My correct odds should have been:
    MEX = 98.3%
    USA = 73.3%
    PAN = 37.7%
    HON = 35.8%
    CR = 30.7%
    JAM = 24.1%

    Sorry about that. Anyway, I'm working on implementing Reccossu's equation's into my Sims. I do think those are yielding pretty reasonable win/draw splits (gets a bit funky at times, but...). Assuming those are based on some real data, I'll go with those, and I like the continuous nature of the functions.

    I do think my old assumptions were yielding pretty reasonable draw%. For the USA we average about 2.75 draws per Hex. In the past 4 we have had, 2,1,2, and 5 draws, for a 2.5 average. Mexico in my sims is at about 2.4 draws, and they have also averaged averaged 2.5 (although 6 of the 10 draws came in the 1998 cycle with only 4 since). Costa Rica averages about 2.8 draws in my sims, but have only averaged 1.75.

    So I think the equations (based on the posts above) will bring down the # of draws a bit, which might mirror reality a bit more. Costa Rica's lesser draw % I believe is influenced by Saprissa, and their stronger home field advantage than Elo believes. They've out performed the Elo win expectancy at home by quite a bit over the last 4 Hexes - 0.80 actual vs. 0.635 expected. Speaking of which, I'm also working on making a home field adjustment change to the sims, but will post my results with the equation implementation first, and then add that a bit later.
     
    SiberianThunderT and Reccossu repped this.
  21. Reccossu

    Reccossu Member+

    Jan 31, 2005
    Birmingham
    Sure. If the ELOs don't seem accurate to you, then they will not predict results that you agree with.

    The US would have about a 73% chance of direct qualification if all the other ELOs stayed the same in the Hex but the US moved up from 1744 to 1784. That would put the US about on par with Japan and Chile instead of Denmark. I can buy that, but the ELO says the results haven't been seen on the field to justify that ranking. Maybe the Hex US will outperform the semis US.

    Of course that's why they play 'em!
     
  22. Suyuntuy

    Suyuntuy Member+

    Jul 16, 2007
    Vancouver, Canada
    Any rating system has several basic drawbacks:

    a) Injuries (or any other reason why a regular player expected to be there must be taken out)
    b) Appearance of new talent (which is unpredictable; Williams, Johnson, Shea and Gomez may be causing this effect for the USA already)
    c) Inconsistency (Panama comes to mind --if they are ON, they are IMO the 3rd best CC-CAF team; but just as easily they can have a dismal showing, yet)
    d) Tactical issues (the most important, and why sometimes a very weak team can surprise a much stronger one; so far, JK is keeping par at best, IMO)
    e) Psychological factors (vital as well; big teams often intimidate others just with their colors)

    Personally, and considering the Elo Ratings as they are, expect the USA to exceed expectations --in great part due to b), but also due to e).

    I also expect Panama to underperform, due to c)

    I'd expect it to finish MEX, USA, HON, PAN, CRC, JAM. PAN should beat the Kiwis though.
     
  23. Reccossu

    Reccossu Member+

    Jan 31, 2005
    Birmingham
    The ELO sim says that the US's direct qual %age goes from 64% to 69% if Guatemala had qualified. Curse you Clint Dempsey!
     
  24. az2004

    az2004 Member

    Jun 5, 2012
    i have mex, usa, hon, pan, cr, jam too

    depth of talent over 10 matches, gives mex and usa my edge...

    and hon over pan and cr

    battle for 3,4,5 will go to the last days..

    i don't give jam a chance due to lack of depth

    BUT IF PERECEIVED DEPTH does not paly out, 3 thru 6 can go in any order
     
  25. az2004

    az2004 Member

    Jun 5, 2012

Share This Page