Simulating the Hex

Discussion in 'USA Men: News & Analysis' started by dlokteff, Oct 18, 2016.

  1. dlokteff

    dlokteff Member+

    Jan 22, 2002
    San Francisco, CA
    So here is ESPN SPI qualification odds for Russia:

    99% Mexico
    93% Costa Rica
    60% United States
    52% Panama
    29% Honduras
    6% T&T

    I went ahead and took the current AFC tables and using a point-weighted system came up with a hypothetical AFC opponent. Elo Rating is 1647 (reasonable, UZB is 1660, KSA is 1577).

    Running my sims through I get:

    Mexico = 99.9%
    Costa Rica = 99%
    Panama = 78%
    USA = 69%
    Honduras = 8%
    T&T = 1%

    Obviously something different in SPI evaluation of Panama and Honduras v. my Elo sims. Panama's road win against their direct comp for 4th, and a draw against the #1 seed don't seem to impress SPI much. I also found it strange that SPI had Panama dropping 2% after Tuesdays results and still find the 39% win% for CONCACAF v AFC odd, so I asked Paul Carr on Twitter.

    He responded, Costa Rica and Honduras wins hurt Panama. I don't really but it. Obviously the Hond. win would hurt them, but I'd expect a US win would be worse than CR win for them.

    For the playoff, he said weaker teams are in the 4th spot for CONCACAF (stating the obvious). I asked him directly about a Panama/Uzbekistan matchup and he said Panama is favorite 53%/47% which is perfectly aligned with Elo, so I'm still a bit baffled. Part of it is he has Honduras getting 4th a lot more often, but if they are expected to lose to AFC so badly, why are they getting to 4th in CONCACAF anyway? I also asked why they don't publish SPI anymore and he said "technical issues" and hopes to solve soon.

    Based on my current distribution of 4th place CONCACAF teams, CONCACAF best the hypothetical 1647 Elo AFC rep 55% of the time.

    USA result against the Saudi Uzbekistan Emirates of Australia...

    Advance 64.3% of the time
    • 6 Points - 20%
    • 4 Points - 24%
    • 3 Points, advance via GD - 11%
    • 3 Points, advance via Away Goals - 4%
    • 3 Points, advance via PKs - 2%
    • 2 Points, advance via Away Goals - 2%
    • 2 Points Advance via PKs - 1%
    Eliminated 35.7% of the time
    • 3 Points, eliminated via GD - 7%
    • 3 Points, eliminated via Away Goals - 2%
    • 3 Points, eliminated via PKs - 3%
    • 2 Points, eliminated via Away Goals - 2%
    • 2 Points eliminated via PKs - 1%
    • 1 Point - 14%
    • 0 Points - 6%
     
    beamish, deuteronomy, Footsatt and 9 others repped this.
  2. dlokteff

    dlokteff Member+

    Jan 22, 2002
    San Francisco, CA
    Went ahead and updated the sims based on the intervening changes in Elo ratings, including the AFC results.

    Shouldn't have changed much given the nature of the games since the last Hex matches, and in fact our overall qualification percentage to Russia has remained steady at 69%.

    We do get there in a slightly different manner however. Our qualify Top3% is at 43% (up about 3%), while we finish 4th 39% of the time now (down from 45% from the last run). We improved slightly against the hypothetical AFC opponent, winning that 68% of the time (up from 64%).

    The movement is primarily driven by the results of the Central American Cup (no clue what kind of rosters the teams brought). Costa Rica & Panama under performed there, that's mainly why our Top3% went up. However, Honduras won it, going undefeated. The strengthening Honduran Elo means they take 4th more often, knocking us out entirely a bit more often.

    Elo changes since last run:
    Mexico: +6
    Costa Rica: -69
    USA: 0
    Panama: -18
    Honduras: +47
    T&T: -61
     
    Footsatt, deuteronomy and Dr. Gamera repped this.
  3. Suyuntuy

    Suyuntuy Member+

    Jul 16, 2007
    Vancouver, Canada
    The Central America Cup is a poor indicator, they played with B-teams.
     
  4. dlokteff

    dlokteff Member+

    Jan 22, 2002
    San Francisco, CA
    I figured that. Elo is what it is, and it works pretty well.
     
  5. dlokteff

    dlokteff Member+

    Jan 22, 2002
    San Francisco, CA
    Huge results for us last night.

    Elo sims have us qualifying direct 72% of the time now. Fourth Place 21% and with about 71% chance against AFC, we net out at 87.5%.

    Point and place distribution from one particular 10,000 run:

    3 - Never
    4 - 2 (all 6th)
    5 - 8 (all 6th)
    6 - 20 (5th-6, 6th-14)
    7 - 54 (4th-2, 5th-30, 6th-22)
    8 - 131 (4th-15, 5th-88, 6th-28)
    9 - 203 (4th-65, 5th-123, 6th-15)
    10 - 371 (3rd-15, 4th-225, 5th-126, 6th-5)
    11 - 592 (3rd-87, 4th-406, 5th-96, 6th-3)
    12 - 746 (3rd-244, 4th-448, 5th-54)
    13 - 1112 (2nd-6, 3rd-580, 4th-510, 5th-16)
    14 - 1123 (2nd-36, 3rd-810, 4th-275, 5th-2)
    15 - 1089 (2nd-103, 3rd-848, 4th-137, 5th-1)
    16 - 1394 (1st-1, 2nd-277, 3rd-1072, 4th-44)
    17 - 938 (1st-2, 2nd-365, 3rd-568, 4th-3)
    18 - 778 (1st-5, 2nd-391, 3rd-381, 4th-1)
    19 - 832 (1st-26, 2nd-539, 3rd-267)
    20 - 206 (1st-31, 2nd-136, 3rd-39)
    21 - 265 (1st-38, 2nd-200, 3rd-27)
    22 - 116 (1st-61, 2nd-53, 3rd-2)
    23 - N/A
    24 - 20 (1st-15, 2nd-5)
     
    Mahtzo1, Footsatt, Cornballer and 7 others repped this.
  6. Southern Man

    Southern Man Member

    Jun 14, 2008
    Thanks for updating. I really enjoy these!
     
  7. donovan2dempsey

    Apr 15, 2010
    Club:
    FC Bayern München
    Thanks for posting all this. Any update on what the qualification percentages would look like with a win, draw, or loss tomorrow? Also, Paul Carr's latest % tweet shows us at 65% chance to qualify. Any idea why SPI and ELO have such a huge difference for us?
     
  8. dlokteff

    dlokteff Member+

    Jan 22, 2002
    San Francisco, CA
    Independent of other results and Elo changes:
    USA win (1-0) - 96%
    Draw (1-1) - 88%
    Loss (0-1) - 78%

    Carr has that at 83%, 69%, 50%.

    As for SPI vs. Elo...

    One big difference is SPI seems to like Asia #5 a lot more than I do. SPI has the 4th place CONCACAF team advancing against AFC #5 only 36% of the time. In my sims, CONCACAF goes through 52.5%. I may spend the time to improve my methodology for picking AFC5 if I get real bored. You can see a few posts up, I contacted Paul Carr though Twitter about this. His explanations were unconvincing.

    The other thing is I think SPI just does not rank the USA as highly as Elo. Myriad of reasons I'm sure, but we don't have any view into it (we don't even know what USA's current world rank is).

    Also, seems to like Honduras quite a bit more - my sims they qualify 11% of the time, SPI=24%. Likewise with T&T (2% v. 8%). That's pulling the US% down. I think there's a general higher valuation of those teams by SPI.

    Lastly, Carr may also be running things a little "smarter" than mine. Mine's a massive Excel spreadsheet. Hopefully Carr has some kind of machine learning going on.
     
    deuteronomy and donovan2dempsey repped this.
  9. donovan2dempsey

    Apr 15, 2010
    Club:
    FC Bayern München
    Any update following all of yesterday's results? By the way, the odds for the U.S. from SPI/Carr improved substantially between post-Honduras and pre-Panama, which is a bit odd. He previously noted that a draw would put us at 69%, but then that jumped to 82%.
     
  10. FirstStar

    FirstStar Hustlin' for the USA

    Fulham Football Club
    Feb 1, 2005
    Time's Arrow
    Nat'l Team:
    United States
    I think that may be the result of CR's poor performance - they dropped like a stone this round (of two games). If they have another disaster round, they will be in trouble.
     
  11. PhillyandBCEagles

    Jul 9, 2012
    NC
    Club:
    Philadelphia Union
    They lost at Mexico and got a point at Honduras. This is the weakest Honduras team in a while but they're still not complete pushovers. I don't think CR had a disastrous round by any stretch, just a tough pair of games.

    For our point, we need to win all of our remaining home games and to really be safe we need 4 points from our remaining road games. I think we're in comfortable shape and if we can manage a draw at the Azteca (and take care of business against T&T) we're all but in. Even if not, we should be able to win at either Honduras or T&T.

    In retrospect, getting curb stomped by CR might have been a huge blessing in disguise, especially after we got our GD back in shape against Honduras. Does Klinsmann get the axe if we lose by just 1-2 to the Ticos?
     
    Mahtzo1 repped this.
  12. dlokteff

    dlokteff Member+

    Jan 22, 2002
    San Francisco, CA
    Will be posting in a sec.

    Yep, that's not just odd, that's a fuk up. No reason for it. The 94/82/64 made a hell of a lot more sense.
     
  13. dlokteff

    dlokteff Member+

    Jan 22, 2002
    San Francisco, CA
    If you haven't seen SPI yet, here is that
    MEX=99.9%
    CR=96%
    USA=82%
    PAN=28%
    HON=13%
    T&T=8%

    And mine are converging with one big (and one minor) exception.

    MEX=99.97%
    CR=98%
    USA=89%
    PAN=51%
    HON=11%
    T&T=1%

    We moved up just a point or so. None of the results yesterday changed the % by more than a percent or 2. Our Draw v. Panama was a small net positive (I think mainly Panama dropping 2 at home was a bit more important vs. us getting a point). T&T losing was +, Hond. getting a point was -, and the results in AFC were -, but all tiny moves.

    Not sure why the love for T&T from SPI, but, meh. Given I'm pretty sure he screwed up before, I'm guessing Carr is just wrong about Panama. He got questioned on it by a couple guys and he wrote this...
    @YaManning 25% because that's how it averages out. 45% of HON + 34% of TRI + 17% of PAN, etc. Against 33% of AUS + 18% of KSA, etc.

    17% for Panama, 34% for T&T, 45% for HON as distribution for the 4th spot, makes absolutely no sense. Just no way.

    Anyhoo

    USA point/qualification distribution:

    4 - Nope
    5 - 5 (all 6th)
    6 - 16 (5th-1, 6th-15)
    7 - 38 (5th-11, 6th-27)
    8 - 109 (4th-11, 5th-67, 6th-31)
    9 - 212 (3rd-1, 4th-76, 5th-128, 6th-7)
    10 - 383 (3rd-27, 4th-234, 5th-117, 6th-5)
    11 - 747 (3rd-143, 4th -507, 5th-97)
    12 - 881 (2nd-1, 3rd-386, 4th-468, 5th-26)
    13 - 1013 (2nd-15, 3rd-642, 4th-344, 5th-12)
    14 - 1565 (2nd-54, 3rd-1215, 4th-296)
    15 - 1161 (2nd-171, 3rd-928, 4th-62)
    16 - 1224 (2nd-314, 3rd-888, 4th-22)
    17 - 1378 (1st-4, 2nd-662, 3rd-712)
    18 - 333 (1st-9, 2nd-201, 3rd-123)
    19 - 613 (1st-16, 2nd-458, 3rd-139)
    20 - 249 (1st-44, 2nd-194, 3rd-11)
    21 - N/A
    22 - 73 (1st-32, 2nd-41)
     
    Winoman, Dr. Gamera, Footsatt and 2 others repped this.
  14. Dr. Sneezy

    Dr. Sneezy Member

    Jul 2, 2016
    This is great thanks for doing it

    It is interesting how few points it looks like the US will need to go through

    I'd say a conservative expectation might be 4 points from home and away with T&T, beating Panama at home, and tying Costa Rica at home and Honduras on the road (and losing at Azteca). That would give us 9 more points for 13 total.

    That gives us ~65% chance of direct qualification and <1% change of finishing 5th.

    Even if we end up with 10 pts we have >50% change of making the playoff!

    I suspect this is because the model sees Mexico and CR as likely to get most of their available points
     
    TimB4Last repped this.
  15. EvanJ

    EvanJ Member+

    Manchester United
    United States
    Mar 30, 2004
    Club:
    Manchester United FC
    Nat'l Team:
    United States
    #65 EvanJ, Mar 29, 2017
    Last edited: Mar 29, 2017
    I would give us less than an 89% chance at qualifying. Furthermore, a loss would have put is 4 points behind third, and I would have given us about a 50% chance at qualifying then, whereas you had USA at 78% with a loss. If you change yesterday's game into a Panama win and use my predicted results before the Hexagonal started for the rest of the games, the final standings would be:

    Mexico 25
    Costa Rica 21
    Panama 14
    USA 12
    Trinidad and Tobago 7
    Honduras 5

    I'm not saying your math and ELO aren't accurate, but I disagree with the results.

    You have USA needing 13 points to have at least a 50% chance at the top three. If USA finishes by winning three home games, drawing two away games, and losing at Mexico, USA would finish with 15 points. You have USA with a 94.7 percent chance at finishing in the top three if they get exactly 15 points.

    What we don't want is for the top four (probably USA, Mexico, Costa Rica, and Panama) to have a distribution like there was in qualifying for World Cup 2010, when Costa Rica finished fourth with 16 points.
     
  16. dlokteff

    dlokteff Member+

    Jan 22, 2002
    San Francisco, CA
    Were you considering the playoff in your 50% qualify scenario? Even with a loss we still get 4th most of the time.

    Here's the math now if I change it to 1-0 loss.

    rounding... 46% + 40%*64% = 71%

    It's down from the 78% I posted the other day mainly because I exchanged the Elo points this time (-19 loss vs tie, plus +19 for Pan), and also the AFC results made for a stronger opponent.

    Also 12 points is pretty conservative. I'm assuming you get there with wins vs. PAN & T&T at home, draw with CR @ home, draw@T&T & @HON, lose at MEX. That's historically bad by the US if that happens.
     
  17. TavoElTico

    TavoElTico Member

    Aug 27, 2010
    Costa Rica
    Club:
    Deportivo Saprissa
    Nat'l Team:
    Costa Rica
    Poor? We've had 3 away games and got 4 points out of them. That's golden in Concacaf, which is notorious for the huge homefield advantages. Even a weak Honduran team is not an easy match at 90°F with high humidity.

    If we win all of our home games like we are supposed to, we make it to 19 points and are in with room to spare. Plus, we usually take points away at Panama.

    Next up, Panama and T&T visit Costa Rica. We are looking pretty solid, actually.
     
  18. FirstStar

    FirstStar Hustlin' for the USA

    Fulham Football Club
    Feb 1, 2005
    Time's Arrow
    Nat'l Team:
    United States
    I agree. We has a disastrous first round and a much, much improved second round of the five in the Hex. Just shows that one shouldn't worry too much about early results or draw too many conclusions from small sample size. However, as a matter of statistics, I stand by what I said - the intra-game changes in US qualification probably came mostly from the increased chance of catching CR since CR didn't over-preform and put the US farther out of reach. Prior to the numbers changing, there was still the chance of CR over-performance being factored into our odds. Once that chance went away, our numbers rose. Just statistics.
     
  19. TavoElTico

    TavoElTico Member

    Aug 27, 2010
    Costa Rica
    Club:
    Deportivo Saprissa
    Nat'l Team:
    Costa Rica
    I see, I understand what you meant better now. Take into account though, Mexico and CR, as the most likely qualifying teams, might just end up being irrelevant. So, if both teams win every round that's not vs. the US, we'd actually be boosting your chance in a way.
     
    Eighteen Alpha and Mahtzo1 repped this.
  20. dlokteff

    dlokteff Member+

    Jan 22, 2002
    San Francisco, CA
    @FirstStar , @TavoElTico is correct, the best thing for us in the game last night was for Honduras to drop points. We want to separate from the Catrachos, making it simply a fight between us and Panama for 3rd/4th. Costa Rica is welcome to second, well actually let's hope they take 1st.

    We were almost ambivalent between CR win and tie, though a 1-0 Tico win would raise our chances by about 0.5%. A Honduran win wouldn't have been a big deal, but that was the result we didn't want. Our chance at Russia loses about 3% if Honduras holds that 1-0 lead.
     
    wixson7 and deuteronomy repped this.
  21. Mahtzo1

    Mahtzo1 Member+

    Jan 15, 2007
    So Cal
    "The enemy of our enemy is our friend!"
     
  22. EvanJ

    EvanJ Member+

    Manchester United
    United States
    Mar 30, 2004
    Club:
    Manchester United FC
    Nat'l Team:
    United States
    You were right about my game predictions. If you change all of those predictions to what USA did in their last WCQ home or away against those teams as applicable, it would change two draws (hosting Costa Rica and at Honduras) to a win and a loss and give us 1 more point than I said, which would still have us behind Panama in the hypothetical standings I posted. If Panama had beaten us, they would have 7 points now. If you give them wins hosting Honduras and Trinidad and Tobago and a draw hosting Costa Rica, they would finish with 14 points even if they lost all three away games. If USA had 3 points now, would you give us a 46% chance at getting at least 11 more points to finish third? Our next game is easy, but if we had 3 points now and you give us a win hosting Trinidad and Tobago to make 6 at the halfway point, would we get 8 points in the second half with three away games? If we lost at Panama, we would probably still qualify, but in that case I would make our most likely position fourth. Any rankings may not be a good predictor if a team is doing much better or worse than usual over a few months or a year. We're far enough ahead of Panama, Honduras, and Trinidad and Tobago in ELO that I think we would still be ahead of them if we had lost all four Hexagonal games, but would you expect us to qualify then?
     
  23. dlokteff

    dlokteff Member+

    Jan 22, 2002
    San Francisco, CA
    Yes
     
  24. donovan2dempsey

    Apr 15, 2010
    Club:
    FC Bayern München
    Can we get an update on this for the upcoming round of matches? Thanks!
     
  25. EvanJ

    EvanJ Member+

    Manchester United
    United States
    Mar 30, 2004
    Club:
    Manchester United FC
    Nat'l Team:
    United States
    Here are probabilities using the ELO rating differences and my guess at how often games are draws:

    USA win: 81.7%
    Draw: 9.8%
    Trinidad and Tobago win: 8.5%

    Costa Rica win: 63.0%
    Draw: 19.5%
    Panama win: 17.5%

    Mexico win: 85.3%
    Draw: 7.9%
    Honduras win: 6.8%

    Mexico win: 70.6%
    Draw: 15.7%
    USA win: 13.7%

    Panama win: 53.6%
    Draw: 23.9%
    Honduras win: 22.4%
    It adds up to 99.9% due to rounding.

    Costa Rica win: 83.3%
    Draw: 9.9%
    Trinidad and Tobago win: 7.7%
    It adds up to 99.9% due to rounding.

    For every game I think the draw probability should be higher compared to the probability of the worse team winning. For example, if you agree that Mexico has a 70.6% chance of beating USA, I would make the draw probability about 18.5% and make the USA win probability about 10.9%.
     

Share This Page