2018 Simulation: Ratings, Ranks, and NCAA Tournament Bracket

Discussion in 'Women's College' started by cpthomas, Aug 17, 2018.

  1. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    #51 cpthomas, Oct 29, 2018
    Last edited: Oct 29, 2018
    ALERT!

    The NCAA's RPI reports, issued this morning, are based on a data base that has a number of errors and two important missing games, so their rankings are not correct. The ratings/rankings at the All White Kit website, however, are correct as will be mine when I publish them later today. At the linked AWK webpage, to put the teams in ARPI ranking order, simply click on the "Adjusted RPI" column heading twice.

    The missing games are 10/28 Florida State 1 v Duke 0 neutral site and 10/28 @ Old Dominion 4 v North Texas 0.

    The errors are:

    10/26 SEMO 0 v Belmont 0 neutral site is in their data base twice.

    10/28 @ Penn 2 v Brown 0 is in their data base twice.

    10/28 @ Stony Brook 0 v Northeastern 2 is in their data base, but there was no such game.

    10/28 @ UNI 0 v Evansville 0 (Evansville advancing on PKs) was reported into their system as an 0-1 win for Evansville when it should have been reported as a 0-0 tie. (This happens occasionally during conference tournaments.)
    Since these problems have ripple effects, some of them significant, the NCAA's RPI rankings are pretty messed up, including in areas related to potentially seeded teams and at large selections.

    I've sent info about these problems to the NCAA staff, so ultimately they all should get corrected. If the Women's Soccer Committee members rely on the NCAA's RPI reports this week as they get ready for their selection process next weekend, however, they're going to be replying on information that isn't correct. Those of you who have contacts with Committee members might want to let them know about the problems.
     
    MiLLeNNiuM repped this.
  2. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    ALERT #2!

    Regarding the issues mentioned in my previous post, the NCAA advises they are working on some system issues and plan to repost their RPI reports later this afternoon.
     
    MiLLeNNiuM, L'orange and socalsoccer23 repped this.
  3. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    I just have posted my weekly actual ARPI rankings and simulation updates, for games through Sunday, October 28, at the Blogspace:

    2018 RPI REPORT 10.29.2018

    2018 SIMULATED RPI RANKS 10.29.2018

    2018 SIMULATED CONFERENCE STANDINGS AND CONFERENCE TOURNAMENTS 10.29.2018

    2018 SIMULATED NCAA TOURNAMENT BRACKET 10.29.2018

    In addition, at the RPI for Division I Women's Soccer website, on the

    NCAA Tournament: Predicting the Bracket, Track Your Team

    page, I've put an attachment at the bottom of the page that will help you see the details of how each team fares in terms of the NCAA Tournament, within the simulation. The title of the attachment for this week is 2018 Website Factor Workbook 10.29.2018. If you're a coach or a fan, you can use this to see how your team's simulated record matches up with a whole series of the Women's Soccer Committee's seeding and at large selection decision-making patterns over the last 11 years. The webpage itself includes an explanation of the attachment.

    My current RPI rankings (and ratings) match the NCAA's corrected rankings and ratings published earlier this afternoon, after the NCAA resolved the problems with its rankings and ratings published earlier this morning.

    I hope those who follow the rankings and ratings appreciate the effort the NCAA staff puts in to get the rankings and ratings exactly right. Especially at this time of year, there's a certain amount of "herding cats" that the staff must do, mixed in with all the other tasks assigned to them. They work really hard.
     
    MiLLeNNiuM and L'orange repped this.
  4. L'orange

    L'orange Member+

    Ajax
    Netherlands
    Jul 20, 2017
    I hope the NCAA appreciates your work backstopping their work, CP!
     
  5. Carolina92

    Carolina92 Member

    Sep 26, 2008
    Looks like there are still some issues. The NCAA RPI is wildly over ranking North Texas.
     
  6. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    That's a function of the RPI, not of any data or programming problems. One more week and we'll see where they end up.
     
  7. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    On looking at this from a statistical perspective, it's probably not as "wildly" as you think, especially for practical purposes. They provide a good case study, so I've spent a little time on it:

    The RPI currently has North Texas at #37. (If they win the Conference USA tournament, my simulation has them rising to #28. If they don't, they'll probably fall below #30.)

    My modification to the RPI (Iteration 5 RPI), which significantly reduces the RPI's discrimination against strong conferences and in favor of weaker ones, has them at #47.

    Massey's ratings, which fully eliminate the RPI's discrimination, has them at #56. In my opinion, his ratings are the best.

    All three rating systems are close to equal in terms of how well their ratings, adjusted for home field advantage, correlate with actual game results.

    These may seem like big differences, but if you understand what statistical rating systems can and can't do, they aren't. The NCAA stats experts themselves have said that it takes about 20 ranking positions to show a real difference between teams -- although it depends on where you are in the rankings. At the top and at the bottom, it takes fewer positions. In the very middle, it takes more positions.

    From a Women's Soccer Committee historical perspective, #30 RPI is the cut-off point for teams being assured of getting an at large selection to the Tournament. #57 is the cut-off point beyond which teams can be assured of not getting an at large selection. So that puts North Texas in the bubble, whichever rating system you use. If they win their conference tournament, they'll likely be inside #30 and would get an at large selection, but they won't need it because they'll be an Automatic Qualifier. If they don't win their conference tournament, they'll likely be outside #30 and thus a bubble team.

    Once in the bubble, history says the Committee moves on to other considerations: head-to-head results in relation to teams potentially in the bracket, results against common opponents in relation to those teams, conference strength and the team's results within its conference, the team's results against "strong" teams and in particular against very highly ranked teams, and so on. The poorer the team's RPI, the better the team needs to do in those other considerations. If you look at North Texas' schedule and results with those other considerations in mind, it looks to me like if they don't win the CUSA tournament, they won't get an at large selection.

    So, in terms of the purpose for which the RPI is intended, I don't think North Texas is "wildly" overrated. They're probably overrated, but not to the extent that it will make a practical difference.
     
    hykos1045 repped this.
  8. Carolina92

    Carolina92 Member

    Sep 26, 2008
    Very interesting. Thanks for this. Just FYI - the NCAA website has North Texas at #28 right now.
     
  9. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    As noted in a couple of posts above, the NCAA had trouble Monday morning with its ratings. They were missing some games; and they had some other gremlins causing problems. It looks like what's posted at the website is an early version of their rankings. Regarding North Texas in particular, those rankings are missing North Texas' loss to Old Dominion on Sunday.

    If you use this link, you'll get to the NCAA's Nitty Gritty report published at the NCAA's RPI Archive. It has the NCAA's corrected rankings based on games played through last Sunday. They match mine and All White Kit's exactly.
     
    MiLLeNNiuM repped this.
  10. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    I just have posted an updated simulated NCAA Tournament bracket, based on the actual results of all games played through yesterday (Friday, November 2) and simulated results of today's and tomorrow's games, at the Blogspace:

    2018 SIMULATED NCAA TOURNAMENT BRACKET 11.3.2018

    At the moment, it looks like the Women's Soccer Committee's decisions are going to be difficult, controversial no matter what they decide, and very interesting.
     
    MiLLeNNiuM repped this.
  11. Crazyhorse

    Crazyhorse Member

    Dec 29, 2007
    Nat'l Team:
    United States
    Thanks cp, always enjoy your input and great information. Chris Henderson just put out hypothetical pods and he has 3 seeded ACC teams in the same pod, gosh I hope that does not happen.
     
  12. mpr2477

    mpr2477 Member

    Jun 30, 2016
    Club:
    Vancouver MLS
    Well done CP. I completely agree with ur top 16 seeded teams
     
  13. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    I just have posted the final simulated NCAA Tournament bracket, based on the actual results of all games played through the entire season, at the Blogspace:

    2018 FINAL SIMULATED NCAA TOURNAMENT BRACKET 11.4.2018

    As I suggested previously, it looks like the Women's Soccer Committee's decisions for both seeding and at large selections are going to be difficult, controversial no matter what they decide, and very interesting.
     
    McSkillz and socalsoccer23 repped this.
  14. Gilmoy

    Gilmoy Member+

    Jun 14, 2005
    Pullman, Washington
    Nat'l Team:
    United States
    Here's the bracket, compared to the predictions. There were some ... surprises!

    I realize that the non-seeded 5-16 teams are, indeed, not seeded, so other factors such as minimizing travel override ARPI. Nonetheless, we can list all 4 brackets as if they were seeded. Deviations from a smooth ARPI curve thus indicate the binding strength of the strong travel force over short distances :D

    IIRC, the overall top seeds are 1.Stanford 2.Florida State 3.North Carolina 4.Georgetown, so that's how I number their regions. Predicted tier 6.5 is abbreviated 'x'.

    r sd P b ARPI = region, actual seed, predicted tier, bid (aQ or at-larGe), ARPI
    ================
    1 01 1 Q 001 Stanford
    1 02 3 G 009 TennesseeU
    1 03 4 G 010 TexasA&M
    1 04 6 G 025 BostonCollege
    1 05 x G 036 WisconsinU
    1 06 6 G 019 TCU
    1 07 6 G 029 ArizonaU
    1 08 6 G 046 Clemson
    1 09 6 G 039 MississippiU
    1 10 5 Q 085 Denver
    1 11 5 Q 042 BYU
    1 12 5 Q 014 Memphis
    1 13 5 Q 063 Hofstra
    1 14 5 Q 033 NorthTexas
    1 15 6 G 052 Louisville
    1 16 5 Q 152 Seattle

    Generally, the AQs (tier 5) are in the bottom half, and 1-16 roughly follow ARPI.
    3 of 4 seeds outperformed their predicted tiers.
    Wisconsin made it as a tier-6.5 team.

    r sd P b ARPI = region, actual seed, predicted tier, bid (aQ or at-larGe), ARPI
    ================
    2 01 1 Q 002 FloridaState
    2 02 2 Q 006 WestVirginiaU
    2 03 4 G 018 SouthCarolinaU
    2 04 2 G 008 SouthernCalifornia
    2 05 5 Q 027 LSU
    2 06 6 G 022 PennState
    2 07 6 G 031 WakeForest
    2 08 6 G 016 SouthFlorida
    2 09 5 Q 096 Albany
    2 10 x G 040 OhioState
    2 11 5 Q 075 BowlingGreen
    2 12 5 Q 090 BostonU
    2 13 5 Q 037 LongBeachState
    2 14 5 Q 073 UNCGreensboro
    2 15 5 Q 059 Radford
    2 16 5 Q 114 LoyolaChicago

    Southern California got devalued a lot.
    Ohio State made it in from the bubble.

    r sd P b ARPI = region, actual seed, predicted tier, bid (aQ or at-larGe), ARPI
    ================
    3 01 1 G 003 NorthCarolinaU
    3 02 3 G 007 UCLA
    3 03 3 G 012 SantaClara
    3 04 2 G 013 TexasU
    3 05 4 G 026 ArkansasU
    3 06 6 G 020 NCState
    3 07 6 G 028 Auburn
    3 08 6 G 023 KansasU
    3 09 5 Q 041 StLouis
    3 10 5 Q 044 MinnesotaU
    3 11 7 G 047 NorthwesternU
    3 12 5 Q 117 UALR
    3 13 6 G 043 VirginiaTech
    3 14 5 Q 089 Milwaukee
    3 15 5 Q 128 SanJoseState
    3 16 5 Q 237 Howard

    UCLA rose, Texas and Arkansas sank.
    Northwestern made it from tier 7 :eek: (and crushed many bubble hearts)

    r sd P b ARPI = region, actual seed, predicted tier, bid (aQ or at-larGe), ARPI
    ================
    4 01 2 Q 004 Georgetown
    4 02 1 G 005 Baylor
    4 03 4 G 015 VirginiaU
    4 04 3 G 011 Duke
    4 05 6 G 017 MississippiState
    4 06 6 G 034 TexasTech
    4 07 6 G 021 Vanderbilt
    4 08 6 G 032 WashingtonState
    4 09 5 Q 166 MontanaU
    4 10 5 Q 064 MurrayState
    4 11 5 G 024 Princeton
    4 12 5 Q 076 Lipscomb
    4 13 6 G 030 Rutgers
    4 14 5 Q 053 Monmouth
    4 15 5 Q 067 AbileneChristian
    4 16 5 Q 098 CentralConnecticut


    The four seeds did two pairs of flip-flops.

    And finally, the snub-ees, in ARPI order:
    7 G 35 GeorgeMason
    x G 38 Butler
    x G 45 ArizonaState
    6 G 48 Pepperdine
    6 G 49 ColoradoU
    6 G 50 IllinoisU
    7 G 51 Providence
    7 G 54 OklahomaState
    7 G 56 VCU
    7 G 57 OregonU
    7 G 58 NebraskaU
    7 G 60 Samford

    Some mild shocks on this list.
    Wisconsin, Ohio State, and Northwestern beat out all of these teams.
     
    L'orange repped this.
  15. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    I'll provide some detailed comments on the Committee's decisions later, but did a quick look this afternoon. For at large selections, contrary to some opinions, I don't think the Committee did anything out of line. The one that surprised me was Northwestern, but once I looked to see the details of why my simulation didn't have them in the group of candidate teams for the last open slots, I got a pretty good idea of what the Committee might have been thinking -- Northwestern probably has Mississippi State, NC State, and Wake Forest to thank.

    On seeding, I think the only really questionable seed is putting Southern California at a #4 seed rather than a #2. Everything else was within the range of what my system said were possible seeds.
     
  16. Illini_Fan

    Illini_Fan New Member

    Illinois
    United States
    Nov 5, 2018
    Shouldn't conference play count for something in Northwestern's case? Even if they look good statistically missing the tournament should have been a big penalty.
     
  17. Glove Stinks

    Glove Stinks Member+

    Jan 20, 2014
    Club:
    Chelsea FC
    I amusing myself by saying the committee got the 2 USCs confused
     
    cpthomas repped this.
  18. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    #68 cpthomas, Nov 6, 2018
    Last edited: Nov 6, 2018
    While I'm getting my thoughts together about the Committee's decisions, here are some interesting "trivia" from the conference tournaments:

    1. There were 164 conference tournament games. Of those:

    The better seeded team advanced by win or PKs in 106 or 64.6% of the games.

    The poorer seeded team advanced in 35.4% of the games.
    Taking into consideration that in a good number of those games the better seeded team had home field advantage, it seems that being the better seeded team in a game doesn't mean a whole lot in terms of which team is going to win.

    2. Of the 164 tournament games, 23 went to PKs. That's about 14.0%, which is a little higher than the average of 10.7% ties for all regular season games. Of the 23 that went to PKs:

    The better seeded team advanced in 12 and the poorer team in 11.

    The team with the better RPI advanced in 10 games and the poorer team in 13.
    This is a small data sample, but it suggests what many suspect, which is that if a game goes to PKs, seeds and rankings go out the window, it's roughly 50-50 as to who will win.
     
    Crazyhorse and MiLLeNNiuM repped this.
  19. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    I just have posted my final end-of-season actual ARPI rankings, conference standings, and conference tournament results, at the Blogspace:

    RPI REPORT 11.5.2018 (END OF REGULAR SEASON)

    2018 CONFERENCE STANDINGS AND CONFERENCE TOURNAMENTS 11.4.2018 (ACTUAL FINAL)

    I previously posted my final simulated NCAA Tournament bracket.

    In addition, at the RPI for Division I Women's Soccer website, on the

    NCAA Tournament: Predicting the Bracket, Track Your Team

    page, I've put an attachment at the bottom of the page that will help you see the details of how each team fared in terms of the NCAA Tournament, within the simulation. The title of the attachment for this week is 2018 Website Factor Workbook 11.4.2018. If you're a coach or a fan, you can use this to see how the details of your team's final record match up with a whole series of the Women's Soccer Committee's seeding and at large selection decision-making patterns over the last 11 years. The webpage itself includes an explanation of the attachment and how to use it.
     
    L'orange and Crazyhorse repped this.
  20. Sockers1

    Sockers1 Member

    Nov 7, 2016
    Club:
    Manchester United FC
    Good info CP! Noticed North Texas went from an RPI of 28 to 33, when you look at the bracket, with an RPI that high, why did they draw Texas A&M the three seed with an RPI of 10? To me the committee should have sent North Texas 30 miles down the road to play TCU - RPI of 19 and BYU who is at 42 play Texas A&M. Your thoughts?
     
  21. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    Sorry, but I don't have any expertise in where the NCAA assigns teams. So far as I know, they select the teams that will get to host first round games and then have their travel computer program assign teams to the host sites so as to minimize total travel costs. Maybe it has something to do with airfares from different travel team sites to the game sites.
     
    stubifier repped this.
  22. stubifier

    stubifier Member

    Real Salt Lake
    United States
    Jan 19, 2018
    Beyond the seeded teams (the 1 and 2 seeds clearly draw the weakest teams in the field every year), travel seems to be the most likely variable in determining match ups. But maybe they also look at other factors? North Texas and BYU have VERY different profiles, so maybe (to muse on Sockers1's question) they scrutinize criteria that aren't so directly related to rpi, such as results against top teams?
     
  23. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    I just have completed and posted a detailed analysis of the Women's Soccer Committee's decisions on #1 seeds for the NCAA Tournament. If you're interested, you can find it here. Teaser: The decision to give Florida State a #1 seed may have set the table for the Committee to give at large selections to teams that finished far down in the regular season standings of highly ranked conferences.

    Slowly but surely, I'll be posting analyses of the other seeds and at large selections.
     
  24. Glove Stinks

    Glove Stinks Member+

    Jan 20, 2014
    Club:
    Chelsea FC
  25. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    I just have completed and posted a detailed analysis of the Women's Soccer Committee's decisions on #2 seeds for the NCAA Tournament. If you're interested, you can find it here. Teaser: As with the Committee's giving Florida State a #1 seed, the #2 seeds appear to indicate that the Committee assigned a lot of weight to a team's conference's rank and not much weight to where the team finished within the conference. This looks to be more of setting the table for later Committee decisions.
     

Share This Page