Use of the RPI for Division I Women's Soccer

Discussion in 'Women's College' started by cpthomas, Jan 25, 2008.

  1. kolabear

    kolabear Member+

    Nov 10, 2006
    los angeles
    Nat'l Team:
    United States
    Ah, well, you're looking at it from Portland's point of view, that is a little different because beating BC is definitely no easy task for most teams in the country.

    You'd get that 3rd RPI element, true, the Opponents' Opponents' win percentage of the other ACC teams, which is bound to be pretty good; but as Craig P and cpthomas have pointed out, that element doesn't count for as much as you might think glancing at the formula.

    More important is the won-loss record of the opponent (in this case we're talking about BC or Miami) - and, I don't know, the ACC is one of those conferences where they beat up on each other. I was thinking of the Duke example again. Next year that could be BC or Miami with a record only slightly better than .500 --- Risk and reward? If I was Machiavelli I would say the RPI offers much better risk vs reward deals than the ACC teams, but I guess I'd have to spend a lot more time calculating the OOWPs, or whatever we call it (Opponents' opponents' winning pct).

    If you're thinking about Portland, maybe you shouldn't be so worried about the RPI. You're going to make it into the tournament anyway. And I think I read somewhere (here or maybe on Pilot Nation) that the NCAA handbook does NOT provide for the use of RPI when it comes to the seedings, only for its use in determining who gets an at-large selection.
     
  2. Cliveworshipper

    Cliveworshipper Member+

    Dec 3, 2006

    Actually we lost to Denver at altitude the first championship year, now that I think of it. First or second game of the season. started out 0-2.

    I was just thinking from Up's point of view 'cause that's what I know.

    I think if you reconstruct the seedings (you need to fill in where the NCAA didn't), after the second round the seedings were in lock-step with the RPI, so it does matter. before that there's the travel mess.

    Here at UP, unless the NW pac 10 teams get better, we're traveling the first two rounds by the current system, and the NCAA will lose $50K in gate receipts.

    you're right, there are probably more guaranteed wins than the bottom of the ACC, but the odds would certainly be better than UNC. you don't get any points in the first element if you lose.

    I'll bet those crafty Stanford math people talked to their AD.....
     
  3. Cliveworshipper

    Cliveworshipper Member+

    Dec 3, 2006

    I'll bite.


    I think it's a much better predictive tool.

    Here at UP, Jones has predicted the wins/losses in the tournament for UP 4 years running. maybe others have different experiences.


    It has it's flaws, too, but its pretty good at pointing out the flaws in the RPI.

    People don't like the carry-over from year to year, but it DOES make it more accurate in predicting. There is such a thing as knowing how to win.
     
  4. kolabear

    kolabear Member+

    Nov 10, 2006
    los angeles
    Nat'l Team:
    United States
    I think there's some misconception abut the carry-over from past seasons in the Albyn Jones system. I don't know for sure but I think it's worth mentioning although, mind you, I'm no authority on how these rating systems work.

    There is a carryover effect but it may not be what people think. Take Duke for instance. People may think that Duke's rating benefits largely from 2006's rating, which serves as starting point for last season. The thinking seems to go, if it wasn't for that high starting point, Duke would've had a much lower rating because of its "mediocre" 8-5-6 regular season.

    Not true - from what I understand of how these Elo-like (chess-rating) systems work. The system is set up to adjust or recalibrate (or maybe "normalize") after a certain number of games. In other words, you play like a chump for 7 or 8 games, your rating is going to start looking like the rating of a chump.

    Let's say Duke, which started with a rating of about 1680, played 12 games in a row against teams rated 1500, winning only 4 and losing 8. That's a winning percentage of only .333 which is the expected result for a team rated 1400 playing against opponents rated 1500.

    I don't know Albyn Jones' recalibration or "normalization" setting (in some formulas known as the k-factor or k-coefficient), but I think I'm pretty safe in saying that Duke's rating at that point would be somewhere between 1400 and 1470. In other words, at least 75% of the difference between Duke's initial rating of 1680 and 1400 (it's performance rating for 12 games) would be erased. (I guess I'm saying this because I can't imagine that the k-coefficient here would be more than 16 games; 12/16 = 75%)

    Duke's rating actually went up last season, from around 1680 to around 1720 (end of regular season). Instead of being dragged down by its "mediocre "season, Duke's rating was actually pulled up. Because of its strength of schedule, Duke's performance in 2007 was actually superior to 2006, according to the A_J method.

    More important to the "carryover" are the ratings of the opponents, which are used to calculate how you're doing this year. For instance, that first team Duke plays that's rated 1500 - what if, like Duke, they're allegedly not the same team as last year and are having their rating propped up by the A-J system and by guys with a "bi-coastal, big conference" bias like me? What if they're more like a 1300 team this year?

    Well, that's possible. But you can see what's different: we're going to say that, not only about Duke, and not only about Duke's first opponent, but all 10 of Duke's opponents? And all of their opponents on whom their 1500 rating is based?

    Not likely, is it?
     
  5. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    My understanding from Jones is that Kolabear's assessment is correct. The carryover effect is very minimal by the end of the season. For the 2006 season, as a favor, he ran his end of the regular season calculations with the carryover effect completely eliminated. As I recall (although I haven't gone back and checked) the only significant difference in the ratings that count -- in other words, the top bunch -- was that Navy's rating was significantly higher without the carryover effect. If you remember, Navy had a great record that year. As I recall, Navy didn't do that well in the tournament, and nowhere near as well as the "no carryover" rating would have indicated. My understanding is that the benefit of the carryover effect, in large part, is to compensate for the lack of enough data in any single year to accurately inter-relate teams that play in different playing "pods," particularly when a team that plays in a "pod" such as Navy does has an outstanding record. According to Jones, the data show that teams just don't change that fast from year to year. It takes time for a team to rise through the ranks. The carryover effect adds that into the equation.

    Of course, there always is the possibility that the carryover effect will hurt a team that truly has had a very unusual rise to the top of the heap. On the other hand, the lack of a carryover effect allows teams that have excellent records in weak "pods" to rank ahead of teams with lesser records in strong "pods." This latter problem is the one the RPI has, and my work indicates it is much more of a problem than the rare problem the carryover effect might cause.:confused:

    I don't suggest, at least this point, that the NCAA not use the RPI. My interest is in getting the Women's Soccer Committee to understand the problems the RPI has and take them into account in its decision-making.

    On the question of whether the Committee uses the RPI for seeding purposes. If you read the Tournament Manual carefully, it does not say how the Committee is to do the seeding. However, the manual also discusses the information that is available to the Committee: team records, head-to-head results, results against common opponents, and the RPI. Here are the seeds for this years tournament, compared to the teams' RPI (unadjusted) rankings. The adjustments move teams in this group at most three positions, but typically only one or two, so the unadjusted rankings will be very close to the adjusted ones:

    Seed Position RPI Rank

    1. North Carolina 1
    2. UCLA 2
    3. Penn State 3
    4. Stanford 5
    5. Texas A&M 6
    6. USC 7
    7. Portland 4
    8. Purdue 10
    9. Georgia 8
    10. Tennessee 9
    11. Florida 11
    12. Florida State 17
    13. Wake Forest 13
    14. West Virginia 12
    15. Virginia 15
    16. Notre Dame 14

    (San Diego held the #16 spot in the RPI, but did not get a seed, a result with which I cannot disagree.)

    Perhaps it is a coincidence that the seeds follow the RPI so closely, but that seems hard to believe.
     
  6. kolabear

    kolabear Member+

    Nov 10, 2006
    los angeles
    Nat'l Team:
    United States
    I'm very curious about this! For some reason or another, I've always assumed that when Albyn Jones ran the rating program with all teams starting with the same rating, they still used the old ratings as the opponent factor.

    So that, if you played and tied, for example, Portland (2006 final rating: 1948) in your first game, you'd get a much bigger boost to your rating than if you tied Oregon St (2006 final rating 1541). I thought (or assumed) this aspect of the previous year's rating had to carryover for the ratings to be accurate.

    But maybe not?

    If the Albyn Jones ratings without any carry-over are pretty close to their standard method, I'm more than ready to suggest that the NCAA scrap the RPI. I'm almost at the point of advocating it regardless. The benefits to an chess-rating like (Elo-type system) are too good to pass up.
     
  7. kolabear

    kolabear Member+

    Nov 10, 2006
    los angeles
    Nat'l Team:
    United States
    Oh, I dunno. I thought it was another example of the West (and the WCC) getting the short end of the stick again, although this wasn't a real blatant case.

    If you saw them in Portland, you probably saw their worst game. Good teams can have those, and if you have one against a top-notch team like Portland, it can be pretty embarrassing. I think the pressure of the situation caught up with them: an 11-game unbeaten streak that began with a tie against UCLA and included 9 straight wins (including a victory over Santa Clara), the possibility of a WCC championship and an almost guaranteed seed in the NCAA, and now you're playing in Portland on a Fox-televised game...

    San Diego was a good team, but in Portland the pressure, and an even better team, got to them. They deserved consideration for a seed but you could call it either way on that one.

    It isn't just Portland that gets the shaft in the West. It's because teams like San Diego don't get the respect they deserve, that their rivals (including Portland) don't get the respect they deserve.
     
  8. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    I think kolabear is right about San Diego. My apologies to the Toreros. Even after all the work I've done, I'm still subject to years of brainwashing about other regions being equal to or stronger than the West. It's true, Portland made San Diego look not-so-good, but it definitely was a new experience for them being a highly rated team, playing in front of 4,000 + fanatic fans on national TV, etc., etc. And, Portland put a pretty good thumping on Tennessee, too, and their post-tournament RPI ranking was #9.

    I think Stanford's record this year is what is really instructive about how strong the West is and what the problem with the RPI is. They had a great record against a pretty strong out-of-region schedule and then had to battle really hard in the West to win a little over half of their games (only 1 loss, but a bunch of ties). It's hard to digest that Stanford's experience may say something very significant about the West relative to the other regions.

    Here are some specifics about Stanford's experience. This counts the tournament games. They played 9 games against non-West teams. The average post-tournament ranking of the teams they played (counting Connecticut twice since Stanford played them twice) was 46.3. Stanford's record against that group was 7-2, meaning it won 77.8% of the games. Stanford played 14 games against West teams. The average RPI ranking of those teams (counting Cal twice) was 80.6. Stanford's record against that group was 8-1-5. Using the RPI approach of counting a tie as half a win and half a loss, this makes Stanford's record against the West 10.5-3.5, which translates to winning 75% of its games.

    So, Stanford won essentially the same percentage of games against non-West teams with an average RPI ranking of 46.3 as it did against West teams with an average RPI ranking of 80.6.
     
  9. Cliveworshipper

    Cliveworshipper Member+

    Dec 3, 2006
    I'll echo cpthomas.

    At the time, San Diego was having a great run. doing well against Pac 10 schools, etc. Before that game, we thought they might be our biggest conference threat.

    I just don't think they were ready for the Merlo Field experience. Lots of teams come in here and do well until they hit adversity, then they can't keep up.

    Some schools are used to it (Santa Clara, for instance) but soccer at UP home games is definitely different.

    I think even the Tennessee fans that showed up for that game appreciated it. One parent of a player said to me " this is what it should be like everywhere". Tennessee is starting to develop a pretty good following of their own.

    I look forward to the time it is like that everywhere....
     
  10. Craig P

    Craig P BigSoccer Supporter

    Mar 26, 1999
    Eastern MA
    Nat'l Team:
    United States
    If they're a nonlinear model, I would assume that they are run recursively until they're self-consistent. (A linear model would admit a direct solution.)

    That needn't preclude some sort of externally imposed bias based on prior-season performance (something that I believe the stats folks call a "prior" -- see Wikipedia).

    There's no question that RPI is a relatively poor choice for the purposes for which it is used. However, the idea of replacing it with something with a real statistical foundation has never gained any traction in hockey (the proposed replacement was the Bradley-Terry method commonly implemented as KRACH), and I wouldn't expect the results of such advocacy to fare better with soccer.

    The way I understand cpthomas's approach, it is assumed that the NCAA is unwilling to scrap RPI, so the goal is that some education might lead to them using it more effectively through better understanding of its failings.
     
  11. Craig P

    Craig P BigSoccer Supporter

    Mar 26, 1999
    Eastern MA
    Nat'l Team:
    United States
    My sense (without having studied it in any detail, nor evaluated its performance) is that it looks very reasonable.
     
  12. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    Craig P, your interpretation of my approach is correct. Specifically, here's what I've suggested, based on the assumption the Women's Soccer Committee is going to continue its dependence on the RPI (this is from the paper that is an attachment to the initial entry on this thread):

    "The Committee should remember at all times that the RPI is unable to take into account differences in regions’ strengths and therefore is not able accurately to rank teams from different regions in relation to each other.

    "One way to remedy this problem would be to compute the average strength by region based on inter-regional games at the end of the 'regular' season and then allocate at large positions among the regions based on a proportional relationship between the number of at large positions, the number of teams in each region, and the regions’ relative strengths. This approach, however, would be based on an assumption that teams’ relative strengths within each region are distributed in a bell curve fashion that matches the distribution in each other region. That is not always the case. Further, it is not obvious what the correct proportionality formula would be for distributing the at large positions among the regions.

    "A more feasible remedy is to compute the average strength by region based on inter-regional games and for the Committee members to take regional differences in strength into account in a disciplined way as it makes its decisions. This means that the Committee would take into account the different regions’ strengths as it makes its seeding and at large selection decisions, making decisions in favor of teams from the stronger regions whenever there is any doubt, especially if the regional strength differences are significant. In addition, if a region is significantly stronger than others, the region should receive at a minimum a number of at large positions proportional to the number of the region’s teams as related to the total number of Division I teams. Further, that region, except under the most unusual circumstances, should receive significantly more at large selections than a strictly proportional number." (Emphasis added.)
     
  13. kolabear

    kolabear Member+

    Nov 10, 2006
    los angeles
    Nat'l Team:
    United States
    Do not lose hope, Comrade! As Comrade Thomas pointed out himself in his PDF paper at the start of this thread, that although "The Selection Committee explicitly does not consider coaches' polls and/or any other outside polls or rankings,"


    While a corrective measure to address the regional disparities noted by Comrade Thomas would be an excellent proposal to bring before the Committee, the NCAA has itself opened the door to the use of something like the Albyn Jones ratings, an outside ranking adapted from the Elo ratings used in chess.

    Allez, allez! Vive la Revolution! Even when you're on a PC instead of a Mac and you can't put in the accent marks!
     
  14. Cliveworshipper

    Cliveworshipper Member+

    Dec 3, 2006
    ¡Alléz, Alléz! yes you can.
     
  15. kolabear

    kolabear Member+

    Nov 10, 2006
    los angeles
    Nat'l Team:
    United States
    Vraiment? C'est impossible, non?

    Actually I know you can, I just don't know how, or maybe you need Word or some program I don't have on this PC.

    Of course all Macs come with the ability built-in, designed, as they are, with a cultured and sophisticated audience in mind...
     
  16. Craig P

    Craig P BigSoccer Supporter

    Mar 26, 1999
    Eastern MA
    Nat'l Team:
    United States
    In Windows XP, use the character map at:
    Start >> All Programs >> Accessories >> System Tools

    Or, if you know the character code in the Windows character set, and you're on a computer with a keypad, you can hold down alt and type the character code on the numeric keypad.
     
  17. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    I periodically check the minutes of meetings of the NCAA Division I's Championships/Competition Cabinet, which has oversight responsibility for the various Division I tournaments, including the various tournament committees' use of the Ratings Percentage Index for selecting teams to participate in the tournaments, seeding them, and placing teams in the brackets.

    At the Cabinet's June 2008 meeting, they approved a change in the formula for computing the RPI for Division I Men's Soccer. Previously, in computing a team's won-loss record (for both a team's winning percentage and a team's strength of schedule, each of which contributes to the RPI), the formula treated a tie as half a win and half a loss. For the Men's RPI, a tie now will be treated as 1/3 a win and 2/3 a loss. Also, the men had bonuses for quality wins, with higher bonus awards for wins (and ties) against teams ranked 1-40 (before awarding the bonuses) and lesser awards for wins (and ties) against teams ranked 41-75. They now have changed the bonus awards to three tiers: teams ranked 1-15, teams ranked 16-30, and teams ranked 31-75.

    For the Women's RPI, however, there is no change. So, ties still count as half a win and half a loss; and there are only two bonus tiers of 1-40 and 41-80.

    So, if the Men's system is better, why aren't the Women using it? Or, conversely, if it isn't better, why the change?

    But, this also reinforces what the evidence for Women's Soccer shows and what apparently also is the case for Men's Soccer. The Men's and Women's Soccer Committees, which run the NCAA Tournaments, think the RPI has a high degree of accuracy. The Women's Committee's seeding, selection, and bracket placement of teams in 2007 demonstrated that the Committee considers the RPI to be very accurate, following very closely the RPI rankings of the teams. The Men's changes this year, particularly going to a three-tier bonus system, also demonstrate it: the only possible reason to have such a refined bonus system is to alter placements of teams in relation to each other from what they would be using the old bonus system. But, the old bonus system (still to be used by the women) itself only results in teams' position slots in the rankings changing by 0 to 3 spots for almost all of the teams that would be under consideration in the Tournament selection, seeding, and bracket placement process. Since the new men's bonus system is only a refinement of the old bonus system, it will result in changes in rankings of probably only 0 to 1 spot and maybe, at most, 2 spots. What this shows is that the Men's Committee thinks that changes of 1 or 2 spots are meaningful, which means that they must consider the RPI rankings to be extremely accurate -- in fact, almost exactly correct in assessing teams' relative strengths.

    In fact, the RPI rankings are much more gross than that. In real life, as I have shown in the paper mentioned below, distinctions of 1 or 2 or 3 places in the RPI rankings lists are meaningless in assessing teams' relative strengths. So, what the Men's Committee's formula change shows is that the Committee has no understanding of the very limited usefulness of the RPI in ranking teams. It reinforces what I have concluded about the Women's Soccer Committee.

    For those interested in the RPI, download and read the two documents attached to the very first entry on this thread. The first document provides a very detailed analysis of the RPI as applied to Division I Women's Soccer (be aware that the document is long) and the second is the NCAA staff's rationale for using the RPI.

    Once we get into the season, I'll be posting Division I Women's RPI calculations periodically on this thread for those who are interested.
     
  18. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    To further validate the great amount of weight the NCAA Division I Women's Soccer Committee gives to the RPI, as discussed in the previous entry on this thread, here is information on how the 2007 seeding in the NCAA Tournament related to teams' RPI ranks. The information is based on the teams' RPI ranks following completion of the regular season, including the conference tournaments. These are the rankings the Committee uses for selection of at large participants, seeding, and structure of the four-team pods playing at each first and second round games site. The first number is the team's seed in the bracket, treating bracket placement as putting teams in a 1-16 order, which is what the NCAA appears to do even though it claims to have four #1 seeds, four #2s, etc. The second number is the team's RPI rank.

    North Carolina 1/1
    UCLA 2/3
    Penn State 3/2
    Stanford 4/5
    Texas A&M 5/4
    USC 6/11
    Portland 7/6
    Purdue 8/8
    Georgia 9/7
    Tennessee 10/10
    Florida 11/9
    Florida State 12/16
    Wake Forest 13/13
    West Virginia 14/12
    Virginia 15/14
    Notre Dame 16/15

    My guess is that head-to-head matches and results against common opponents are responsible for most of the divergences between the seed positions and the RPI ranks. For example, USC beat Portland, which could account for USC being elevated in the seeding process.

    To further show how highly the Committee values the RPI, I looked at the RPI ranks of teams the Committee selected for at large positions. After taking out of the ranks those teams that were conference champions, the Committee selected its 34 at large participants from those ranked in the top 37 on the RPI at large candidate list. Only two teams in the top 34 of this list were left out by the Committee: Dayton at #28 and Arizona State at #32. The two teams that got in that were not in the top 34 were Auburn at #36 and Miami at #37. The other team of the top 37 that was left out was Washington State at #35.

    Finally, the Committee, at each host site for the first and second round games, always matched the seeded team with the team from that site's foursome that had the lowest RPI.

    If anything is clear, it is that the Committee gives the RPI great, and nearly determinative, weight in the Tournament participant selection, seeding, and bracket formation process.
     
  19. Craig P

    Craig P BigSoccer Supporter

    Mar 26, 1999
    Eastern MA
    Nat'l Team:
    United States
    The bonus is kind of an odd thing, because on the one hand, it shows a lot of faith in RPI, in terms of confidence in the classifications, but on the other hand, it shows a lack of faith in RPI, because they feel like they need to offer bonuses for beating good teams. I doubt they've really thought about it in detail. (In fact, they really ought to get out of the stats business entirely. I would like to see the committees determine what aspects of performance interest them, and then ask some real experts on statistics to design a metric to handle it. As it is, we've got a bunch of people who don't know what they're doing misusing statistics that have never been all that good...)

    As far as the weight of ties... that's an interesting question. On the one hand, it really is about midway between a win and a loss in terms of performance. On the other hand, a tie is only worth one third of a win in points terms. I lean toward the half a win/half a loss, if a true evaluation of the strength of the team is the goal.
     
  20. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    Oops, I made an error in post 68. Before I ran the RPI program, I forgot a couple of steps that would have revised the bonuses/penalties assigned for good wins/ties and bad losses/ties. I've since made the corrections, which provide a piece of very interesting information.

    Here are the correct seeds/RPI ranks. This time, instead of guessing that the Women's Soccer Committee really seeds teams 1-16, I've taken them at their word that they simply give four teams #1 seeds, four #2 seeds, etc. The first number assigned to each team is its seed and the second is its RPI rank:

    North Carolina 1/1
    Penn State 1/2
    UCLA 1/3
    Stanford 1/4
    Portland 2/5
    Texas A&M 2/6
    Purdue 2/7
    USC 2/11
    Georgia 3/7
    Florida 3/9
    Tennessee 3/10
    Florida State 3/16
    West Virginia 4/12
    Wake Forest 4/13
    Virginia 4/14
    Notre Dame 4/15

    In addition, the Women's Soccer Committee's 34 at large selections all came from the teams with RPI rankings in the top 36 after taking out the teams that were conference champions. The teams ranked in the top 34 that did not make it were Dayton (28 of that group) and Arizona State (32); and the teams ranked outside the top 34 that made it were Georgetown (35 of that group) and Miami (36).

    Here's an interesting tidbit of information. You'll notice that Stanford's RPI rank was #4, and it received one of the four #1 seeds. Makes a certain amount of sense, right? And Portland followed with the #5 RPI rank. But guess what: The data to which the NCAA applied the the RPI formula had two errors. (Don't ask me how I found this out and validated it, but I did.) The errors involved games from the Sun Belt Conference tournament, which were shootout games. They were entered into the NCAA database as wins by Denver and Arkansas Little Rock and losses by Florida Atlantic and Middle Tennessee. This was an error, because shootout games are supposed to be treated by the RPI as ties. The first result of this error was that Denver's and ALR's RPIs were mistakenly high and Florida Atlantic's and Middle Tennessee's were mistakenly low. No big deal.

    But, Denver played Stanford. Because of the way the RPI works, Denver's mistakenly high win-loss-tie ratio resulted in Stanford's RPI being overstated. In fact, if the NCAA had used the correct data, rather than Stanford being #4 on the RPI list and Portland #5, they would have been switched around with Portland #4 and Stanford #5. Unfortunately, this is a just-discovered error and no one knows what the Women's Soccer Committee would have done if it had the correct RPIs. It could have given any of Portland, USC, Texas A&M, or Stanford the fourth #1 seed, which could have changed significantly the way the tournament went.

    This illustrates one problem with the NCAA's approach to the RPI. Too much secrecy, in this case involving the data base it uses, which is not available to the public for review and verification.
     
  21. theguru

    theguru Member

    May 7, 2008
    Does the RPI take into consideration AWAY wins ? It seems the Top 20 schools always play HOME games and usually play each other early in the year. This does not allow the next 40 ranked teams an opportunity to get a quality a WIN.
     
  22. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    The main RPI formula does not consider whether games are home or away. Some, including me, consider this a very serious problem. As an example, Portland's regular season loss last year to UCLA in double overtime at UCLA was treated just like any other loss. No credit for it being away and no credit for it being in double overtime. Wow! Once the NCAA computes the basic RPI, however, it adds bonuses and subtracts penalties for good wins/ties and bad losses/ties, and the bonuses and penalties vary depending on whether the games are home-away-neutral. However, good losses are just losses. This is different than for basketball, where a home win is not treated as a full win and an away loss is not treated as a full loss.

    The ability to get games against high quality teams is a problem for the slightly lower ranked tems. Playing high quality teams is a good way for a team to improve its RPI. For that reason and also to get good head-to-head comparisons for NCAA seeding purposes, you are correct that many of the good teams schedule each other in early non-conference games. This doesn't leave a lot of room for slightly lower ranked teams to get games with them.

    Some of the good teams schedule as many home non-conference games as possible, but others don't. Here are numbers from the 2007 season, for the final RPI's top 20 teams. The numbers exclude Final Four games, which were at a neutral site.

    UCLA 14 home, 8 away, 1 neutral
    North Carolina 11 home, 7 away, 6 neutral
    USC 9 home, 13 away, 2 neutral
    Penn State 11 home, 10 away, 3 neutral
    Florida State 13 home, 8 away, 4 neutral
    Portland 11 home, 8 away, 3 neutral
    West Virginia 16 home, 8 away, 1 neutral
    Notre Dame 14 home, 9 away, 3 neutral
    Tennessee 9 home, 10 away, 3 neutral
    Texas A&M 12 home, 7 away, 5 neutral
    Stanford 13 home, 6 away, 4 neutral
    Virginia 13 home, 7 away, 3 neutral
    Purdue 16 home, 5 away, 4 neutral
    Florida 13 home, 9 away, 3 neutral
    Georgia 13 home, 7 away, 4 neutral
    Wake Forest 8 home, 8 away, 7 neutral
    Texas 11 home, 9 away, 5 neutral
    James Madison 12 home, 9 away, 2 neutral
    San Diego 10 home, 10 away, 1 neutral
    Central Florida 10 home, 7 away, 6 neutral
     
  23. Cliveworshipper

    Cliveworshipper Member+

    Dec 3, 2006
    I think there really aren't that many games that a top 10 school can use as discretionary to play lower ranked teams.

    It seems to me that If a (for want of a better term) second tier team scheduled all of it games only with top 40 teams and then managed to win them, it would enhance their own RPI and Make them pretty attractive targets for the top 10 teams to play. A top 10 school isn't going to want to schedule you if their RPI suffers. sometimes it can be critical to placement in the tourney. Oregon didn't get in a couple years ago despite a good record, because it played a team or two with a very low RPI. Portland Schedules a game every year with U.Washington as a tradition, and it's hurt them a little the last couple years. They also try to schedule at least two of the three in-state schools every year. and that also has hurt. I wouldn't be at all surprised if they start scheduling Seattle university regularly also, because of the historical ties of the two schools. That will be more likely if Seattle succeeds in the NCAA RPI-wise.


    The above listing appears to show all games, which would include conference games, which will average to 50-50, I trust. But it also shows tournament games, which are often scheduled by record that year, and not before the season. Also, Tournament games are seeded by RPI.

    It would be interesting to see what the home-away ratio actually is for non-conference games during the regular season only. That would give a better idea of what teams are trying to do to enhance their chances. It appears to me that schools are only scheduling 3-4 top ten schools a year, perhaps only 2-3 out of conference. Figuring many schools have teams they play for other reasons than RPI, (like UP with UW, Oregon, Oregon St. and Portland State) it leaves only 2 or 3 discretionary games. sometimes even those are decided by other schools, such as the UW tournament UP plays in each year being actually decided by UW.

    Portland has another issue. With its great attendance, UP women's soccer has become a revenue sport and the flagship of the athletic program. That's great, in terms that they can afford go anywhere to play a team that will return a visit, but it also means they will want to schedule teams that will draw well at the UP game of a home-and-home contract. Generally speaking, UP will sell out if a team shows up that has a top 20 rating in the polls. Unranked teams are the main reason UP games aren't totally sold out.

    For an idea of how UP women's soccer is influencing UP athletic policy in general, go Here - Portland Tribune
     
  24. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    As requested, here are the home/away/neutral games for pre-NCAA tournament non-conference games of the top 20 RPI-ranked teams (using the RPI as of the end of the regular season, post-conference tournament). In addition, I have included the number of non-conference games against the top 20 that each team played.

    North Carolina 3/2/3 2
    Penn State 3/5/1 5
    UCLA 5/4/1 3
    Stanford 5/2/4 2
    Portland 5/4/2 4
    Texas A&M 5/2/2 3
    Georgia 5/2/1 1
    Purdue 8/1/1 1
    Florida 5/3/0 2
    Tennessee 2/3/2 3
    USC 4/5/1 3
    West Virginia 5/1/1 4
    Wake Forest 3/3/3 3
    Virginia 6/1/1 2
    Notre Dame 4/2/1 2
    Florida State 4/3/1 2
    San Diego 6/7/0 2
    James Madison 6/2/0 1
    Texas 6/2/1 1
    South Carolina 3/3/3 1
     
  25. cpthomas

    cpthomas BigSoccer Supporter

    Portland Thorns
    United States
    Jan 10, 2008
    Portland, Oregon
    Nat'l Team:
    United States
    The RPI ranks for games through September 1 are in the pdf attachment to this post. They are not meaningful for purposes of the ranks, as the teams have not played enough games. (See also a few explanatory items at the end of this post.) However, for those interested in understanding how the RPI works, following the ranks as they change from week to week can be a big help.

    The ranks through September 1 illustrate a key aspect of the RPI. Those checking the attachment will see that the RPI ranks St. Johns and Colorado College as ## 5 and 6 respectively. They are undefeated. Other undefeated teams ranked below them include: North Carolina (#7), Stanford (#10), Notre Dame (#25), UCLA (#32), Duke (#63), Wake Forest (#115), Virginia (#165), and Clemson (#190). Very few, if any, people would consider St. Johns or Colorado College to be better than any of these teams. So, why does the RPI rank the two teams higher?

    The RPI starts from a basic assumption that every team is of equal strength to every other team. To put it differently, the RPI gives each team a starting equal weight. Teams tend to play opponents in certain "pods." The RPI, as part of its basic assumption, initially weights the different pods as equal. The RPI is able to offset this equal weighting only if the pods have cross-over games and, even then, only to the extent of the number of cross-over games. (Over the course of an entire season, the "pods" are defined mostly by geographic reason, with some extension across regions for teams playing in multi-region conferences. At the beginning of the season, however, the "pods" are defined simply by who has managed to play whom during the limited number of games played.)

    The RPI consists of three elements. The first is a team's winning percentage. The second is a team's opponents' winning percentages against other teams. The third is a team's opponents' opponents' winning percentages. The latter two elements are considered together to represent a team's strength of schedule.

    Since all of the teams I've mentioned above have won all their games, the reason for the different RPI ranks is that the RPI considers the teams to have had different strengths of schedule. So, the RPI calculates St. Johns and Colorado College to have had stronger strengths of schedule than the teams below them. If someone who knows the teams were to look to see who at least some of the teams have played, it would be clear that in fact some of the lower ranked teams have played stronger schedules in the few games played so far. (For example, North Carolina has played Charlotte, Texas A&M, and Tennessee. Colorado College has played Oklahoma, Valparaiso, and Idaho State; and St. Johns has played Fairfield, Towson, and Delaware.) But, the RPI can't recognize this because of the initial weighting of all the teams as equal. Instead, it simply looks to see that St. Johns' and Colorado College's opponents and/or opponents' opponents have had better records than those of the teams ranked below them and assumes that each of the teams' opponents and opponents' opponents are equal. Thus St. Johns and Colorado College are ranked higher.

    A major question regarding the RPI is whether teams play enough "cross-pod" games to overcome the initial weighting of all teams as equal. The paper attached to the inital post on this thread goes into this question in detail and demonstrates that, in fact, teams do not play enough games to overcome this initial weighting. Since teams' "pods" as of the end of the season largely are geographical, what this means is that teams do not play enough cross-regional games over the course of an entire NCAA season for the RPI to be able to accurately compare teams from the different regions. Since the RPI starts out weighting all teams, and thus pods, as equal, this means that the RPI under-rates teams from stronger regions and over-rates teams from weaker regions.

    As the season progresses, I'll be trying to post the RPI ranks weekly. Those interested will be able to follow the RPI trends and reach their own conclusions about the strengths and weaknesses of the RPI.

    Explanatory items:

    1. 56 teams are not included in the current RPI list. This is because either they or their opponents have not yet played enough games for the program to be able to compute their RPIs. (For example, the Ivy League teams won't have played any games until next weekend.) Once all teams have played at least 2 games, the program will compute RPIs for all of them.

    2. In computing a team's winning percentage, a tie counts as half a win and half a loss.

    3. The RPI formula is: (1 x Winning Percentage) + (2 x Average of Opponents' Winning Percentages (Against Other Teams)) + (1 x Average of Opponents' Opponents' Winning Percentages), all divided by 4. This gives the appearance that a team's Winning Percentage is given 1/4 of the weight and a team's Strength of Schedule is given 3/4 of the weight. However, this is not true. Teams' winning percentages have a wide disparity, ranging from 100% to 0%. The averages of teams' opponents' winning percentages cover a narrower range; and the averages of teams' opponents' opponents' winning percentages cover an even narrower range. What this means, in effect, is that a team's winning percentage accounts for about 50% of its RPI; a team's opponents' winning percentages account for about 40% of its RPI; and a team's opponents' opponents' winning percentages account for about 10% of its RPI.

    4. The RPI formula includes adjustments for good wins/ties and bad losses/ties. This early in the season, trying to make adjustments would be meaningless. Further, the adjustments do not result in big changes in teams' RPI ranks. The attachment shows teams' unadjusted RPIs. At some point during the season, my weekly RPI reports will include the bonus/penalty adjustments. Those adjustments are the one part of the process the NCAA keeps secret, so the bonus/penalty amounts I use are my best educated "guesses." If anyone thinks they know what the exact amounts are, please let me know.

    5. The NCAA Division I Women's Soccer Committee gives the RPI very strong consideration in selecting at large teams for the NCAA Tournament and in seeding teams. In 2007, the 34 at large teams came from the top 36 RPI-rated teams (after taking out the teams that received automatic conference-champion berths); and the 16 seeds came from the top 16 RPI-rated teams.
     

Share This Page