It’s Time to Change the RPI

I am declaring war on the NCAA version of the RPI currently used for Division 1 Women’s soccer. It is time to change to a better version, the Balanced RPI. The current NCAA version is resulting in 3 to 4 teams per year getting at large positions in the NCAA Tournament they should not be getting. The conferences that are having positions taken away from them because of the current RPI’s defects are the Big 10 (to the tune of about 1 lost position per year), the West Coast Conference, and the Pac 12 and, to a lesser extent, the ACC. The conferences that are getting those positions instead are the Big East (one of my favorite conferences), SEC, Big 12, American, and Colonial.

Don’t take my word for it. Go to the RPI: Modified RPI? page of the RPI for Division I Women’s Soccer website for a full description of the Balanced RPI, a detailed demonstration of why it is far superior to the current NCAA version of the RPI, and a link to an Excel workbook that shows in detail the likely at large selection differences were the Women’s Soccer Committee to be using the Balanced RPI rather than the current NCAA version.

During the Fall season this year, in my weekly RPI and bracketology reports, I am going to be showing the differences between what the Committee is likely to do given the NCAA requirement it use, and only use, the current NCAA RPI and what the Committee likely would do if instead it were using the far superior Balanced RPI. And, at the end of the season I will compare the Committee actual at large selections to what they likely would have been if the Committee had used the Balanced RPI.

IT’S TIME TO CHANGE THE RPI

 

Yes, you are right that the RPI is intended to measure performance during the course of the season.

If you read the linked webpage and the other related pages at the RPI for Division I Women’s Soccer website, however, you will see that the currently used RPI is neither objective nor fair.

 

I do not know if the NCAA intended for the RPI to be objective at the time it originally started using it. It probably did. However, intending it to be objective is different than its actually being objective. In fact, the RPI discriminates among conferences and regions based conference strength, with the regional discrimination perhaps also based on how strong and weak conferences are dispersed among the regions. The discrimination is due to its particular structure and is well documented. There are reasons why the NCAA stopped using the RPI for basketball and this probaby is one of them, combined with the fact that the RPI is game-able through smart non-conference scheduling.

Fairness is not based on the eye of the beholder, rather it is objective. I am guessing that you did not read the linked material in my original post. If you would read that and the related material at that website, you would see that by objective measures (fully described), the RPI is discriminatory and thus not fair.

The unfortunate thing about the NCAA, for the sports that currently use the RPI, is that the NCAA mandates the Committees use it and forbids them to look at any other rating systems. The better approach would be for the Committees to look at multiple rating systems. For D1 women’s soccer, there is the Balanced RPI and the Kenneth Massey ratings. FYI, the Balanced RPI and Massey produce quite similar rankings year after year, that are quite different than the current NCAA RPI rankings.

If you are seriously interested in having fair and objective ratings, I really urge you to check out the linked material in the original post.

 

All of the sports that use the RPI consider it as only one of the factors for selecting teams for their tournaments. However, the RPI infiltrates other factors. For example, one of the other factors is results against other teams ranked #xx or better. The ranks, for that factor, are the RPI ranks. Almost all coaches are well aware that the RPI is a very powerful factor, especially since it is blended into other factors, in the selection process.

It seems like you do not think it is a problem that the RPI discriminates against teams from stronger conferences and regions. That is your prerogative. It is not fair in how it rates teams from different conferences and regions in relation to teams from other conferences and regions, but that is ok with you. So be it.

In terms of the result I want, it does not have to do with individual teams or conferences. Rather, what I want is a rating system that does not systematically discriminate and also one where a team rank and its rank as a strength of schedule contributor within the ranking system are essentially the same.

Interestingly, in ice hockey, the review I have done of it previously suggests that their modification of the standard RPI was intended to rebalance the strength of schedule computation so that it has effective weights of half opponents winning percentage and half opponents opponents winning percentage, which is one of the first steps in computation of the Balanced RPI.

 

I will take a try at showing details of what the differences would have been in prior years if the Committee had used the Balanced RPI rather than the current NCAA RPI.

There is one overall caution: If the NCAA had used the balanced RPI, non-conference scheduling likely would have been different. This is because teams can game the current NCAA RPI via their non-conference scheduling -- if you want to play a team within a certain RPI rank range such as ranked roughly #60, there are some teams within that range you want to avoid from an RPI perspective and there are others in the range that would be good to play. Generally, the ones you want to avoid are the ones whose ranks are based more on their strength of schedule than their winning percentage. The ones you want to play are those whose ranks are based more on their winning percentage than their strength of schedule. This is because the current RPI formula values your opponent’s winning percentage at four times the value it gives their strength of schedule. This is not the case with the Balanced RPI where, for example, the reward for playing teams ranked #59 through #61 always will be about the same.

Also, there are assumptions built into what I will be showing. The main assumption is that the Committee would follow the same patterns with the Balanced RPI as it follows with the current NCAA RPI. This makes sense to me, since if you put yourself in a Committee member’s shoes, you would be looking at the same types of rating numbers and ranks, the other factors you consider would look the same, and you would have no reason to act any differently than how you act with the current NCAA RPI. Following on this:

Since 2007, based on the curent NCAA RPI, all teams getting at large selections have come from teams ranked #57 and better. Thus the first assumption is that the candidate group for at large selections with the Balanced RPI would remain as teams ranked #57 and better.

An excellent predictor of which teams the Committee will select for at large positions is a metric that combines two factors: (1) teams’ RPI ranks and (2) teams’ ranks in terms of their results against Top 50 opponents (using my scoring system for the value of those results). Since 2007, this metric by itself on average matches all but two Committee at large selections per year. The second assumption is that with the Balanced RPI this metric would be just as effective as with the current NCAA RPI. [Note: The value of a team’s results against Top 50 opponents may be different under the Balanced RPI than under the current NCAA RPI, since the teams that are in the Top 50 and their specific ranks may have changed.]

Since 2007, all of the Top 30 team that are not automatic qualifiers have received at large positions. The third assumption is that this would continue to be the case with the Balanced RPI.
​

With that as context, below is a hopefully informative comparison of the Committee actual at large decisions in 2022 to what they likely would have been under the Balanced RPI, using the metric combining Balanced RPI rank and Top 50 results rank as the indicator of what the Balanced RPI decisions likely would have been.

As you may notice, the main changes are in teams that did not get at large selections (with one exception) dropping out of the Top 57 and being replaced in the Top 57 by teams that likely would get at large selections, thus bumping other teams out of the at large selections they actually got. Put differently, the defects in the current NCAA RPI are causing some teams, that should be getting serious looks from the Committee and likely would get at large positions if they did, to not get serious looks and thus not get the positions they deserved.

1. Top 30 Teams. The current NCAA RPI and Balanced RPI Top 30 teams do not match exactly. However, under either system, the other system’s Top 30 would have received at large positions.

2. Additional At Large Under Both Systems. There are another 7 teams that would have been at large under either system.

3. Teams Moving Outside the Top 57. There are 8 teams that were in the current NCAA RPI Top 57 that would have been outside the Top 57 under the Balanced RPI. They break down as follows:

a. Automatic Qualifiers. Two Automatic Qualifiers would move outside the Top 57. Since they were Automatic Qualifiers, this would have no impact on the Tournament field. They are Quinnipiac (Metro Atlantic conference, North region, moving from #43 to #82) and Lamar (Southland, South, 38 to 83).

b. Received Actual At Large Position. One team that got an at large position would move outside the Top 57 and thus would not get an at large position -- although it dropped only to #58, so perhaps the Commitee would consider it although it likely would not have gotten an at large position anyway. It is:

Utah Valley (WAC, West)

RPI rank change: 36 to 58
Top 50 results rank change: 57 to 53
Combined rank change: 63 to 80​

c. Did Not Get Actual At Large Positions. Five teams that did not get at large positions would be outside the Top 57. They are Auburn (SEC, South, 50 to 59), South Dakota State (Summit, Middle, 57 to 72), Monmouth (Colonial, North, 53 to 67), Liberty (Atlantic Sun, South, 48 to 85), and Northeastern (Colonial, North, 56 to 95).
​

Correspondingly, there are 8 teams that were outside the current NCAA RPI Top 57 that would have been inside the Top 57 under the Balanced RPI. They break down as follows:

a. Likely At Large Position. One team likely would have gotten an at large position. It is:

Nebraska (Big Ten, Middle):​

RPI rank change: 58 to 46
Top 50 results rank change: 16 to 21
Combined rank change: 66 to 57​

[Note: The combined rank is not simply a matter of averaging the RPI rank and Top 50 results rank since the Top 50 results ranks of the Top 57 RPI ranked teams cover a wider range than 57 positions.]​

b. Winning Percentage Below 0.500, Disqualified. Two teams would have been disqualified from an at large position. They are Utah (Pac 12, West, 61 to 55) and Lousiville (ACC, South, 70 to 57).

c. Would Not Get At Large Position. Five teams likely would not have gotten at large positions. They are:

Gonzaga (West Coast, West)​

RPI rank change: 59 to 35
Top 50 results rank change: 77 to 64
Combined rank change: 98 to 67​

Colorado (Pac 12, West)

RPI rank change: 65 to 53
Top 50 results rank change: 45 to 29
Combined rank change: 88 to 68​

San Francisco (West Coast, West)

RPI rank change: 74 to 47
Top 50 results rank change: 89 to 66
Combined rank change: 119 to 80​

Texas Tech (Big 12, South)

RPI rank change: 81 to 51
Top 50 results rank change: 59 to 67
Combined rank change: 111 to 85​

Washington State (Pac 12, West)

RPI rank change: 71 to 50
Top 50 results rank change: 104 to 85
Combined rank change: 123 to 93​

Finally, there are two actual Top 57 teams that got at large positions that likely would not have gotten them. They are:

Arizona State (Pac 12, West)

RPI rank change: 32 to 36
Top 50 results rank change: 91 to 68
Combined rank change: 78 to 70​

Wake Forest (ACC, South)

RPI rank change: 45 to 43
Top 50 results rank change: 58 to 57
Combined rank change: 74 to 72​

Conversely, two actual Top 57 teams that did not get at large positions likely would have gotten them. They are:

Arizona (Pac 12, West, 52 to 42)

RPI rank change: 52 to 42
Top 50 results rank change: 41 to 20
Combined rank change: 73 to 52​

Pepperdine (West Coast, West)​

RPI rank change: 54 to 33
Top 50 results rank change: 81 to 35
Combined rank change: 95 to 66​

 

2021 Season:

1. Top 30 Teams. The current NCAA RPI and Balanced RPI Top 30 teams do not match exactly. However, under either system, the other system’s Top 30 would have received at large positions.

2. Additional At Large Under Both Systems. There are another 9 teams that would have been at large under either system.

3. Teams Moving Outside the Top 57. There are 7 teams that were in the current NCAA RPI Top 57 that would have been outside the Top 57 under the Balanced RPI. They break down as follows:

a. Automatic Qualifiers. Five Automatic Qualifiers would move outside the Top 57. Since they were Automatic Qualifiers, this would have no impact on the Tournament field. They are Samford (Southern, South, 33 to 71), New Mexico (Mountain West, West, 57 to 61), Milwaukee (Horizon, Middle, 48 to 65), Old Dominion (Conference USA, South, 53 to 67), and Monmouth (Metro Atlantic, North, 43 to 85).​

b. Received Actual At Large Position. None​

c. Did Not Get Actual At Large Positions. Two teams that did not get at large positions would be outside the Top 57. They are Houston (American, South, 54 to 74) and VCU (Atlantic 10, South, 56 to 68).​

Correspondingly, there are 7 teams that were outside the current NCAA RPI Top 57 that would have been inside the Top 57 under the Balanced RPI. They break down as follows:

a. Likely At Large Position. None​

b. Winning Percentage Below 0.500, Disqualified. None​

c. Would Not Get At Large Position. Seven teams likely would not have gotten at large positions. They are:​

Louisville (ACC, South)​

RPI rank change: 66 to 52
Top 50 results rank change: 27 to 29
Combined rank change: 80 to 67​

Oregon State (Pac 12, West)​

RPI rank change: 59 to 53
Top 50 results rank change: 71 to 44
Combined rank change: 95 to 75​

Michigan State (Big 10, Middle)​

RPI rank change: 58 to 46
Top 50 results rank change: 90 to 78
Combined rank change: 103 to 85​

Iowa (Big 10, Middle)​

RPI rank change: 70 to 44
Top 50 results rank change: 93 to 87
Combined rank change: 117 to 88​

California (Pac 12, West)​

RPI rank change: 68 to 50
Top 50 results rank change: 83 to 80
Combined rank change: 110 to 90​

Pittsburgh (ACC, North)​

RPI rank change: 64 to 47
Top 50 results rank change: 93 to 89
Combined rank change: 111 to 92​

Gonzaga (West Coast, West)​

RPI rank change: 60 to 42
Top 50 results rank change: 113 to 105
Combined rank change: 117 to 95​

Finally, there are two actual Top 57 teams that got at large positions that likely would not have gotten them. They are:

Providence (Big East, North)​

RPI rank change: 36 to 54
Top 50 results rank change: 29 to 38
Combined rank change: 51 to 73​

Alabama (SEC, South)​

RPI rank change: 49 to 55
Top 50 results rank change: 48 to 71
Combined rank change: 73 to 91​

Conversely, two actual Top 57 teams that did not get at large positions likely would have gotten them. They are:

Oregon (Pac 12, West)​

RPI rank change: 52 to 37
Top 50 results rank change: 41 to 13
Combined rank change: 73 to 44​

Indiana (Big 10, Middle)​

RPI rank change: 50 to 43
Top 50 results rank change: 26 to 26
Combined rank change: 63 to 56​

 

2019 Season:

1. Top 30 Teams. The current NCAA RPI and Balanced RPI Top 30 teams do not match exactly. However, under either system, the other system’s Top 30 would have received at large positions.

2. Additional At Large Under Both Systems. There are another 6 teams that would have been at large under either system.

3. Teams Moving Outside the Top 57. There are 6 teams that were in the current NCAA RPI Top 57 that would have been outside the Top 57 under the Balanced RPI. They break down as follows:

a. Automatic Qualifiers. Four Automatic Qualifiers would move outside the Top 57. Since they were Automatic Qualifiers, this would have no impact on the Tournament field. They are Loyola Chicago, Missouri Valley, Middle, 56 to 63), Milwaukee (Horizon, Middle, 50 to 67), North Texas (Conference USA, South, 47 to 75), and Monmouth (Metro Atlantic, North, 44 to 90).​

b. Received Actual At Large Position. None​

c. Did Not Get Actual At Large Positions. Two teams that did not get at large positions would be outside the Top 57. They are Yale (Ivy, North, 37 to 68) and Columbia (Ivy, North, 49 to 74).
​

Correspondingly, there are 6 teams that were outside the current NCAA RPI Top 57 that would have been inside the Top 57 under the Balanced RPI. They break down as follows:

a. Likely At Large Position. None​

b. Winning Percentage Below 0.500, Disqualified. Two teams would have been disqualified. They are Purdue (Big 10, Middle, 64 to 47) and Wake Forest (ACC, South, 61 to 56).​

c. Would Not Get At Large Position. Three teams likely would not have gotten at large positions. They are:​

Illinois (Big 10, Middle)​

RPI rank change: 74 to 55
Top 50 results rank change: 72 to 72
Combined rank change: 110 to 91​

Gonzaga, West Coast, West)​

RPI rank change: 83 to 53
Top 50 results rank change: 101 to 101
Combined rank change: 134 to 104​

Boston College (ACC, North)​

RPI rank change: 77 to 57
Top 50 results rank change: 101 to 101
Combined rank change: 128 to 108​

d. Automatic Qualifier. One team was an Automatic Qualifier. It was Cal State Fullerton (Big West, West, 102 to 45).​

Finally, there are three actual Top 57 teams that got at large positions that likely would not have gotten them. They are:

Utah (Pac 12, West)​

RPI rank change: 51 to 42
Top 50 results rank change: 48 to 48
Combined rank change: 75 to 66​

Texas (Big 12, South)​

RPI rank change: 48 to 50
Top 50 results rank change: 34 to 34
Combined rank change: 65 to 67
​

TCU (Big 12, South)​

RPI rank change: 55 to 46
Top 50 results rank change: 43 to 43
Combined rank change: 77 to 68​

Conversely, three actual Top 57 teams that did not get at large positions likely would have gotten them. They are:

Tennessee (SEC, South)​

RPI rank change: 53 to 59
Top 50 results rank change: 13 to 13
Combined rank change: 60 to 61​

Oregon State (Pac 12, West)​

RPI rank change: 52 to 36
Top 50 results rank change: 53 to 53
Combined rank change: 79 to 63
​

Georgia (SEC, South)​

RPI rank change: 57 to 52
Top 50 results rank change: 24 to 24
Combined rank change: 69 to 64​

 

First of all, they do not do all their scheduling 2 or 3 years in advance. (By the way, so far as I know, for D1 women’s soccer the coaching staffs do the scheduling, not the ADs. I am aware, however, there may be ADs who give their coaches certain parameters they went the coaches to meet in their scheduling.)

Second, the coaches whose teams are potential NCAA Tournament at large participants for quite a while (for more than 2 or 3 years) have been well aware of the need to schedule as I have described -- balancing that with being sure to play enough teams likey to be ranked in the Top 50 to get enough good Top 50 results. It is a tricky process that requires careful judgments, taking risks, and trying to predict where teams will be a couple of years down the road. The coaches have to make the best judgments they can, using the best information available.

 

Stepping back from the year by year reports, here is an overview of combined data from all the years from 2007 to 2022 (excluding Covid-affected 2020). (I will continue the year by year reports in later posts.)

On average there are 6 to 7 teams per year that are in the current NCAA RPI Top 57 that are not in the Balanced RPI Top 57 and conversely the Balanced RPI has 6 to 7 in its Top 57 that are not in the current NCAA RPI Top 57. Since 2007 all told, if the Committee were using the Balanced RPI, that would mean 101 teams drop out of the Top 57 and 101 different teams move into the Top 57. Of the 101, 44 are Automatic Qualifiers and thus in the Tournament regardless of the rating system. This leaves 57 teams that were in the current NCAA RPI Top 57 that drop out as at large candidates under the Balanced RPI and 57 teams that are at large candidates under the Balanced RPI but were not under the current NCAA RPI. Of the 57 teams that drop out in a switch to the Balanced RPI, only 6 got at large selections. Thus almost of all the teams dropping out did not get at large selections. [Note: It is a coincidence that the 57 team switches match the Tournament at large Top 57 rank field.]

In addition, among the teams that are in the Top 57 under both systems, there are rank positions changes. There also are changes in those teams’ opponents’ ranks, thus affecting other aspects of their profiles such as their Top 50 results ranks.

Given this, there are two main things that are happening in the year by year likely changes in the at large selections: (1) Teams that are new to the Top 57 are bumping other teams out of at large positions because the new teams have better profiles; and (2) teams that were in the Top 57 but did not get at large selections are bumping other teams out because the teams that did not get at large selections now have better profiles than the teams they are bumping out. This makes an important point: The changes in at large selections, in almost all cases, are not because teams that got actual at large positions have dropped out of contention -- they still are in the candidate group. Rather, it is because teams that were excluded from consideration due to defects in the RPI now have joined the candidate group and proved more deserving of at large positions; or it is because once the defects are eliminated, other teams that previously were candidates but did not get at large positions now have proved more deserving of those positions.

 

In the context of the preceding post, the following table shows, by conference and for the period from 2007 to 2022 (excluding 2020), the distribution of changes in teams among the Top 57 in moving from the current NCAA RPI to the Balanced RPI:

 

2018 Season:

1. Top 30 Teams. The current NCAA RPI and Balanced RPI Top 30 teams do not match exactly. This year was a little unusual for Top 30 teams. There are two current NCAA RPI Top 30 teams that likely would not have gotten at large positions under the Balanced RPI:

Auburn (SEC, South)​

RPI rank change: 28 to 36
Top 50 results rank change: 44 to 61
Combined rank change: 50 to 67
​

Mississippi State (SEC, South)​

RPI rank change: 17 to 43
Top 50 results rank change: 41 to 41
Combined rank change: 38 to 64
​

Also, there is one current NCAA RPI Top 57, but outside the Top 30, team that did not get an at large position that would have been in the Balanced RPI Top 30 and thus likely would have gotten an at large position:

Colorado (Pac 12, West)​

RPI rank change: 49 to 26
Top 50 results rank change: 61 to 55
Combined rank change: 80 to 54​

Under either system, all of the other system’s Top 30 would have received at large positions.

2. Additional At Large Under Both Systems. There are another 5 teams that would have been at large under either system.

3. Teams Moving Outside the Top 57. There are 3 teams that were in the current NCAA RPI Top 57 that would have been outside the Top 57 under the Balanced RPI. They break down as follows:

a. Automatic Qualifiers. One Automatic Qualifier would move outside the Top 57. Since it was an Automatic Qualifier, this would have no impact on the Tournament field. It is Monmouth (Metro Atlantic, North, 53 to 92).​

b. Received Actual At Large Position. None​

c. Did Not Get Actual At Large Positions. Two teams that did not get at large positions would be outside the Top 57. They are George Mason (Atlantic 10, South, 35 to 60) and VCU (Atlantic 10, South, 56 to 67).
​

Correspondingly, there are 4 teams that were outside the current NCAA RPI Top 57 that would have been inside the Top 57 under the Balanced RPI. They include Colorado, discussed under 1, above. The other three break down as follows:

a. Likely At Large Position. Two teams likely would have gotten at large positions. They are:​

Nebraska (Big 10, Middle)​

RPI rank change: 58 to 37
Top 50 results rank change: 48 to 40
Combined rank change: 82 to 57​

Utah (Pac 12, West)​

RPI rank change: 80 to 51
Top 50 results rank change: 36 to 16
Combined rank change: 98 to 59​

b. Winning Percentage Below 0.500, Disqualified. One team would have been disqualified. It is Notre Dame (ACC, Middle, 71 to 56)​

c. Would Not Get At Large Position. None

d. Automatic Qualifier. None​

Finally, in addition to Auburn and Mississippi State, discussed under 1 above, there are two additional actual Top 57 teams that got an large position that likely would not have gotten one. They are:

Mississippi (SEC, South)

RPI rank change: 39 to 47
Top 50 results rank change: 38 to 37
Combined rank change: 58 to 66​

Louisville (ACC, South)

RPI rank change: 52 to 35
Top 50 results rank change: 50 to 74
Combined rank change: 77 to 72​

Finally, one actual Top 57 teams that did not get an at large position likely would have gotten one. It is:

Illinois (Big 10, Middle)

RPI rank change: 50 to 42
Top 50 results rank change: 32 to 24
Combined rank change: 66 to 54​

 

Perhaps I have not expressed myself well on this thread, but in any event you clearly do not understand what I meant (no criticism of you intended, your comment just does not match what I intended to say). Rather than me giving a lengthy explanation about scheduling in relation to the NCAA Tournament here, the following is a resource for coaches on scheduling in relation to the Tournament: RESOURCES FOR SCHEDULING IN RELATION TO THE NCAA TOURNAMENT. If you read that resource, you will see what I mean.

If you really are interested in having a complete understanding of the RPI and its relationship to the Committee process so far as Division I Women’s soccer is concerned, I recommend you go to the RPI for Division I Women’s Soccer website and go through its pages from beginning to end. That would include reviewing material I have linked at the website. It would involve a lot of work on your part, but in addition to the information on the RPI and the Committee process, it would give you a complete picture of where the material I publish comes from and would let you draw your own conclusions on whether my methodology is valid or not.

I know you have some experience with the RPI, so if you are interested in a more direct discussion, you might consider sending me a personal message.