2024 Ratings and Ranks

I have done pre-season predictions for the 2024 season that include end-of-season (including conference tournaments):

Team NCAA RPI, NCAA Non-Conference RPI, and my Balanced RPI ratings and ranks

Conference champion automatic qualifiers for the NCAA Tournament

Evaluations of teams in terms of getting NCAA Tournament at large selections

Conference NCAA RPI, NCAA Non-Conference RPI, and my Balanced RPI ratings and ranks​

Following each week of the season, I update the predictions by substituting the week's actual results for the week's predicted results. The predictions are less reliable early in the season and increasingly reliable as the weeks pass.

If interested, you can find my pre-season predictions here: 2024 Article 1: Pre-Season Predictions for the 2024 Season. The article includes an explanation of how I do the predictions plus some information on their reliability.

 

This is just a quick note that I may write more about later:

There are going to be some changes in the RPI formula:

Whereas in the past, the formula has treated ties as 1/2 a win, they now will be treated as 1/3 of a win. The new treatment matches how conferences do conference standings and is a change that already had been made for men's soccer. Although this in theory could mean teams will be less likely to play for ties, I doubt many teams play for ties anyway. It may have unforeseen side effects, but we will not know that for a while.

There will be changes to the bonus and penalty system. In the past, there were bonuses for wins and ties against teams ranked 1-40 (higher bonuses) and 41 to 80 (lesser bonuses) and penalties for ties and losses against teams ranked in the bottom 40 (higher penalties) and the second to bottom 40 (lesser penalties). Now, there will be bonuses for wins and ties against teams ranked 1-25 (highest bonuses), 26 to 50 (middle bonuses), and 51 to 100 (least bonuses) and penalties for ties and losses against teams ranked 151-250 (lesser penalties) and 251 and poorer (higher penalties). These changes will have the bonus and penalty system better match how the data the Committee sees is broken down and how the Committee actually thinks about things. Currently, the new amounts of the bonuses and penalties have not been published and they may not be publicly available until the NCAA publishes its first RPI rankings after the fifth week of the season.​

 

I doubt it will make the NCAA RPI and the Balanced RPI any closer. Once we know the amounts of the new bonuses and penalties, I will run a detailed analysis of how the NCAA RPI performs as compared to both the previous NCAA RPI and the Balanced RPI. If have a suspicion that the new formula will slightly increase the NCAA RPI's discrimination problem, as the previous NCAA RPI was slightly more discriminatory than the NCAA Unadjusted RPI, which suggests that the bonuses and penalties actually slightly increase the discrimination. But I won't know for sure until I can do the detailed analysis.

On the change of ties to 1/3 the value of a win versus 1/2, I have had a further thought. It may cause teams to play more for wins in games that are tied near the end. My reasoning is that the coaches might think, "If we tie, it is not much different than a loss so far as the RPI is concerned, so if we play for a win, we are not risking that much." On the other hand, I do not know how coaches actually think in those situations, so who knows?

 

One other note about changes the Women's Soccer Committee wants to make: They want to discontinue using the KP Index as a supplemental rating system because they found it not very useful and instead propose using the Massey ratings as a supplemental system, starting in 2025. If this happens, it is terrific news as Massey has far less conference and region discrimination than the NCAA RPI and the KP Index.

 

Here is my 2024 Report 2, with current news and updated end-of-season rank and at large selection predictions: 2024 Article 2: This Week's News and Updated Predictions.

 

Another note on the change in the RPI formula counting ties as 1/3 of a win rather than 1/2. This is going to reduce almost all teams' winning percentages, since almost all teams have some ties. That is significant, due to the requirement that a team have a winning percentage of 0.500 or better in order to be considered for an at large position in the NCAA Tournament.

I did a check of what this would have meant for the seasons from 2010 through 2023. Historically, the candidate teams for at large selections have been teams ranked #57 or better. Since 2010 (13 seasons after disregarding Covid-affected 2020), there have been 24 teams in the Top 57 that had winning percentages of 0.500 or better with ties weighted at 1/2, but that would have had winning percentages below 0.500 with ties weighted at 1/3. Thus it appears that almost 2 teams per year that would be at large candidates under the old 1/2 weight now will not be at large candidates under the new 1/3 weight. Further, of these teams, 13 in fact received at large selections, which averages out to 1 team per year.

All but one of the teams that received at large selections under the 1/2 weight but not under the 1/3 weight are Power 4 teams, with the one exception being an American Athletic Conference team.

This suggests that as we approach the end of the season, where a potential NCAA Tournament team has a close to 0.500 winning percentage and would fall below 0.500 at the end of the season with a further tie, the team at all costs should play for a win since even with a winning percentage at or barely above 0.500, it may qualify for an at large position. Typically, this would be a team from one of the strongest conferences that is in the mid-ranks within its conference.

 

One more post, for now, about the 2024 RPI formula changes. I suspect that the changes in bonuses and penalties will include bonuses for wins and ties over teams ranked 1 to 25 that are higher than what we have seen in the past. Combine that with the reduction in the value of ties in the winning percentage portion of the RPI (which spills over to the strength of schedule portions) and with the elimination of overtimes two years ago and I have an educated guess that the RPI will have even more discrimination among conferences and regions. I am beginning to do the work to see if my guess is correct, but it will take a while.

If I am correct, then the Big Ten and teams from the West will be even more screwed by the RPI than they are already.

 

I think the Committee initiated the RPI formula change, although I don't know which Committee member or members.

In my experience, the way to find out about RPI changes is to check periodically to see when the Committee meeting minutes get published (which I believe was some time in August, for the April minutes in which the Committee recommended the RPI changes) and when the Competition Oversight Committee minutes of the meeting reviewing and approving or disapproving the Committee recommendations get published (I don't know when the publication was, but the meeting was in June). The other place to find out is in the NCAA Tournament Pre-Championship Manual, which ordinarily gets published part way through the season. The Manual typically sets out the factors that go into the Committee decisions and also describes the RPI formula including the bonus and penalty structure (but historically, not the bonus and penalty amounts unfortunately).

I think it is extremely unfortunate that this means coaches do not know about RPI formula changes until the season is well underway. I think such changes should be published at least a year ahead of the season in which they will be implemented so that the coaches will know in advance the rules of the game -- in this case, the "game" being qualifying for the NCAA Tournament. It also would allow an advance review of the changes to determine whether they will have any unintended consequences. But that, apparently, is too much to ask.

 

Here is my 2024 Report 3, with updated end-of-season rank and at large selection predictions, incorporating the actual results of games played through Sunday, August 25. Among other things, the report shows my current predictions of teams in the RPI Top 57 that look like they will end the season with winning percentages below 0.500 and thus will be disqualified from getting NCAA Tournament at large selections. The predictions are not good enough at this point to get perturbed about who the specific teams are, but the report provides a good illustration of one potential effect of changing the RPI formula to value ties at 1/3 of a win rather than 1/2. The report predicts 5 Top 57 teams will be disqualified using the 1/3 valuation, whereas only 1 of them would be disqualified using a 1/2 valuation.

 

They are a team that my pre-season ratings (based on the average of their last 7 years' ranks) do not do a good job with. If I use their and their opponents' assigned pre-season ratings, they have underperformed in every game so far. That rating was too optimistic, obviously. Once we get past week 5, I change my assigned pre-season ratings (for predicting the results of future games) to teams' actual NCAA RPI ratings. At that point, I think I will have a better picture of where they will end up. But even with the over-optimistic pre-season rating, it looks likely they will have a losing record. This is not good for them and also not good for the other Big 10 teams.

 

Formidable and, historically, underrated. It will be interesting to see how the addition of the former Pac 12 teams affects that, it might make it worse since teams from the West, including the Pac 12, also have been underrated.

 

A data observation that may be due to limited data so far, but maybe not ...?

Based on past history, with the relatively new "no overtime" rule (except for conference tournaments), the expected number of overtime games is 21.1%. The actual number so far this year is 17.2%.

Could the apparent change in RPI winning percentage valuation of ties from 1/2 a win to 1/3 a win (which makes a tie closer to a loss) be causing teams to be more adventurous at the end of games, thus resulting in fewer ties? Or is this just a function of limited data?

 

You could be right.

 

Here is my 2024 Report 4, with updated end-of-season rank and at large selection predictions, incorporating the actual results of games played through Sunday, September 1. As with Report 3, it shows a significant number of Top 57 teams being disqualified from NCAA Tournament at large consideration due to predicted Winning Percentages below 0.500. This is due to the expected change in the weight of tie games from 1/2 a win to 1/3 of a win when calculating Winning Percentage.

 

My answers to your questions:

1. I don't know. Your question is a very good one. It would seem odd for ties to count as half a win for the 0.500 winning percentage requirement but a third of a win for the RPI, but it could happen. Perhaps it will be clarified when the NCAA publishes the Pre-Tournament Manual, which always includers the 0.500 requirement.

2. Once there is verification about the one-third change and about the change in the bonus and penalty structure and amounts, which should be after the 5th week of the season, Chris Henderson will revise the formulas used at the scoreboard. In the meantime, his numbers are based on the formula up until now and my numbers include the one-third change. My number do not yet include the revised bonus and penalty structure and amounts, since we don't know what the amounts are.