RPI Success with Regions and Conferences

Discussion in 'Women's College' started by cpthomas, Dec 1, 2011.

  1. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    As I've written about, the RPI has trouble rating teams from the different geographic regions within a unified system. It also has a similar problem rating teams from different conferences within a unified system.

    I've run a process that shows how well the RPI does in each of these areas -- regions and conferences -- based on five years of merged data, the 2007 through 2011 seasons. The process looks at teams by region or conference and calculates whether the region's or conference's teams are underrated (i.e., they perform better in non-region or non-conference games than the ratings say they should) or overrated. In the system, a 100% measure means the region's or conference's teams perform in accord with the norm. A measure above 100% means they are underrated (i.e., they perform better than the norm) and a measure below 100% means they are overrated. The numbers below show the regions' and conferences' performance in games where the rating difference between opponents was less than 0.05. I used this rating difference because all teams had sufficient games within that rating difference and at the same time there were enough games to make the results meaningful.

    For regions, based on the teams' Adjusted RPIs, here are the regions' "performance percentages" over the five-year period:

    Code:
    West             121.5
    Southeast        105.0
    Middle           101.7
    Northeast        93.6
    Southwest        80.3
    In this list, I believe that the Southwest is lower than it really should be, although it should be at or near the bottom of the list. The problem for the Southwest is that it is home of the Southwestern Athletic Conference. SWAC teams vastly underperform their ratings (see the table below) and skew the Southwest region's ratings.

    For conferences, here are their "performance percentages" over the five year period:

    Code:
    SEC             117.1
    Big East        112.0
    Big West        109.4
    Pac Twelve      109.3
    WAC             108.7
    Missouri Valley 108.0
    Horizon         107.1
    Mid American    106.0
    Big Ten         105.6
    West Coast      104.9
    ACC             104.8
    Big Sky         103.4
    Big Twelve      102.5
    Atlantic Sun    101.5
    Ivy             101.1
    Mountain West   100.4
    Big South       99.5
    Atlantic Ten    98.0
    Northeast       97.8
    America East    97.4
    Summit          95.2
    Conference USA  94.6
    Metro Atlantic  94.4
    Patriot         93.6
    Colonial        93.5
    Southern        91.8
    Sun Belt        90.0
    Great West      88.9
    Southland       87.2
    Ohio Valley     80.7
    Southwestern    47.3
          
  2. uscue13 Member

    Member Since:
    Nov 11, 2009
    Club:
    FC Barcelona
    Country:
    United States
    I understand what all the numbers mean, but in your opinion/analysis, what does this tell us? That (in the regular season because we know how the post season ends up for them) that the SEC does better against non-cons than they should? Would that explain why they get a lot of tourney teams (with the highest underrated number, possibly means their rpi's are good enough to get them in ncaas) even though they don't have an upper tier team to make it far.
  3. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    Good question, regarding conferences, and I don't know that there's an obvious answer.

    In some cases, it's probable that the conferences being underrated are suffering the effect of being in a strong region so that all the teams in that region's conferences tend to be underrated. For example, all of the West region conferences appear to be underrated at least to some extent.

    In other cases, this definitely would not explain a conference appearing to be underrated. For example, the Big East is in the Northeast region. The Northeast region's teams, on average, appear to be significantly overrated. Yet the Big East shows up as the second most underrated conference. Why would this be occurring? Could it be that the Big East, as a relatively strong conference, primarily is playing non-conference teams that are overrated, so that its results against those teams make it look underrated? Could this also be true of the SEC? Are there, perhaps, more opportunities to play overrated teams in their geographic areas than there are in the West (or Middle)? Trying to figure this out would take a detailed study of who the Big East and SEC teams are playing as non-conference opponents and whether the bulk of those teams are from apparently overrated conferences.

    Another factor can be home/away imbalances, although I don't think that would account for the full extent of the SEC's and Big East's apparent underratings. In general, strong conferences tend to have favorable home game imbalances. If a strong conference appears to be underrated, part of the reason for that may be that the conference's actual performance is better than its rating due to a favorable imbalance. In order to determine the actual extent to which the underrating is due to a favorable home/away imbalance, I would have to filter out the effect of the imbalance. I've done this in the past for individual years, and the home/away imbalances do account for part of the overratings and underratings, but not close to all of them.

    One of my upcoming tasks is to filter out the effects of the imbalances for the combined five-year period and see the extent to which they are influencing the overrating/underrating numbers. When I've done that, I'll publish the results.

    In the meantime, my guess is that the SEC's teams, on average, are scheduling a higher than normal percentage of overrated opponents.
  4. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    Something didn't look right to me about the Pac 10/Pac 12 Conference in my table, so I went back and took a look at my programming. I was right. That darned conference name change messed up my performance percentage programming.:cool:

    Also, the Great West Conference didn't have a full five years of data, so I decided to treat the predecessor (though not identical) United Conference as though it had been the Great West so I could get five years' data.

    All the other conferences remain the same as in the table in Post 1.

    As I suspected, the Pac 12's performance percentage rose, from 109.3 to 119.6, making the Pac 12 the most underrated conference over the five year period.

    The Great West percentage changed slightly from 88.9 to 88.2.

    (Note: All the percentages treat the teams as in the conferences they actually were in for each of the five years.)
  5. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    Game Sites, Regions, and Conferences Part I: Home Field Advantage

    Yes, there is a relationship between game sites, on the one hand, and the RPI's difficulty rating regions and conferences in a unified system, on the other hand. Unfortunately, the relationship is complex.

    I'm going to try to provide information on the relationship that seems relevant to me, but I'm going to need to do it in in bite-size pieces. Part I will be basic information on which conferences and regions appear consistently to be beneficiaries of home field imbalances. A caution: Don't jump to conclusions!

    To get at home field imbalances, I assembled the game site data for the last five years, 2007 through 2011. I did two sets of calculations:

    1. Of games that were not at neutral sites, what percentage of the conferences' and regions' games were home games, over the five year period; and

    2. Of games that were not at neutral sites, what percentage of the conferences' and regions' non-conference and non-region games were home games?

    Here are the results:

    Conference/Of All Non-Neutral Site Games, % Home/Of all Non-Conference Non-Neutral Site Games, % Home

    Code:
    ACC	        58.1%	69.7%
    BigTwelve	55.3%	61.7%
    SEC	        53.5%	58.9%
    BigTen	        53.8%	58.7%
    WestCoast	54.2%	57.0%
    BigEast	        52.4%	57.0%
    MountainWest	53.7%	56.3%
    PacTwelve	52.4%	55.1%
    ConferenceUSA	51.9%	54.7%
    Ivy	        52.1%	53.9%
    BigWest	        51.9%	53.7%
    SunBelt	        51.1%	52.9%
    Patriot	        50.6%	51.0%
    Southern	50.2%	50.6%
    Colonial	50.1%	50.4%
    MissouriValley	49.3%	48.9%
    AtlanticSun	49.2%	48.1%
    AtlanticTen	48.7%	46.9%
    Southland	48.5%	46.9%
    WAC	        48.0%	46.3%
    Summit	        47.8%	45.5%
    AmericaEast	47.7%	45.2%
    GreatWest	46.7%	44.7%
    BigSky	        46.9%	44.5%
    OhioValley	47.3%	44.4%
    Horizon	        47.1%	44.2%
    BigSouth	46.8%	43.2%
    MetroAtlantic	46.5%	42.8%
    MidAmerican	47.2%	42.3%
    Northeast	45.7%	40.4%
    Independent	40.4%	39.8%
    Southwestern	43.7%	34.4%
    
    Region/Of All Non-Neutral Site Games, % Home/Of all Non-Region Non-Neutral Site Games, % Home

    Code:
    Southeast	51.5%	57.0%
    West	        51.0%	56.1%
    Southwest	50.2%	51.0%
    Middle	        48.8%	44.7%
    Northeast	49.1%	42.9%
    
    For a description of the regions and how I arrived at them, use this link: https://sites.google.com/site/rpifordivisioniwomenssoccer/non-region-rpi
  6. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    Game Sites, Regions, and Conferences Part II: Does Home Field Advantage Matter?

    Yes, home field advantage matters.

    As those who read my RPI posts know, I have developed a tool for measuring how teams perform in relation to their ratings. A 100% performance is the norm. A performance above 100% means a team is performing better than its rating says it should. This means the team is winning more games, in which it is the higher rated team, than is the norm; or it is winning or tieing more games, in which it is the lower rated team, than is the norm. A performance below 100% means a team is performing more poorly than its rating says it should.

    I can apply the tool to determine the extent to which home field advantage affects game results.

    In making the determination, I've concluded that it is best to look at games in which opposing teams' RPI rating differences are less than 0.02. This gives a good sample of games, but avoids skewing results due to games in which home field advantage is irrelevant because one team is much better than the other.

    In that context, the average performance percentage for home teams in games in which the Adjusted RPI rating difference is less than 0.02 is 121.7%. Conversely, the average performance percentage for away teams is 78.3%. In other words, home teams perform better, relative to their ratings, than away teams. (What a shock!)

    Next, I can quantify the benefit of home field advantage, through a trial and error process of upwardly adjusting home teams' ARPI ratings and downwardly adjusting away teams' ARPI ratings. By trying different adjustment amounts, I can identify the amount of upward/downward adjustment needed in order for home teams and away teams to have the same performance percentage regardless of game location.

    Through this trial and error process, I have determined that a team playing at home, on average, performs as though it has an 0.009 boost to its ARPI. And, a team playing away, on average, performs as though it has an 0.009 diminishment to its ARPI. If, on a game by game basis, I increase and decrease teams' ARPIs accordingly, then the average performance percentage for home teams is 100.3% and for away teams is 99.7%, which match for practical purposes.

    (If the adjustment were 0.008, the home team percentage would be 102.5% and the away team 97.5%. If the adjustment were 0.01, the home team percentage would be 98.2% and the away team 101.8%. So, the 0.009 adjustment is "just right." A note: Some may have seen me refer to an 0.008 adjustment previously. That is for a variant of the Adjusted RPI. For the NCAA's version of the ARPI, the adjustment amount is 0.009.)

    Coming next: How Significant Is Home Field Advantage?
  7. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    Game Sites, Regions, and Conferences Part III: How Significant Is Home Field Advantage?

    In Part II, I indicated that when a team plays at home, it performs as though its ARPI rating is 0.009 higher than its calculated ARPI; and when it plays away, it performs as though its rating is 0.009 lower.

    In terms of significance, what does this mean? How significant is this difference?

    One can think of the calculated ARPI as representing, on average, teams' ratings as though all games were played at neutral sites. (This will not be exactly correct for individual teams, since there are game site imbalances, but on average it is correct.) Thinking of the calculated ARPI in that way, when a team plays at home, this represents a relative shift of the difference between the two teams' ARPIs of 0.018 (the home team gains 0.009 and the away team loses 0.009). The following table shows what an 0.018 difference means relative to teams' 2011 ARPI rankings among the top 60 or so teams.

    0.018 = the change in rating difference from the neutral site rating difference, when Team A is at home and Team B is away. The 0.018 represents the rating difference between teams ARPI-ranked:

    #1 and #3

    #3 and #5

    #6 and #7

    #8 and #12

    #13 and #24

    #25 and #32

    #33 and #47

    #48 and #62

    It's worth noting that the last two groupings in the table represent 14 teams each. These two groupings span the at large "bubble" selection area, and my studies suggest that the "bubble" group consists of 15 teams.

    The above table covers the difference between two teams playing at a neutral site and playing at one team's home site. What about the contrast between Team A playing at home and Team B playing at home. In that case, the rating difference between the two scenarios is 0.036 -- Team A loses its 0.009 home advantage and takes on an 0.009 away disadvantage; and vice versa for Team B.

    0.036 = the change in rating difference from Team A Home/Team B Away to Team A Away/Team B Home. The 0.036 represents the rating difference between teams ARPI-ranked:

    #1 to #6

    #7 to #19

    #20 to #45

    #46 to #77


    When I try to simulate the Women's Soccer Committee's at large selection and seeding process, the question of game sites comes up for me in evaluating head-to-head results, results against common opponents, and results against teams already selected for the bracket:

    For a head-to-head example, suppose Team A plays at home against Team B and the result is a tie. How do I evaluate that? Do I treat the results as indicating the teams are equal? Or do I take the game site into account? In my process, I treat the game result as a positive result for Team B and a negative result for Team A.

    For a common result example, suppose Team A plays at home against Team C and ties; whereas Team B plays at a neutral site against Team C and ties. How do I evaluate that? In my process of comparing Team A to Team B, I treat the results as positive for Team B and negative for Team A.

    For a "results against teams already selected for the bracket" example, suppose Team A has an away tie against Team D (ARPI Rank #4) and Team B has a home win against Team E (ARPI Rank #20). How do I evaluate that? In my process, I treat Team D's tie as a better result.

    My process, in each example, involves my attributing a strong significance to game location. It represents my conclusion that home field advantage is very significant.
  8. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    Game Sites, Regions, and Conferences Part IV: Conferences, Regions, and Home Field Advantage

    Is there a patterned relationship between conferences and home field advantage? What about between regions and home field advantage?

    Yes, there is. Conferences with higher average ARPIs tend to have favorable home field imbalances; and conferences with lower average ARPIs tend to have unfavorable home field imbalances. The same is true for regions. This does not apply to every conference and region, but on average it is true.

    The following table, based on the five years from 2007 through 2011, shows the relationship between conferences' average ARPIs and their percentages of home games (excluding neutral site games from the calculation). The conferences are in order from highest average ARPI to lowest:

    5-Year Average ARPI/% of Non-Neutral Games at Home/% of Non-Conference Non-Neutral Games at Home

    Code:
    ACC	        0.6188	58.1%	69.7%
    PacTwelve	0.5953	52.4%	55.1%
    BigTen	        0.5747	53.8%	58.7%
    SEC	        0.5741	53.5%	58.9%
    BigTwelve	0.5733	55.3%	61.7%
    WestCoast	0.5625	54.2%	57.0%
    BigEast	        0.5529	52.4%	57.0%
    ConferenceUSA	0.5364	51.9%	54.7%
    MountainWest	0.5234	53.7%	56.3%
    Ivy	        0.5227	52.1%	53.9%
    BigWest	        0.5225	51.9%	53.7%
    Colonial	0.5187	50.1%	50.4%
    AtlanticTen	0.4923	48.7%	46.9%
    Southern	0.4831	50.2%	50.6%
    MidAmerican	0.4823	47.2%	42.3%
    SunBelt	        0.4787	51.1%	52.9%
    WAC	        0.4688	48.0%	46.3%
    Patriot	        0.4687	50.6%	51.0%
    Horizon	        0.4672	47.1%	44.2%
    MissouriValley	0.4664	49.3%	48.9%
    MetroAtlantic	0.4515	46.5%	42.8%
    AtlanticSun	0.4492	49.2%	48.1%
    Southland	0.4478	48.5%	46.9%
    Northeast	0.4440	45.7%	40.4%
    BigSky	        0.4413	46.9%	44.5%
    AmericaEast	0.4380	47.7%	45.2%
    OhioValley	0.4313	47.3%	44.4%
    Summit	        0.4306	47.8%	45.5%
    BigSouth	0.4257	46.8%	43.2%
    GreatWest	0.3557	46.7%	44.7%
    Southwestern	0.3442	43.7%	34.4%
    
    The following table, based on the five years from 2007 through 2011, shows the relationship between regions' average ARPIs and their percentages of home games (excluding neutral site games from the calculation). The regions are in order from highest average ARPI to lowest:

    5-Year Average ARPI/% of Non-Neutral Games at Home/% of Non-Region Non-Neutral Games at Home

    Code:
    West	        0.5205	51.0%	56.1%
    Southeast	0.5086	51.5%	57.0%
    Northeast	0.4860	49.1%	42.9%
    Southwest	0.4820	50.2%	51.0%
    Middle	        0.4776	48.8%	44.7%
    
    If I could insert charts for the two tables, with trend lines showing the relationship between average ARPI and % of home games, I would. But I can't. So, you'll have to trust me that the trend lines show, as you probably can get from the tables themselves, that on average (but not in all cases), the conferences and regions with the higher average ARPIs have a higher percentage of home games. Conversely, the conferences and regions with the lower average ARPIs have a lower percentage of away games.

    This, of course, leads to the next question: Do conferences and regions with higher average ARPIs have the higher values simply because of home field advantage imbalances?
  9. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    Game Sites, Regions, and Conferences Part V: Do Home Field Imbalances Affect Ratings?

    Do home field imbalances affect ratings?

    Although a first reaction to this question might be, "Of course," the real answer is, "It depends."

    When I get to Part VI, I'm going to show what happens to conferences' and regions' performance in relation to their ARPI ratings after I filter out of the ratings the effects of home field imbalances. The resulting changes in conferences' and regions' performance won't always be what one might expect, and I'm going to use this post to explain what I believe to be the reason.

    Basically, whether home field imbalances affect ratings depends on which games are home and which are away:

    To use an extreme hypothetical example for purposes of illustration, suppose that in 2011 all of Stanford's non-conference games had been home games. And, suppose that all of Stanford's non-conference opponents were teams from the Southwestern Athletic Conference. Stanford is so much stronger than any of those teams that Stanford would have won every game no matter where played. Thus Stanford's home field imbalance would have no effect on its rating. (I know that playing only SWAC teams as non-conference opponents would have affected Stanford's rating, but that's a different matter unrelated to the game sites.)

    The point of the hypothetical is that whether home field imbalances affect teams' ratings depends on the relative ratings of the opponents. In games where two opponents are mismatched, the game site generally is irrelevant. It's only in games, where the opponents are closely enough matched for the game sites to potentially make a difference, that home field imbalances are relevant.

    This is an important distinction. I'll come back to it when I get to Part VI, which is where things get complicated.
  10. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    Game Sites, Regions, and Conferences Part VI: Back to the Conferences and Regions Problem

    Now I'm back to the original question on this thread, which is: Does the RPI have a problem rating the different conferences and regions in a unified system and, if so, how great is the problem? This is the question that originally got me learning about the RPI.

    For a couple of years now, I have maintained that the RPI in fact does have a problem. Theory says it should have a problem, and actual experience confirms it has a problem. The question is, How great a problem?

    To get at the "how great" question, I developed my system for measuring conferences' and regions' performance in relation to their average RPI ratings. When I first reported my results on that, indicating that on average, stronger conferences and regions are underrated by the RPI and weaker conferences and regions are overrated, a Big Soccer poster suggested that my results might be more due to home game imbalances than other factors. Morris20, was that you? That lead me to assemble information on conference and regions home game imbalances, on which I reported a couple of posts previously on this thread.

    Since my information demonstrated that stronger conferences and regions tend to have favorable home game imbalances, it appeared to me that there were two potential contributors to the stronger conferences and regions appearing to be underrated. One potential contributor is the stronger regions' and conferences' favorable home game imbalances. The other is the RPI's inherent problem rating teams from different conferences and regions in a unified system.

    In order to separate out these two contributions, I needed to be able to "filter out" the effects of home game imbalances from the ratings. That is why I did my analysis to figure out how to quantify home field advantage, coming up with the 0.009 RPI "correction" amount also discussed in previous posts on this thread.

    With that amount in hand, I have gone through the process of analyzing the conferences' and regions' performance with teams' "corrected" ratings for individual games built into the system. What I did was, for every home game I adjusted the home team's ARPI rating upward by 0.009 and the away team's rating downward by 0.009. Thus, for each game, I used each team's "site adjusted" effective rating rather than its "site blind" calculated rating. The effect of this was to take out of the calculations any effects of home field imbalances. Having done that, I then re-computed the conferences' and teams' "site adjusted" performance in relation to average ratings over the five years 2007 through 2011.

    The result of the entire process produced the results shown in the following tables:

    Conference/Average ARPI (5 Years)/NCAA Performance Percentage/Home % of Non-Conference Non-Neutral Games/Site-Corrected Performance Percentage

    Code:
    PacTwelve	0.5953	114.9%	55.1%	116.3%
    SEC	        0.5741	111.8%	58.9%	116.1%
    BigTen	        0.5747	104.5%	58.7%	110.7%
    WAC	        0.4688	102.7%	46.3%	109.6%
    BigEast	        0.5529	83.3%	57.0%	107.7%
    BigWest	        0.5225	111.8%	53.7%	107.7%
    Northeast	0.4440	103.4%	40.4%	107.3%
    Horizon	        0.4672	91.4%	44.2%	107.2%
    BigSouth	0.4257	88.9%	43.2%	106.9%
    ACC	        0.6188	177.8%	69.7%	105.1%
    MountainWest	0.5234	104.3%	56.3%	104.2%
    MidAmerican	0.4823	100.0%	42.3%	103.3%
    AtlanticTen	0.4923	95.5%	46.9%	102.3%
    BigTwelve	0.5733	120.0%	61.7%	102.2%
    Ivy	        0.5227	100.0%	53.9%	100.7%
    BigSky	        0.4413	92.7%	44.5%	100.5%
    MissouriValley	0.4664	75.0%	48.9%	100.4%
    AmericaEast	0.4380	85.7%	45.2%	98.6%
    WestCoast	0.5625	122.0%	57.0%	98.5%
    ConferenceUSA	0.5364	96.6%	54.7%	94.9%
    AtlanticSun	0.4492	95.7%	48.1%	94.7%
    OhioValley	0.4313	104.3%	44.4%	93.8%
    Patriot	        0.4687	114.3%	51.0%	92.9%
    MetroAtlantic	0.4515	84.6%	42.8%	92.8%
    Summit	        0.4306	109.1%	45.5%	92.1%
    Colonial	0.5187	103.2%	50.4%	91.5%
    Southland	0.4478	84.8%	46.9%	87.4%
    Southern	0.4831	73.7%	50.6%	87.1%
    GreatWest	0.3557	75.0%	44.7%	87.1%
    SunBelt	        0.4787	100.0%	52.9%	85.4%
    Southwestern	0.3442	30.0%	34.4%	53.4%
    
    Region/Average ARPI (5 Years)/NCAA Performance Percentage/Home % of Non-Region Non-Neutral Games/Site-Corrected Performance Percentage

    Code:
    West	        0.5205	121.5%	56.1%	119.9%
    Middle	        0.4776	101.7%	44.7%	104.3%
    Southeast	0.5086	105.0%	57.0%	101.0%
    Northeast	0.4860	93.6%	42.9%	96.3%
    Southwest	0.4820	80.3%	51.0%	80.8%
    
    More to follow in Part VI (Continued).
  11. Soccerhunter Member

    Member Since:
    Sep 12, 2009
    Just a question about the value of 0.009 for home vs away tournament games. Does this figure come from regular season play and it is accurate for the top teams going off to play in the tournament? (My thought is that top teams might be less susceptible to H/A advantage disadvantage. In particular, for higher seeds any hiccups will mean losing at home, and lower seeds will score "upsets" in a "short tournament season".)
  12. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    Game Sites, Regions, and Conferences Part VI: Back to the Conferences and Regions Problem (Cont'd)

    Dang once more, I just noticed that there is a series of errors in the conference table in the previous post. Here is the correct table:

    Conference/Average ARPI (5 Years)/NCAA Performance Percentage/Home % of Non-Conference Non-Neutral Games/Site-Corrected Performance Percentage

    Code:
    PacTwelve	0.5953	119.6%	55.1%	116.3%
    SEC	        0.5741	117.1%	58.9%	116.1%
    BigTen	        0.5747	105.6%	58.7%	110.7%
    WAC	        0.4688	108.7%	46.3%	109.6%
    BigEast	        0.5529	112.0%	57.0%	107.7%
    BigWest	        0.5225	109.4%	53.7%	107.7%
    Northeast	0.4440	97.8%	40.4%	107.3%
    Horizon	        0.4672	107.1%	44.2%	107.2%
    BigSouth	0.4257	99.5%	43.2%	106.9%
    ACC	        0.6188	104.8%	69.7%	105.1%
    MountainWest	0.5234	100.4%	56.3%	104.2%
    MidAmerican	0.4823	106.0%	42.3%	103.3%
    AtlanticTen	0.4923	98.0%	46.9%	102.3%
    BigTwelve	0.5733	102.5%	61.7%	102.2%
    Ivy	        0.5227	101.1%	53.9%	100.7%
    BigSky	        0.4413	103.4%	44.5%	100.5%
    MissouriValley	0.4664	108.0%	48.9%	100.4%
    AmericaEast	0.4380	97.4%	45.2%	98.6%
    WestCoast	0.5625	104.9%	57.0%	98.5%
    ConferenceUSA	0.5364	94.6%	54.7%	94.9%
    AtlanticSun	0.4492	101.5%	48.1%	94.7%
    OhioValley	0.4313	80.7%	44.4%	93.8%
    Patriot	        0.4687	93.6%	51.0%	92.9%
    MetroAtlantic	0.4515	94.4%	42.8%	92.8%
    Summit	        0.4306	95.2%	45.5%	92.1%
    Colonial	0.5187	93.5%	50.4%	91.5%
    Southland	0.4478	87.2%	46.9%	87.4%
    Southern	0.4831	91.8%	50.6%	87.1%
    GreatWest	0.3557	88.2%	44.7%	87.1%
    SunBelt	        0.4787	90.0%	52.9%	85.4%
    Southwestern	0.3442	47.3%	34.4%	53.4%
    
    It may help if I explain the the table, as well as the region table in the previous post. For the conference table, the columns represent, in order:

    1. The Conference

    2. The Conference average RPI over the five year period, 2007 through 2011. This consists of the average Adjusted RPIs of the Conference's teams for each year, with those averages then averaged together to produce a five-year average.

    3. The Conference's "performance percentage" in relation to its average ARPI, over the five-year period. The ARPI here is computed strictly according to the NCAA formula. A performance percentage of 100% is the norm. A percentage above 100% means that the conference's teams on average, in relation to their ARPIs, perform better than the ARPI says they should, which means they are underrated. A percentage below 100% means they perform more poorly than the APRI says they should, which means they are overrated.

    4. The Conference's percentage of non-conference games (excluding neutral site games) played at home over the five-year period.

    4. The Conference's "performance percentage" in relation to its "site-corrected" ARPI, over the five year period. I described the site correction method in the previous email. This number represents strictly the extent to which the ARPI underrates or overrates conferences due to its problem rating them in a unified system.

    For the region table, the columns are the same except they relate to regions rather than conferences.

    I have set the columns up so that you can look at the third column to see the extent to which the NCAA formula overrates or underrates a conference (or region). Next, you can look at the fourth column to see the extent of the conference's home field advantage/disadvantage; finally you can look at the fifth column to see the extent to which the site-corrected formula overrates or underrates a conference. This lets you see how home field imbalances relate -- or don't relate -- to conferences' performance in relation to their ratings.

    In examining the conference table, I see some interesting results:

    1. There are some conferences that outperform their NCAA ratings and that have favorable home field imbalances that, after "site correcting" the ratings, outperform their NCAA ratings even more. How can this be the case? I believe this is due to what I mentioned a couple of posts ago, which is that the effect of the "site corrections" depends on the specific games that were home and away, who the opponents were, and what their rating differences were. A good example is the ACC, with its highly favorable home field imbalance. Contrary to possible expectations, when I correct for the imbalance, the ACC actually performs a very little bit better in relation to its ratings. It appears that the favorable home field imbalance is not artificially improving its ratings because its teams would have won the games no matter where they played them. And, for away games, the ACC's teams are doing very slightly better than their ratings say they should. This phenomenon is more significant for the Big Ten.

    2. For a good number of the conferences, the results are what one would expect: teams with favorable home field imbalances have lower performance percentages when the imbalances are factored out; and teams with unfavorable imbalances have higher performance percentages. In most cases, however, these differences are not extreme. For example, the Pac Twelve goes from a performance percentage of 119.6% to 116.3% after factoring out their favorable home field imbalance. And, the America East conference goes from 97.4% to 98.6% after factoring out their unfavorable imbalance.

    3. If I chart conferences' average ARPIs in relation to their site-corrected performance percentages and then create a trend line to show the relationship of the numbers, the stronger conferences on average still outperform their ratings and the weaker conferences still underperform. The levels of overperformance and underperformance are less, but only by about 20%. Thus, the average overrating and underrating of the NCAA formula is due only 20% to home field imbalances and is due 80% to the RPI's problem rating conferences in a unified system.

    The region table shows similar results to the conference table. A trend line for that table indicates that on average, the overrating and underrating of the NCAA formula is due about 25% to home field imbalances and about 75% to the RPI's problem rating regions in a unified system.
  13. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    If I understand your question:

    The 0.009 figure is derived only from regular season games (including conference tournaments). NCAA Tournament games do not figure into it at all. Indeed, my use of the 0.009 figure is not intended to relate to NCAA Tournament games. Rather, I purposefully derived it only from the regular season games because my interest is in how the various limitations of the RPI affect the Women's Soccer Committee's at large selections and seeding.

    On the other hand, although you may be right that the dynamics of the NCAA Tournament may be different than during the regular season, I'm not sure they're that different for top teams. The very top teams are competing very hard for #1 seeds in order to garner home field advantage, and they know that every game lost is a blow to their chances at a #1 seed. This year that was less the case, but in a number of years for a lot of teams, one loss could cost a #1 seed. I think the worry about a single loss is less when you get to the #2 through #4 seeds.

    And consider this: Suppose in the Tournament North Carolina had played UCF at home. For that game, I'm going with a likely 0.018 shift in the two teams' performance and saying North Carolina would have had a significantly better chance of winning.

    But again, the 0.009 is derived from and intended for the regular season.
  14. Time&Space Member

    Member Since:
    Sep 15, 2009
    Club:
    --other--
    When will final Rpi come out?
  15. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
  16. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    Game Sites, Regions, and Conferences Part VII: Should the RPI Consider Game Sites?

    My final question regarding the RPI and Game Sites, Regions, and Conferences is:

    Should the NCAA adjust the RPI so that it filters out the effects of game sites, thus being "corrected" for home field imbalances?

    The NCAA could do this. I believe the best way would be to subtract 0.009 from a team's RPI for each home game and add 0.009 for each away game. In effect, the formula for each team's overall site adjustment would be as follows, where H is the number of home games, A the number away, and N the number at neutral sites:

    Site adjustment = ((A x 0.009) - (H x 0.009))/(H + A + N)

    I believe the answer to this is a definite "NO," and here's why:

    1. As I've discussed earlier on this thread, there are home field imbalances and, on average, they follow a pattern. Stronger regions and stronger conferences tend to have favorable imbalances; and weaker regions and weaker conferences tend to have unfavorable imbalances. As a result of this, the fact that the RPI currently is not adjusted to reflect game sites, on average, benefits the ratings of the stronger regions and conferences and hurts the ratings of the weaker regions and conferences.

    2. At the same time, however, due to the structure of the RPI (and of any other statistical rating system) and to practical limitations on inter-region and inter-conference competition, the RPI tends to underrate stronger regions and conferences and to overrate weaker regions and conferences. On average, this benefits the ratings of the weaker regions and conferences and hurts the ratings of the stronger regions and conferences.

    3. When comparing the relative effects of these two aspects of the RPI, the impact of the RPI's region and conference problem is much greater than the impact of the game site problem. More specifically:

    For regions, the average benefit to the stronger regions of the RPI's not taking game sites into account is only 1/4 of the harm to the regions of the RPI's problem rating all the regions in a unified system (and the converse is true for the weaker regions). The fraction is even smaller for the Non-Conference RPI. Thus the RPI's not taking game sites into account is good because it partially offsets the RPI's region problem. If the RPI were modified to take game sites into account, the region problem would be even worse.

    For conferences, the average benefit to the stronger conferences of the RPI's not taking game sites into account is only 1/3 of the harm to the conferences of the RPI's problem rating all the conferences in a unified system (and conversely for the weaker conferences). The fraction is about the same for the NCRPI. Thus the RPI's not taking game sites into account is good because it partially offsets the RPI's conference problem. If the RPI were modified to take game sites into account, the conference problem would be even worse.

    Given the structure of the RPI, as well as the inherent practical problem of limited inter-region and inter-conference competition, any attempt to adjust the RPI to fix the region and conference problem would be very crude. Any attempted adjustment also would face significant "political" opposition within the NCAA. That being the case, it appears certain that the RPI is going to continue having its problem rating regions and conferences in a unified system; and so long as the RPI has that problem, the RPI should not be adjusted to take game sites into account, since such an adjustment would make the much more serious region and conference problem even worse.
  17. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    Your question got me to thinking. When a conference performs well over time in relation to its RPI ratings, this could be due to any or all of several possible causes:

    1. The conference is strong and is underrated due to the RPI's inherent tendency to underrate strong conferences;

    2. The conference has a favorable home field imbalance;

    3. The conference's opponents are overrated, so that the conference's wins and ties against the opponents are not as good as they seem.

    These are the causes I can think of, but there may be more.

    As discussed on this thread, I have created a system that separates out the effects of favorable and unfavorable home field imbalances. I realized it also would be possible to set up a system to approximate how well a conference's opponents perform, on average, in relation to their ratings. This can give a sense of whether at least part of the reason a conference performs better than its ratings is because most of its opponents are overrated.

    To illustrate how the system works, I'll use the ACC as an example. For each ACC team non-conference game over the five years from 2007 through 2011, I identify the non-ACC team's conference and the conference's five-year performance percentage in relation to the conference's five-year average rating (site-adjusted to take out the effects of home field imbalances). I then take the average of the opponents' conferences' performance percentages from all of these games to arrive at an average ACC opponents' performance percentage over the five year period. It's not an exact system, but it should give an approximation. Using this method, the ACC's five year opponents' performance percentage is 101.1%. This means the ACC's opponents, on average, perform close to their ratings, although they slightly outperform their ratings and therefore are slightly underrated.

    I set up the system for all of the conferences and for the five regions I use, and it has yielded some very interesting information. Since you asked about the SEC, and because it looked interesting to compare it to the Pac Twelve, I'll use those two conferences as an example of how the system works.

    As shown in prior posts on this thread, after making site adjustments to the RPI so that teams' performance percentages in relation to their Adjusted RPIs do not reflect the results of home field imbalances, the SEC's performance percentage is 116.1% as compared to the Pac Twelve's 116.3%. This suggests that both conferences are underrated, to about the same extent.

    BUT: What happens when I look at the two conferences' opponents' performance percentages (again after making site adjustments)?

    The SEC's opponents' conferences, on average, underperform in relation to their ratings. In other words, the SEC's opponents' conferences are overrated by the ARPI. Their average performance percentage is 97.5%.

    The Pac Twelve's opponents' conferences, on the other hand, on average out perform their ratings, meaning they are underrated by the ARPI. Their average performance percentage is 103.1%.

    What this means to me is that part of the SEC's high performance percentage in relation to its ARPI rating is not real: It is due to its opponents, on average, being overrated. On the other hand, the Pac Twelve actually is performing even better than its already high performance percentage indicates: This is due to its opponents, on average, being underrated.

    Also of interest to me, the West Coast Conference's non-conference opponents, on average, appear to be the most underrated, with an average performance percentage of 107.5%. Further, of the six conferences with the highest opponents' average performance percentages, five are from the West region: West Coast (107.5%), Big West (104.5%), Big Sky (103.6%), Pac Twelve (103.1%), and Mountain West (102.9%). The other of the six is the Big Ten (103.7%). The West region situation reflects the fact that the West region teams' performance in relation to ratings, in games against teams from other regions, is far better than the performance in relation to ratings of any of the other regions.
  18. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    For those (few?) of you who are tracking the question of how the RPI does rating conferences and regions in a unified system, here are two tables with resource information, one for conferences and the other for regions. Each table contains three pieces of information:

    Average ARPI: This is the average ARPI for the conference or region, after correcting the ratings game-by-game based on the game location so as to remove the impact of home field imbalances.

    Performance Percentage: This is the conference's or region's teams' performance percentage in non-conference or non-region games, in games in which the ARPI rating difference was less than 0.05 (again after correcting based on game location). A performance percentage of 100% is the norm. A percentage above 100% means the conference's or region's results are better than the ratings say they should have been. The converse is the case for percentages below 100%.

    Opponents' Performance Percentage: This is an approximate measure of the performance percentage of the conference's or region's opponents, in games in which the site-corrected ARPI rating difference was less than 0.05 (again after correcting based on game location).

    Conferences

    Conference/Average ARPI/Performance Percentage/Opponents' Performance Percentage

    Code:
    ACC	        0.6205	105.1%	101.1%
    PacTwelve	0.5943	116.3%	103.1%
    BigTen	        0.5753	110.7%	103.7%
    SEC	        0.5752	116.1%	97.5%
    BigTwelve	0.5735	102.2%	102.1%
    WestCoast	0.5632	98.5%	107.5%
    BigEast	        0.5538	107.7%	102.0%
    ConferenceUSA	0.5370	94.9%	98.2%
    MountainWest	0.5252	104.2%	102.9%
    BigWest	        0.5231	107.7%	104.5%
    Ivy	        0.5229	100.7%	100.4%
    Colonial	0.5188	91.5%	100.6%
    AtlanticTen	0.4923	102.3%	99.8%
    Southern	0.4836	87.1%	101.0%
    MidAmerican	0.4825	103.3%	101.7%
    SunBelt	        0.4793	85.4%	93.9%
    WAC	        0.4699	109.6%	98.6%
    Patriot	        0.4693	92.9%	100.2%
    MissouriValley	0.4662	100.4%	99.4%
    Horizon	        0.4654	107.2%	101.7%
    MetroAtlantic	0.4515	92.8%	100.2%
    AtlanticSun	0.4492	94.7%	94.9%
    Southland	0.4482	87.4%	84.3%
    Northeast	0.4441	107.3%	97.0%
    BigSky	        0.4413	100.5%	103.6%
    AmericaEast	0.4385	98.6%	99.7%
    OhioValley	0.4310	93.8%	95.5%
    Summit	        0.4306	92.1%	99.7%
    BigSouth	0.4258	106.9%	92.7%
    GreatWest	0.3693	87.1%	97.7%
    Southwestern	0.3428	53.4%	92.2%
    
    Regions

    Region/Average ARPI/Performance Percentage/Opponents' Performance Percentage

    Code:
    West	        0.5212	119.9%	83.8%
    Southeast	0.5095	101.0%	100.9%
    Northeast	0.4864	96.3%	104.9%
    Southwest	0.4833	80.8%	98.4%
    Middle	        0.4779	104.3%	96.8%
    
  19. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    Based on the Conferences table in the preceding thread, here are my tentative judgments about how the conferences' average ARPI ratings (after removing the impact of home field imbalances) relate to the conferences' actual strength based on their performance over the last five years. The conferences are in order of their average ARPIs:

    ACC: ARPI underrates.

    Pac Twelve: ARPI very greatly underrates.

    Big Ten: ARPI greatly underrates, but not nearly as badly as the Pac Twelve.

    SEC: ARPI underrates, perhaps quite a bit but not as badly as one might think.

    Big Twelve: ARPI underrates.

    West Coast: ARPI underrates, perhaps quite a bit. Although the WCC slightly underperforms in relation to its teams' ratings, its opponents outperform their ratings by significantly more than for any other conference. This indicates that its slight underperformance in relation to its teams' ratings is misleading.

    Big East: ARPI quite significantly underrates, though not as greatly as the Big Ten.

    Conference USA: ARPI quite significantly overrates.

    Mountain West: ARPI underrates, about to the same extent as the ACC.

    Big West: ARPI quite significantly underrates, moreso than for the Big East but not as greatly as the Big Ten.

    Ivy: ARPI rates just about right.

    Colonial: ARPI quite significantly overrates.

    Atlantic Ten: ARPI slightly underrates.

    Southern: ARPI quite significantly overrates.

    Mid American: ARPI underrates.

    Sun Belt: ARPI greatly overrates.

    WAC: ARPI underrates.

    Patriot: ARPI quite significantly overrates.

    Missouri Valley: ARPI rates just about right.

    Horizon: ARPI very significantly underrates.

    Metro Atlantic: ARPI quite significantly overrates.

    Atlantic Sun: ARPI greatly overrates.

    Southland: ARPI tremendously overrates.

    Northeast: ARPI rates just about right.

    Big Sky: ARPI underrates.

    America East: ARPI slightly overrates.

    Ohio Valley: ARPI greatly overrates.

    Summit: ARPI quite significantly overrates.

    Big South: ARPI overrates.

    Great West: ARPI greatly overrates.

    Southwestern: ARPI grossly overrates.
  20. Bobby R New Member

    Member Since:
    Nov 15, 2011
    Club:
    Manchester United FC
    Cpt
    If Clemson ranked 147 in rpi played in a weaker conference like the Atlantic 10 or Atlantic sun would you guess a better year end rpi assuming their non conference schedule were the same?
  21. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    In almost all cases, a team's RPI gets hurt if the team loses a game and gets helped if a team wins a game. Looking at the Atlantic Sun, it's possible Clemson could have run the table in the conference. That would have left it with a 17-2 record or better, depending on the Atlantic Sun Tournament outcome. I don't think there's any doubt that it's RPI would have been better. Forgetting about qualifying for the NCAA Tournament by winning the conference tournament, however, that record would not have gotten Clemson into the NCAA Tournament. It's non-conference schedule would have been way too weak. (Its non-conference RPI rank was #121.) In order to get into the Tournament, it would have needed some good wins/ties. To get some good wins/ties, it would have had to play a significant number of non-conference games against very good teams. If you assume it would have lost those games, then all of a sudden its RPI isn't so good.
  22. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    I already have written about the likelihood that part of the reason the SEC outperforms its RPI ratings is that its average opponent is overrated by the RPI. So, for the SEC, what look like good results for its teams in relation to their ratings may not actually be that good -- rather, the results look good because SEC opponents, on average, actually aren't as good as their ratings indicate.

    The West region also outperforms its RPI ratings. I started wondering if the same thing could be true of the West region.

    On Post 18 on this thread, I provided a table showing, for each region, its five year performance percentage in relation to the region's average RPI and also its opponents' five year approximated performance percentage in relation to the opponents' RPIs. Unfortunately, I forgot to give my computer the right instruction before producing that table so it isn't right. Here's the corrected table. It's for the five years from 2007 through 2011, and it is "corrected" to remove the effects of home field imbalances. It's in order from the region with the highest average ARPI to the region with the lowest:

    Region/Average ARPI for Region's Teams/Region's Average Performance Percentage/Opponents' Average Performance Percentage

    Code:
    West	        0.5212	119.9%	93.2%
    Southeast	0.5095	101.0%	95.9%
    Northeast	0.4864	96.3%	103.1%
    Southwest	0.4833	80.8%	107.2%
    Middle	        0.4779	104.3%	97.9%
    
    I'll make a couple of initial comments about this:

    First, obviously the West region's ARPI is much higher than any other region's. The Southeast is next, with a big gap between it and the other three regions.

    Second, notwithstanding that the West region's average ARPI already is much higher, it also very greatly outperforms its ratings. In fact, the region outperforms its ratings by a greater margin than any single conference outperforms its ratings. The Southeast region performs just about in accord with its ratings. The Northeast performs somewhat more poorly than its ratings say it should; and the Midwest performs better than its ratings say it should. The Southwest, on the other hand, performs much more poorly than its ratings say it should.

    Third, when I look at how each region's non-region opponents performed in relation to their ratings, they are something of a mirror image. In other words, if a region outperforms its ratings, then its non-region opponents underperform and vice versa.

    What I was wondering about was the West region in particular. I thought that perhaps the reason it does so much better than its ARPIs say it should is because it primarily is playing non-region opponents that are overrated. In other words, maybe its situation in the regional setting is like the SEC's seems to be in the conference setting.

    Because this seems to me to be an important question, I decided to break down each region's opponents, by conference, to see if by chance the West region is scheduling less games against strong conferences with strong performance percentages than the other regions are. I looked to see the percentage of its total non-region games that a region scheduled against each other conference. I then focused in on the percentage that were against each of the top 7 conferences in terms of average ARPI. The seven conferences, in order of average ARPI (corrected for home field imbalances) are the following, with their ARPIs and also their performance percentages:

    Conference/Average ARPI/Average Performance Percentage

    Code:
    ACC	        0.6205	105.1%
    PacTwelve	0.5943	116.3%
    BigTen	        0.5753	110.9%
    SEC	        0.5752	116.1%
    BigTwelve	0.5735	102.2%
    WestCoast	0.5632	98.5%
    BigEast	        0.5538	107.7%
    
    As is apparent, not only do these conferences have the highest average ARPIs, but they also all -- except for the West Coast Conference -- perform better than their ARPIs say they should. The WCC performs slightly more poorly than its ARPIs say it should, but as pointed out in a previous post, this appears due to the fact that its opponents, on average, significantly outperform their ratings, to a significantly greater extent than for any other conference.

    The following table shows the percentages of each region's non-region games that were against these conferences, followed by the total percentage against these conferences:

    Code:
    Region        	ACC	PacTwelve	BigTen	SEC	BigTwelve	WestCoast	BigEast	Total
    Middle        	1.9%	3.6%        	0.8%	5.9%	9.1%        	2.5%        	12.2%	35.9%
    Northeast	13.1%	3.8%        	10.4%	4.1%	3.0%        	3.7%        	0.0%	38.2%
    Southeast	0.0%	3.7%        	4.0%	0.0%	5.3%        	2.3%        	5.9%	21.2%
    Southwest	4.2%	6.4%        	4.2%	14.2%	0.0%        	3.8%        	2.3%	35.2%
    West        	5.3%	1.9%        	8.7%	4.8%	13.5%        	0.0%        	4.8%	39.0%
    
    You'll have to scroll to the right on the table to get the totals.

    The table is interesting:

    First, rather than suggesting that the West region is playing teams that are overrated, the region is playing a higher percentage of its games than any other region against the top conferences.

    Second, the Southeast region is playing very significantly lower percentage of its games than any other region against the top conferences. In relation to the Middle, Northeast, and Southwest regions, this is partly explainable by the fact that the Southeast region has two conferences among the top 7, so in the table it gets credit for games against only 5 of the strong conferences rather than 6. The West region, however, is in the same boat as the Southeast region.

    Overall, this appears to indicate that the West region, in fact, is as strong as the combined ARPI and Performance Percentage numbers indicate and is greatly underrated by the ARPI. Looking at the numbers as applied to each region leads me to believe the following regarding the regions overall:

    West region: Is very significantly underrated.

    Southeast region: Is somewhat overrated.

    Northeast region: Is rated about right.

    Southwest region: Is significantly overrated.

    Middle region: Is rated about right.

    Of course, within each region, conferences themselves can be overrated or underrated.
  23. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    In an earlier post, I provided a five-year analysis, based on how regions' teams performed in actual games as compared to the regions' average ratings, and included my conclusions as to how the RPI fares at rating the regions in a unified system. My conclusions were:

    I decided to use Massey's rankings, to see if they corroborate my conclusions. I only have two years' data for Massey, so I compared his rankings in 2010 and 2011 to the RPI's rankings. What I looked at was the average difference between Massey's rankings and the RPI's rankings for the teams from each region. The following table shows the results. The number in the right hand column is the average difference between Massey's ranking and the RPI's ranking. A negative number means that Massey gave the region's teams a better ranking (suggesting, from Massey's perspective, that the RPI underrates the region's teams) and a positive number means that Massey gave the region's teams a poorer ranking (suggesting, from Massey's perspective, that the RPI overrates the region's teams).

    Region/Average Ranking Difference

    Code:
    Middle        	-7.8
    Northeast	3.2
    Southeast	8.2
    Southwest	19.8
    West        	-27.0
    
    Comparing what Massey indicates to my own conclusions:

    West region: Massey corroborates that the West region is very significantly underrated.

    Southeast region: Massey corroborates that the Southeast region is somewhat overrated.

    Northeast region: Massey corroborates that the Northeast region is rated about right, but suggests it possibly is slightly overrated.

    Southwest region: Massey corroborates that the Southwest region is significantly overrated.

    Middle region: Massey indicates that the Middle region is somewhat underrated, which is different than my conclusion it is rated about right.

    Overall, Massey pretty much agrees with my analysis. It would be better, however, if I had three more years of his data.
  24. cpthomas BigSoccer Supporter

    Member Since:
    Jan 10, 2008
    Location:
    Portland, Oregon
    Country:
    United States
    I did the same analysis using Massey for conferences as I described in the preceding post for regions. Here are my previous conclusions regarding how the RPI rates the conferences:

    The following table shows how Massey ranked the conferences' teams as compared to the RPI's rankings. Again, a negative number means Massey gave the conference's teams better rankings (meaning that, according to Massey, the RPI underrated the teams) and a positive number means Massey gate the conference's teams poorer rankings (meaning that, according to Massey, the RPI overrated the teams):

    Conference/Average Ranking Difference

    Code:
    ACC        	1.4
    AmericaEast	-3.1
    AtlanticSun	9.5
    AtlanticTen	5.7
    BigEast        	-9.5
    BigSky        	-44.5
    BigSouth	10.2
    BigTen	-17.9
    BigTwelve	-1.2
    BigWest        	-32.0
    Colonial	4.8
    ConferenceUSA	14.9
    GreatWest	-1.7
    Horizon        	-11.9
    Ivy        	7.9
    MetroAtlantic	6.8
    MidAmerican	-7.6
    MissouriValley	-9.6
    MountainWest	-28.2
    Northeast	11.5
    OhioValley	4.6
    PacTwelve	-19.2
    Patriot        	11.5
    SEC        	0.3
    Southern	19.6
    Southland	48.7
    Southwestern	10.4
    Summit        	-5.5
    SunBelt        	34.5
    WAC        	-21.4
    WestCoast	-15.5
    
    Comparing Massey to my own conclusions:

    ACC: Massey says the RPI rates them about right, whereas I had them underrated.

    PacTwelve: Massey says that the RPI greatly underrates the PacTwelve. This corroborates my conclusion.

    BigTen: Massey says that the RPI greatly underrates the BigTen. This corroborates my conclusion, except that Massey has them more underrated than I did.

    SEC: Massey says that the RPI rates the SEC just about right. I had them quite a bit overrated.

    BigTwelve: Massey says that the RPI rates the BigTwelve just about right. I had concluded that the RPI underrated them.

    WestCoast: Massey says that the RPI significantly underrates the WestCoast. This corroborates my conclusion, but I was a little less definite.

    BigEast: Massey says that the RPI significantly underrates the BigEast. This corroborates my conclusion.

    Conference USA: Massey says that the RPI significantly overrates CUSA. This corroborates my conclusion.

    Mountain West: Massey says that the RPI greatly underrates the MountainWest. I concluded that the RPI underrated the MountainWest, but nowhere near to the extent that Massey suggests.

    BigWest: Massey says that the RPI greatly underrates the BigWest. This corroborates my conclusion, except that Massey's suggested underrating is much greater.

    Ivy: Massey says that the RPI overrates the Ivy. I had the RPI rating them just about right.

    Colonial: Massey says that the RPI slightly overrates the Colonial. I had the RPI quite significantly overrating them.

    Atlantic Ten: Massey says that the RPI overrates the AtlanticTen. I had the RPI slightly underrating them.

    Southern: Massey says that the RPI greatly overrates the Southern. This corroborates my conclusion.

    MidAmerican: Massey says that the RPI underrates the MidAmerican. This corroborates my conclusion.

    SunBelt: Massey says that the RPI greatly overrates the SunBelt. This corroborates my conclusion.

    WAC: Massey says that the RPI greatly underrates the WAC. This corroborates my conclusion.

    Patriot: Massey says that the RPI significantly overrates the Patriot. This corroborates my conclusion.

    MissouriValley: Massey says that the RPI significantly underrates the MissouriValley. I had the RPI rating them just about right.

    Horizon: Massey says that the RPI significantly underrates the Horizon. This corroborates my conclusion.

    MetroAtlantic: Massey says that the RPI overrates the MetroAtlantic. This corroborates my conclusion, although I had the overrating somewhat greater.

    AtlanticSun: Massey says that the RPI overrates the AtlanticSun. This corroborates my conclusion, although I had a greater overrating.

    Southland: Massey says that the RPI tremendously overrates the Southland. this corroborates my conclusion.

    Northeast: Massey says that the RPI overrates the Northeast. I had the northeast rating them just about right.

    BigSky: Massey says that the RPI tremendously underrates the BigSky. This corroborates my conclusion, except that I had nowhere near as great an underrating.

    AmericaEast: Massey says that the RPI slightly underrates the AmericaEast. I had the RPI slightly overrating them.

    OhioValley: Massey says that the RPI slightly overrates the OhioValley. I had the RPI greatly overrating them.

    Summit: Massey says that the RPI slightly underrates the Summit. I had the RPI quite significantly overrating them.

    BigSouth: Massey says that the RPI overrates the BigSouth. This corroborates my conclusion.

    GreatWest: Massey says that the RPI rates the GreatWest just about right. I had the RPI greatly overrating them.

    SWAC: Massey says that the RPI overrates the SWAC. I had the RPI grossly overrating them. However, given my methodology in using Massey, these really aren't different conclusions. The SWAC teams are at the very bottom of Massey's rankings, so they couldn't have ranked any lower.

    Overall, although my conclusions and the Massey results don't always match, the Massey results in general corroborate my conclusions about the conferences.
  25. justdoit Member

    Member Since:
    Aug 11, 2009
    Club:
    Manchester United FC
    What about having NCAA play in games??? The conferences in which the championship team representing their conference has an RPI of 65 or high must play a playin game.


    Metro Atlantic: Marist - RPI#124

    Atlantic Sun: FGCU - RPI#71

    Southland: Texas St - RPI#72

    Northeast: LIU - RPI#89

    Big Sky:Montana - RPI#267

    Ohio Valley: UT Martin RPI#116

    Summit: Oakland RPI#177

    Big South: Radford RPI#132

    Southwestern: Ark PB RPI#219

    Mountain West: New Mexico RPI#68

Share This Page