On the misleading nature of hatstats ratings

[ur_land]

I guess I've been obsessing about ratings lately (and this is another continuation of a converstaion I mostly had with pal in my series conference)......

The way that hatstats ratings are set up, midfield, offense, and defense are roughly equally weighted (there are thrre ratings each for offense and defense, and and one rating weighted three times for midfield).

But there is also a consensus out there (one that I agree with), that the three types of ratings are not equal when it comes to the game engine. My best guess is that midfield is most important, followed by offense, and then defense. According to conventional wisdom, a passable offense should be able to regularly score on a passable defense-they are not equal ratings.

So how should we adjust hatstats-type ratings to reflect this? Should we perhaps multiply midfield by 4 and each defensive rating by 1.5? Why did I have a dream las tnight that Leanne Rimes had eczema on her herpes outbreak? Does anyone know of any data that looks at the relative effects of offense vs. defense? Anyone have any better proposals (and I'm not advocating mine is the correct one, since I pretty much just pulled it out of my ass)?

[Pal11]

But there is also a consensus out there (one that I agree with), that the three types of ratings are not equal when it comes to the game engine. My best guess is that midfield is most important, followed by offense, and then defense. According to conventional wisdom, a passable offense should be able to regularly score on a passable defense-they are not equal ratings.

I believe Bjorn said as much in a recent thread in global. I think it was something to the effect of you can't compare an attack rating to a defense rating because they don't necessarily have the same weight in the match engine. Which I think most people figured anyway based on experience.

Have you checked out this site? Hattristics (http://www.hattristics.org/pub/index.php)
Their conversion page has a breakdown of how many goals you can expect to score based on the attack/defense sector ratings. However, with the recent defensive adjustment that happened before this season I'm not sure if we really have enough data to estimate the current a/d weights within the match engine. Although that adjustment didn't help me out at all vs. NYFC... :o

Oh, and on the Leanne Rimes question, seek help. Soon! :eek: ;)

[ur_land]

Have you checked out this site? Hattristics (http://www.hattristics.org/pub/index.php)
Their conversion page has a breakdown of how many goals you can expect to score based on the attack/defense sector ratings. However, with the recent defensive adjustment that happened before this season I'm not sure if we really have enough data to estimate the current a/d weights within the match engine. Although that adjustment didn't help me out at all vs. NYFC... :o


Yes, the hattristics site was exactly what I was looking for. Looking at the numbers (rather quickly) it seems that the goal rate (goals)/(goal chances+ missed chances) doesn't even out until defense is a full level (four sublevels) higher than the offense. This is probably overstating the case, since some chances aren't reported, so maybe a good fix to the offensive vs. defensive hattrick ratings would be to add 8 points to the offensive ratings. Now the only question is how to weight midfield.

Actually, there is another question. What is the criterionthat we are comparing the rating to? In other words, we look at hattstats ratings to determine which team is the "better" team. Is there an outside criterion to tell us that making an adjusted ratings system would be a better measure of which team is "better"? I'm thinking maybe seeing whether hatstats ratings or adjusted hatstats ratings better predict goal differential or win% or total points, or something. Any thoughts?

[ur_land]

Eureka!

I think I've got it.

To predict what the goal differential of your game is going to be, use this formula:

10*[[your mf/(your mf+opp mf)][the conversion chance for your avg. offense zone rating versus his avg defense zone rating]-[opp mf/(your mf+opp mf)][the conversion chance for opp avg. offense zone rating versus your avg defense zone rating]]

You get the conversion ratings from hattristics: http://www.hattristics.org/pub/statConversion.php

For example: in my recent game against pal (which I won), the ratings were:

my offense: 47
my mid: 36
my D: 41

pal's offense:37
pal's mid:18
pal's D:72

this is a classic match of a balanced team vs. a defense trainer. Who should win?

my average offensive rating was ~8 levels lower than his average defensive rating [(47-72)/3=8.33]. According to hattristics, that means I should score on ~40% of my chances.

pal's average offensive rating was ~3 lower than my average defensvie rating. He should score on ~53% of his chances.

plugging into the formula we get:

10*[[36/(36+18)][.4]-[18/(18+36)][.53]]=10*[.667(.4)-(.333)(.53)]=10(.267-.176)=10(.091)=.91

I should have beaten pal by a little less than one goal. Take away his counterattack goal and my penalty kick, and the score was 1-0.

I've only tried this out on 4 matches so far, but it's worked every time......

[ur_land]

One thing not mentioned in the above formulation is that this takes care of the relative weighting of offense and defense by using the chance conversion ratio, and takes care of the criterion question by trying to predict goal differential.

[Craig P]

I think all Hatstats ever set out to do was create a quick "total rating", and that just multiplies midfield in 3* for symmetry with attack and defense (which are broken down zonally whereas midfield is applied all over the field).

It has been stated that the attack ratings were scaled back several seasons ago, in a foreshadowing of the switch to new-style ratings: apparently, it was too easy to get divine attack ratings. As a result, equal ratings text-wise actually favor the attack.

I need to see if I can do something automated with that margin predictor -- I've got a lot of match results stored and I'd be interested in seeing how well they agree.

[ur_land]

I need to see if I can do something automated with that margin predictor -- I've got a lot of match results stored and I'd be interested in seeing how well they agree.

Please feel free to test it all you can. It's not going to be 100% accurate (it will be affected by the distribution of chances and by special events and set pieces), but it's looked pretty good so far in the games I've spot checked. The biggest discrepancy came in the the recent Shaolin Landfills--Cyclones game, where the formula predicted a 2-goal landfills win. Landfills actually had a 5 goal margin (when you remove set pieces an dspecial events).

[ur_land]

I think all Hatstats ever set out to do was create a quick "total rating", and that just multiplies midfield in 3* for symmetry with attack and defense (which are broken down zonally whereas midfield is applied all over the field).


It's taken me a few days to get back to this point.....

I love the hatstats ratings, and I think the best feature of the firefox hattrick extension is the ability to display the hatstats ratings on the game page. I think the hatstats ratings are a good snapshot, and very useful in measuring team inprovement over time. But I also think that having equal weights for offense, defense, and midfield is misleading when you want to quantitatively predict who will win a game or why a certain team lost a game (witness all of the "random is evil" or "Why did I lose" threads on the conferences).

In that sense I think that the formula I came up with is better. Let's step back a second and think about the function of each of the types of ratings in the game engine:


midfield: helps determine possession, which helps determine the number of chances a team gets.

offense: helps determine wheter a chance will lead to a goal.

defense: helps determine whether a chance will not lead to a goal.

So in hattrick, midfield should not be grouped with offense or defense--it should be on a different metric entirely, becasue its function is different. And offense of one team should always be considered in the context of the defense of the other team, since the tension between those two will be what determines the success of the chance. That's what that formula describes. I don't think I'm describing anything new here, and I really hope that either craig or I or someone has the time to check this against actual results (I'll be doing that for my series this season).

[puddleduck]

firefox hattrick extension

Say what batman? How did i miss this? *Runs to google*

[ur_land]

Say what batman? How did i miss this? *Runs to google*


Search for foxtrick.

It's got some fun, some useful, and some useless features.

[ur_land]

Formula update:

The chance of getting an attack on one of the wings is a little less than the chance of getting an attack in the middle. The first formula is good for a quick and dirty analysis, here's a more updated version:

10*[[your mf/(your mf+opp mf)][(.25)(the conversion chance for your right offense zone rating versus his left defense zone rating)+(.25)(the conversion chance for your left attack versus his right defense)+(.3)(the conversion chance for your center attack versus his center defense)]-[opp mf/(your mf+opp mf)][(.25)(the conversion chance for opp. right offense zone rating versus your left defense zone rating)+(.25)(the conversion chance for his left attack versus your right defense)+(.3)(the conversion chance for his center attack versus your center defense)]]

Again, you get the conversion ratings from hattristics: http://www.hattristics.org/pub/statConversion.php

You might notice that .25+.25+.3 = .8 That's b/c the other 20% of chances seem to come from set pieces (~10%) and special events (~10%). I have no idea how to add those into the formula. If you have a crappy GK and he has a great sp taker (or vice versa) i might add .1*possesion goals to the GD.

[kuhnscoot]

With this whole thread, I'm just nodding my head and smiling, :D

All right over my head. I'll just rep you for this now ur_land if I could

[ur_land]

With this whole thread, I'm just nodding my head and smiling, :D



In my other life I'm a statistics geek. ;P

[kuhnscoot]

In my other life I'm a statistics geek. ;P

Other life? Every life is more like it, get a girlfriend. ;)

[ur_land]

Other life? Every life is more like it, get a girlfriend. ;)

I don't think my wife would be very happy about that.