[Outdated] Rating correction and changes

What’s wrong with handicaps now?

I haven’t played any handicap games recently but last time I did, it seemed the distance between 2 ranks was way less than 1 stone, so I could probably win close to 100% of my games having 4 stones vs someone allegedly 4 ranks stronger, and really struggle when giving 4 stones to someone 4 ranks weaker. This isn’t a problem I found on DGS or KGS. By the sounds of things this update should help.

1 Like

Yeah that’s not a handicap issue per-se, that’s a rank issue.

Any plans to make this rating correction a monthly “feature”?
As OGS will most likely grow continuously before archieving world domination. The ranks will be pretty chaotic because of the influx of new people.
I don’t know if you have plans to change the ranking system(most likely not) and OP said not to discuss that here.
So doing these “corrections” once a month would be a hybrid ranking system :slight_smile:
Pure ELO/EGF system but once a month all the recent games gets “analyzed”(whatever your method was) and then the ranks will be “corrected”.

2 Likes

The reason for this adjustment is that we are injecting a lot of points before we replay history to bring peoples ratings more in line with the new desired rank range, and then we are also getting rid of the negative effect of most of the corr. timeouts, so re-running the ratings keeps things fair moving forward and actually changes things.

Going forward though, unless we were injecting points or changing something else about the rating adjustment algorithm, if we were to re-run the “correction” every month people would end up in the exact same spots :smile:

So while we wouldn’t rule out doing future adjustments if things got out of wack somehow, I’d much rather fine tune the rating system so such big adjustments would not be necessary.

2 Likes

Second what @anoek said, and I’ll add that provisional ranks are a good solution against new player’s inacurate ranks.

1 Like

So when is this change due to come into effect? Someone mentioned the 27th, but it is past that now.

It would seem my question has been answered. :slight_smile:

It’s still the 27th PDT, so it still counts!

Patch is in and active, let a moderator know if you believe your rank is very wrong and want it reset without the normal provisional penalty, but please compare yourselves to other players you know on the system to see how you relate to their new ranks before jumping to the conclusion that you’ve been set to high :slight_smile:

6 Likes

Had a quick look through my friends list, and it all seems entirely as I’d expect.

Great stuff, well done team.

1 Like

Overall it looks good.

The only strange thing I noticed is that the Nanny (GnuGo Lvl1) bot is ranked higher than the Billy (GnuGo Lvl10) bot.

I suspect a lot more newer people play against Nanny (and lose), giving it points. Maybe the bot-herder could get the bots to play a bunch of games against each other at high speed to keep the ranks from drifting? Not sure I’ve thought that idea through very well though.

Uhm, I was promoted to 11 kyu :open_mouth:

This means I’ll get a lot of beating now since my actual rank seems to be more like 13k (at least that’s what my EGF rank is and all my other ranks were, IIRC, before I came here).

I think I may set it back to 13k after gathering some experience with the current rank setting.

Greetings, Tom

“Heavy baby”.
I am a relative beginner and have jumped from 30 K to 18 K.
My reaction is “Wow”.
I have skimmed through the explanation. It will be interesting to see how it pans out.
My respects
C

1 Like

This post was flagged by the community and is temporarily hidden.

1 Like

Yeah, I also suspect that it has a lot to do with the different types of crowd that play each bot. It could be interesting to enter the bots into some ladders, with them accepting all challenges and randomly issuing some of their own, to give them exposure to a broader crowd of players.

I don’t believe any of these bots learn/improve, so their true rating should be theoretically stationary.

Doing something to stabilize the bot rankings would probably make sense. Maybe a smaller K factor should be used for adjustments to the bot’s rating (while still using a normal K factor in adjusting the human opponent’s rating). This would slow down the drift and decrease the impact any one game, but the bots still play more than enough games to converge to an accurate rating. Perhaps, the K factor (for the bot) should be decreased even further for games against players with low rankings or few games.

Having them play lots of games against each other would do the trick, wouldn’t it?

Yeah, that would probably work toward fixing their relative rankings, but it might also have some drawbacks:

  1. It doesn’t address the issue of random players causing needless swings in the bot ratings in the first place.
  2. I imagine that the bots might play more consistently than humans. By that, I mean that a match-up between two bots that are only slightly different in strength (when compared via their performance against humans) might wind up being much more one-sided (in terms of winning percentage) than expected. The stronger of two humans may win more often, but there is still the variability inherent to human nature. The Elo system is designed to take that into account when estimating the expected win rate. Playing bots against each other might quickly correct an inverted relative rating, but it might end up driving their rankings further apart than they actually should be.
  3. It seems like a waste of CPU cycles.
2 Likes

These are all very good points.

Perhaps there could be some sort of automatically detecting big swings in a bot’s ranking. If, say, a 20k resigns to RandomBot because something suddenly comes up that means they can’t complete the game, then this would cause a big spike in RandomBot’s ranking. If OGS detects that, then Randy could play a few games against other bots to settle things down again.

That is true. One way of dealing with this is with handicaps. If Bot A beats Bot B most of the time with 3 stones handicap, but almost never with only two stones, then the bots are between 2 and 3 ranks apart. If they play even games all the time then, yes, the ranking difference will be inflated because of the consistency of computer players. Besides, a little bit of “forced separation” in the system would not be a bad thing because the major drawback of the previous rating system was that the ranks were all bunching up.

So far so good. But there are now some challenges on the open games list that don’t make sense.

(For example, players who challenged at 29k and are now 15k after adjustment, but I can’t accept them as 12k because the rank difference is supposedly more than nine stones as it’s measured against the time they put the challenge up.)