Are OGS rankings inflated, deflated or neither?

Semi-fork of Is rating volatility a bug or a feature? [Forked]

Accurate rankings are useful to Go players for many reasons:

  • Finding suitable opponents for even games.
  • Setting a fair handicap for handicap games.
  • Context in discussions, especially game reviews and teaching games.
  • Tracking their progress.
  • Choosing study materials (many are marked as for certain skill levels).

The earlier topic was about volatile ranks, but I believe highly inflated or deflated ranks can equally cause credibility issues (like suspicions of cheating).

In this game, my opponent (12 kyu) asked me whether I was 6 kyu (this was before I even finished a game):

Turns out new accounts appear as 6 kyu, which is too strong. A game between a new (TPK) player and a real 6 kyu would not be fun for either player.

After five losses to 11-12 kyu opponents and one win, I finally got ranked 16 kyu, which seems reasonable.

What is strange is my only win. My 14 kyu opponent started with saying “I don’t know how to play. I hope you can give me more advice!” That does not sound like a 14 kyu. I thought a 14 kyu would be able to kill my top group and would not spend 50 moves on an obviously pointless invasion.

Just wondering what the OGS community thinks about whether OGS ranks are inflated or deflated and the impact of such inflation or deflation.

2 Likes

It depends on what you’re comparing OGS ranks with, and it also depends on which rank range you’re looking at.
The 2021 rating update aimed at aligning OGS 1d between AGA 1d and EGF 1d. My impression is that this goal was achieved.
As for OGS DDK ranks, I feel they are softer than EGF DDK ranks. I don’t know how they compare to AGA DDK ranks.

2 Likes

The title of the thread doesn’t correspond to the first message which points out the fact (already mentioned multiple times on this forum) that starting all new accounts at 6k makes no sense, especially for beginners.

4 Likes

EDIT: turns out I was wrong, provisional ranks are indeed at 6 kyu, even though they are displayed in the graph as 11.9 kyu! [End of Edit]

 Starting all accounts at 12k (which is where players actually start, by the way, I think 6k might have been an older rating system) makes a lot of sense, especially as a partial counter-measure to sandbagging. We can’t have beginners self-identify, and AFAIK we don’t have the resources to have every beginner be vetted by a trusted user (unless we organize such a system very carefully, but even then I wonder), so a good method is to let the system, which is well-suited for it, guess the rating over the course of a few games.

 From the point of view of a beginner, it’s a bit of a trial of fire, but I personally believe it’s mostly painless for most people, as it only takes a few games, and if you’re a beginner hopefully you don’t expect to win all your first games.

 What definitely doesn’t make sense is to have a user’s rank be visible (in the profile page) until it’s established. There are preconceptions as to what a “12 kyu player” should be able to do, and most people are unable to interpret the concept of “uncertainty” (11.9kyu ±4.9 basically means “anywhere between 25kyu and 3dan”, if I’m correct).

 So I’d say it’s better to completely hide the rank, plotting and profile page included, until there’s a good amount of info necessary to guess it.

1 Like

Compared to EGF - slightly inflated in dan ranks, inflated in ddk region. Compared to other servers - should be deflated, 1d OGS can be 5d on Foxy.

6k (oh, so it’s not even 6k anymore) as provisional rank makes some sense because lots of players new to OGS aren’t new to go at all.

Sounds exactly like 14k.

Looking at that top group it’s doubtful.

3 Likes

I consider this relevant because

To me, this is relevant because most new OGS players have highly inflated ratings that affect the experience for them and others:

  • The 12 kyu opponent who wrongly thought I am 6 kyu wanted to cancel our game.
  • A game between a new TPK player and a “real” 6 kyu would be a poor experience for both players.
  • When you create a custom game, you can only restrict rank to 9 below or above you. A new TPK player cannot use custom games to find TPK opponents.
3 Likes

Just to be clear: at no point in your history you displayed a 6kyu rank. Your opponent who said that was just guessing on their own. (EDIT: uh, maybe? see reply below)

(To the OGS community:) Can we stop with the misinformation, please? :laughing:

LATER EDIT: uh, this is embarrassing :sweat_smile:

1 Like

Wait, could there be a bug that causes this to happen? I’m not so sure about what I said anymore.

I know for sure accounts are started out at 11.9 kyu (1150 rating), but maybe in some circumstances their displayed rank can be “6 kyu”?

EDIT: again, I was wrong. 11.9 kyu is the rank displayed in the graph, but there’s a hidden “true” rank at 6 kyu.

I might argue that they did know how to play, given by how they played :slight_smile:

They didn’t really play like a player that didn’t know how to play :slight_smile:

So probably it’s some kind of modesty.

1 Like

I remember I used to do something very similar when I first came to OGS, and I was around 18 kyu at the time, so definitely a possiblity. I also suspect the ranks have further inflated since then, so that 18 kyu is probably today’s 16 or 15 kyu.

1 Like

I wonder. Yesterday I got to play a ranked live custom-challenge game against provisional (only one game on record) 11.0k as ~1.3d. How does that work. That’s more than 9 stones?

There is an incredible and huge confusion here between being a beginner and a 12k.

Offering to play a dozen of humiliating games to a beginner for him to get finally his rating is not the way i think to welcome him and stay around playing go.

7 Likes

Once again, to clear up the perennial confusion on this issue, players start at a pseudo-rank of 6k. It used to be 12k, but it changed in 2021. You can see this in the rank shown on the thumbnail board of any beginner’s first game. The thumbnail display is the only place it shows, so far as I know. The profile and inside the game will show a question mark.

2 Likes

You are right, and it’s so strange. I searched a bit and am now watching a new player play ranked games against 1d players. They lost the first one and the rank went up from 11.8k or whatever it was to 10.8k. Then the second loss brought it up to 10.3k. Now it’s showing 7k in the thumbnail.

Update: three losses against 1d apparently gets you up to 10.0k.

3 Likes

Thanks for pointing out the thumbnail thing, but the rest is wrong, as you can see from @Feijoa’s screenshot and as you could have seen by just looking yourself at any new player’s profile page yourself.

So again, my guess is there’s a bug, potentially a remnant from either an older rating system or a remnant of some experiment run by the devs. The rank shown in those thumbnails does not correspond to the actual rank that provisional players have.

Edit: by the way, I’m seeing more and more indications that while the rating system (usually) works decently enough, there are clearly a lot of bugs festering it, and even how it communicates with the GUI, apparently.

EDIT 2: I was wrong, the true provisional rank is 6 kyu: explanation

It definitely changed to 6k in 2021. It was never 6k before that. There was a lot of discussion about it at the time, and it prevented beginners from playing anyone weaker than 15k, which was a big problem. If it is now 12k again, then it was changed again after that, which suggests that the 6k in thumbnail display is an uncorrected remnant from the 2021 standard.

1 Like

That would make sense. Do you have any sources supporting that it was intentionally changed to 6 kyu in 2021? I’m too new to know, but I haven’t seen that anywhere.

There were extensive discussions about it at the time, and it was concluded that the old “provisional” ranking system was no longer working. You will have to do you own research.

I just found a beginner with no ranked games, and it does indeed show 11.9 in the Ratings chart, and 6k in the thumbnail. However, it remains unclear as to which number is used for auto-matching.

2 Likes

It does not take “dozens of games” to drop from your starting rank to ~30k. It’s more like half a dozen games.

2 Likes

Also, it’s subjective whether the experience is “humiliating”. Losing as a beginner is natural (have you ever heard “lose your first 100 games as fast as you can”?), though I do agree that it would be better if beginners could be educated on that.

Many years ago, the first time I learned Go rules, I went to a Go server, I believe KGS, and I arrogantly expected to be very good and talented from the start. That Go server had an Elo system and started players at 1500, and wasn’t as good as Glicko-2 at ranking new players, so I had to lose several games, and every game I saw my rank slooowly fall down.

This was probably a big reason why I gave up on Go for years. So while I agree that it can be a lesson in humility to lose many games for a beginner, the current OGS system is leaps and bounds better than that, and players rarely have to play more than 5 games to get a good estimate.

2 Likes