Quantifying OGS's rank deflation over time

General Go Discussion
1

OGS rank deflation is a topic near and dear to my heart ;). So I spent a week and $20 in GPUs to to look at the data.

Here’s the short summary:

Katago trained a human policy network conditioned on various server ranks and other metadata. I used that to create a rank guesser (KGS scale, medium time settings, try it here: https://rankmle.david.ma/). Then I applied this rank guesser to a sample of games on OGS.

Before the 2021 adjustment, OGS ranks below about 5 kyu were very tough. Many double-digit kyu OGS players were estimated by HumanSL as much stronger on the KGS-style axis.

Pre-2021 OGS rank histogram
Pre-2021 OGS rank histogram1769×758 103 KB

From 2021 through 2024, the system looks much closer to the HumanSL KGS-style scale. The lower kyu ranks move toward the diagonal, and the median estimates are less severely compressed.

2021-2024 OGS rank histogram
2021-2024 OGS rank histogram1773×764 97.4 KB

In the 2024-2025 slice, the lower ranks appear to be drifting harder again. The effect is not as extreme as the pre-2021 data, and the sample is shorter, but the direction is visible.

Post-2024 OGS rank histogram
Post-2024 OGS rank histogram1777×761 99.5 KB

The yearly medians tell the same story more compactly: a sharp correction around the 2021 adjustment, then a later bend suggesting renewed rank deflation, especially below single-digit kyu.

Yearly median predicted KGS rank by OGS rank
Yearly median predicted KGS rank by OGS rank1413×728 71.2 KB

More detailed repo with code: GitHub - djma/ogs-rank-analysis: Analyzing the strength profiles of OGS players over time · GitHub

17 Likes
2

I am curious how effective this method will be against “styles” change, what would happened if we match these with games prior to 2017? And the distribution of plays, the effect in 2024-2025 at the lower ranks seemed to be more about the spread and distribution of different kinds of playing styles/strength than the actual change. Can we ruled out the hypothesis that it has to do with players’ declared ranks mismatch (like how many of these “spread” are from players with established long term ranks, and how many are from new accounts? or rarely active account?)

1 Like
3

That’s some pretty cool work. :clap:

1 Like
4

noice

5

Distributed the link to the ranking tool in several discords.
Any plans to develop it a little further? The sgf review is a nice touch.

Have you compared the it to existing tools like @Animiral ‘s https://howdeepisyourgo.org/