Percentile Drift

As years goes on, rating needed to be Xth percentile getting lower. Probably we’re getting more new weak players dragging requirements down, after all it’s easy to be stronger than 50% if 50% are TPKs. Less generous interpretation is that rating itself changes - strength vs rating equivalence is changing.

5 Likes

just from eyeballing it, the biggest percentile changes seem to roughly correspond with the larger player influxes… so I favour theory A

1 Like

In the last few years it seems to stabilise.
Probably too early to tell if this is the start of an upward development.
Or maybe we hit the absolute minimum.

1 Like

Nice topic :+1: Thanks :grin:

There are two factors at play here:

  1. Distribution of absolute true strengths among the OGS population

  2. Relationship between absolute true strength and OGS rating

Both of these can potentially change in time. We can be effectively certain that item 1 changes in time. The question is whether item 2 changes in time… :thinking:

A few points about that:

  • These factors are too intertwined in the above data to make any firm conclusions without further investigation.

  • The rating system has changed multiple times in the period shown, so comparisons over time are not really fair comparisons.

  • Does it matter? :face_with_raised_eyebrow: Rank / rating is just a tool to measure relative strength so that we can find fair match-ups / decide on appropriate handicap. So, even if the ratings do drift downward over time, as long as: (a) players of similar absolute true strength today have a similar rating to each other today; and (b) players whose relative true strengths today differ by X stones are separated by X ranks today (so that auto-handicap is appropriate), then the rating system still serves its purpose. Indeed, the rating system by design basically does this automatically (e.g. via ranked handicap games), so those conditions will always be met (within the inherent uncertainty of ratings, etc.).

  • Some would argue that the rating system is also a tool to measure an individual’s progress over time. Even if we take that to be true and ignore all the other points above, the drift in rating is relatively slow. For example, I have progressed from 30 kyu in 2017 to 6 kyu in 2021. Decent progress but not super fast. I know many who have progressed at a similar rate or even much faster. 25 kyu today is 600 rating points, so less than 10th percentile. Following the 10th percentile line back to 2017, let’s say 25 kyu was about 1000 rating points back then. 6 kyu today is 1500 rating points, so I have gained about 500 absolute rating points. Even if we just look at today’s percentiles, 1000 rating points is 20th percentile and 1500 is about 65th percentile. That is sill a gain of 45 percentiles, even without accounting for the drift. My point is, the drift is so slow, that rating can still show progress without accounting for drift in all but the most marginal of improvers. And if progress was that marginal, you are approximately plateaued so you’d probably say you were plateaued anyway.

3 Likes

Two questions:

  1. Which players are considered / counted? 40000 is way lower than the total player count on OGS.
  2. What is the data source?
4 Likes

Rating Graphs in profiles should be modified. Currently its impossible to compare rating now and rating some years ago by it. So they are useless for players who are for a long time on OGS.
It only creates illusion that you become to play worse while your real strength keeps same, or that you improve very slow. Its discouraging.

5 Likes

Did you exclude inactive and ‘?’ accounts?

2 Likes
  • Deviation < 100
  • Downloaded termination-api rating graph pages
1 Like