Possible improvement to the ranking system

A friend of mine that’s 4d on KGS has tried playing some on this site, but he’s run into issues with the inconsistency of the ranks here. It would probably help if players whose rating the site was pretty unsure of were marked as provisional. As it stand, the question mark next to a player’s rating seems to go away pretty quickly, even in cases where the player’s rating might still be wildly inaccurate.

If we used a glicko-esque system [1], we could show a user’s rating as provisional up until the point their Rating Deviation (RD) went below some threshold. If a player claims to be 2d, but is actually 25k, their RD will still be quite high after they’ve played and lost several games (and thus, their rank would still show up as provisional). In this circumstance, they also won’t affect other players’ ranks much until their RD settles. If people wildly over- or under- estimate their rank initially, they won’t tend to pull other players’ ranks around as much. Additionally, their ratings will show up as provisional for longer. As I understand it, if someone self-assigns an accurate rank at the start, and plays at that level, their RD will drop quicker. (If we wanted, we could even vary the number of question marks after a player’s rank with the amount of uncertainty about their rating. e.g., 3d??? --> 1k?? -> 8k? -> 9k.)

I’d like this feature, because while you’re learning, it’s nice to know if you’re improving or not. If you’re constantly being beaten (or beating) players with inaccurate ranks, it obscures your sense of progress. And knowledge of your progress (or lack thereof) is important – if you’re not improving, perhaps it’s time to have a stronger player check your play, etc.

The behavior of ratings systems is complicated, so please take my analysis about the consequences of using glicko with several grains of salt.

[1] http://www.freechess.org/Help/HelpFiles/glicko.html

1 Like

Afaik switching to one of the Glicko Systems* has already been considered by the devs here on OGS and they had reasons to decide against using it. I even recall a post where the developers asked to stop suggesting moving to another rating system. (I guess as it must have been neverending topic back then)

*(I never get why Glicko 1 is so popular despite Glicko 2 being able to predict rating volatility in addition to the rating deviation)

1 Like

I searched the forums but didn’t see much discussion of moving away from Elo. If the devs have already considered this and thrown it out, then nevermind!

1 Like

Probably in the Old Forums. Madita is right, though. The discussions about different rating systems have been endless, and we decided to stick to ELO and modify it more and more, as the site develops, and as we get more data. It’s a constant process. :blush: (If you remember our rating system from late 2013, you know what I mean)

We made this correction in Jun of this year and it has worked extremely well

I’d be interested to know what issues your friend experienced and who his opponents were as dan ranks are more stable overall.

High-ranking KGS players can be a bunch of cry babies here if they are emotionally attached to their rank and the work they put in to get it.

You can’t depend on inflation here and deflation is not much of a problem anymore.