The OGS ranking system is very strange, should maybe be re-thought (discussion)

OGS Development
1

I like OGS and I am a site supporter but the following few things kind of irk me and I wonder why they are the way they are:

  1. Different board sizes should not blend into an overall average rank. Earlier when I started playing Go, I played on 9x9 and 13x13 which influences my 19x19 rank. Ranked games should just be 19x19 otherwise you can get matched with someone in a 19x19 game that is super over or under ranked from using the smaller boards which leads to odd pairings for ranked games.

  2. Different time settings should not blend into an average rank. Blitz should have it’s own ranking system, live should have it’s own ranking system and correspondence should have it’s own ranking system. I play correspondence on the site ladder and this rank is higher than my live ranking as I have more time to read…which leads to harder live games because my rank is reported higher on live than it should be because the blended rank is used to pair up games.

  3. Ranked games should have set settings. It’s too easy to game the ranking system with any weird time settings you want or weird handicap. Pandanet does a great job at this, all ranked games are under certain time settings and certain pre-determined handicaps.

  4. You should not be able to play bots in ranked games. Period. It’s way too easy to game the system playing a bot.

  5. New players that are unranked should either get to declare a rank and then play games to reach a new calibration or they should just only be allowed to play bots until a rank is determined for them. Bot play would only be for unranked games and ranking “?” players. Way too many times are new players with a “?” getting treated as 12kyus when they are 20kyus. This discourages a ton of beginners that could be great future go players.

I have no idea why I am ranked 7kyu when all my individual ranks in the breakdown are 8.1 - 8.7kyu. The blended rank isn’t a weighted average so what is it doing? How is my overall rank higher than all the breakdowns??

For a ranking system to be solid it has to be consistent and I think that is what is lacking right now.

5 Likes
2

Great topic :smiley: Lots of interesting points to discuss so I look forward to hearing what people have to say :grin:

I agree with some of what you say in principle but there are practical reasons why the alternative wouldn’t work.

For example:

I have the same problem, so I agree in principle, but…

The problem is that time per move is a continuous variable. 10 seconds per move has less reading time than 1 minuter per move, which has less than 1 hour per move, which has less than 1 day per move, etc. Where do we draw the line between “hard” games that have less time than “easy” games that are longer. Is a 9 seconds per move blitz game really so different from a 10 seconds per move live game? Obviously not. So practically, I can’t see how we could reliably use the ranks split by time.

However, I do not agree with this:

Go skill on one board size translates very well to other board sizes. Sure, there are differences in the balance of influence vs territory, strategy vs tactics, etc but I think these are small compared to the effect of overall player skill so blended rank is probably a good idea across board sizes.

Just to clarify Re these points:

Yes, but only a little bit because:

Actually, I think it is an average weighted by the number of games. So essentially it’s just an average of all games. Can someone else confirm please?

Also, the rank breakdown by size / time is only meaningful if you play lots of games of all settings regularly. For example, I haven’t played much 9x9 on OGS recently but I have played 9x9 elsewhere and improved so my OGS 9x9 rank is out of date. But the improvement shows in my 19x19 rank as that is mainly what I play on OGS. In that case, it would be really bad to use my 9x9 rank rather than the overall.

And just for completeness to respond to the other points:

Disagree. I think we should go complete opposite and have any game, any size, any komi, and handicap, any time, etc can be rated. DGS does it and it works great. :sunglasses:

I don’t understand. How can someone game the system with settings? :confused:

I don’t know enough about this to have a strong opinion. As I understand it, there are pros and cons, like calibrating rank of new accounts (or established accounts for that matter) against a fixed strength opponent, vs rank manipulation respectively. I think @Conrad_Melville and @BHydden will probably have something more useful to day about this.

This keeps coming up and has been discussed in numerous topics. But I agree, declared rank is nice. Question for the wider community: why can’t we replace [?] with [20k?]?

This seems dodgy. Surely the overall should be 9k.

Anyway, link I say, great topic. Thanks for posting :beer: I’m looking forward to the other replies :smiley_cat:

2 Likes
3

Without really addressing any of your points, posts like this, asking for a better ranking system, have become very rare since our last rank update, while they used to be frequent enough that we would get about one new topic each month.

I therefore feel like we’re currently at the best place so far with our ranking system.

10 Likes
4

The individual ranks are separate rank pools, and cannot be directly compared with the overall rank, or with each other.

6 Likes
5

Sure, but that doesn’t mean there can’t be further improvements.

1 Like
7

Points 1 & 2: It is easy enough to create separate dedicated accounts for each of these various settings. You would need 9 accounts for combinations of the three board sizes and three speeds.

Point 3: Thankfully, for my point of view, OGS believes in freedom, rather than dictating the specifications of a game. IIRC, non-standard komi and I believe nonstandard handicaps automatically invalidate the ranked status of the game. In any case, komi cheats (e.g., 650) and handicap cheats (9 stones in 9x9) have been warned and banned in the past. People who use ridiculous time settings (e.g., 1 or 2 sec byo yomi) have also been banned, although there is less agreement on this issue among the mods I worked with. There is also a warning attached to some of these weird specifications, but I suspect that most players don’t pay attention to it.

Point 4: Bots are very useful for ranking up a new account without sandbagging the real beginners. I like your later carve-out for ranked bot games only against “?” players. Prohibiting ranked bot games by ranked players would prevent some major cheating: (1) sandbaggers who deliberately lose to bots in order to rank down, (2) children and child-like adults who beat the bots repeatedly by using an exploit in order to rank up, and (3) the same child-adults who abuse the bots by playing one against another.

Point 5: There is abundant discussion about the current ranking system already in the Forums. Self-declaration of rank was used at one time here, but was abandoned because too many players requested or needed rank adjustment. I like the Glicko system because it is rapidly responsive. However, it has recently been discovered that the “humble rank” matchmaking system is apparently broken. But that can and should be fixed separately.

8 Likes
8

Your overall rating takes all your games into account, therefore it can adjust faster (faster since more games per time). Your breakdown ratings are just slower to adjust to you becoming better.

This is basically the reason 1) and 2) are not in effect.

We changed the breakdown ratings to compute against you opponent’s overall rating, so the ratings should be on comparable scales. This change is in effect as longaas you can toggle between ratings and rank on the breakdown chart.

As mentioned above the breakdowns have the tendency to lag behind and never catch up.

4 Likes
9

Looking at chess’s blitz settings, I’m not so sure there is a lower bound to reasonable time per move.
There’s also always another player which has to accept the settings, and which you have to beat as well (e.g. being better)

but …

4 Likes
10

btw. do you have any bad experience with the ranks (matchmaking) feeling wrong, or is it just a general discomfort?

2 Likes
11

I personally feel like ogs’s biggest selling point is the vide variety of different board sizes, timesettings, rulesets, and other settings which players can choose from. And i feel like the biggest flaw of pandanet are the super heavy restrictions on what kinda settings can players play ranked games. So for 1-3 i am totally against for, if anything i would love to have even more settings to choose from, for example i would love to play ranked games with reverse komi instead traditional handicaps.

For 4 i dont really know… I dislike playing with bots regardless if its ranked or not, but i can see many beginners playing dozens of games with those in order to get stronger and gain confidence before they start playing with human opponents. I dont know if ‘gaming the system’ is really a problem in any other way than purposefully losing in order to sandbag later.

For 5 i totally agree, i hope we could have a sytem where real beginners could immediately be ranked around 30k. The current system which puts all new users on the median rank means that a) beginners will get absolutely crushed by near-sdk level opponents, and b) players around 15-8k will often be paired with opponents whom dont really know the rules yet nor recognise when the game is hopelessly lost and resignation is the only correct/polite move.

3 Likes
12

A little comment on this. Without getting into the math behind the ranking system, ‘?’ really means that the rank is so uncertain that it won’t affect the opponent’s rank much.

So, on one hand, they’re in practice not treated as a 12k for example, because, although the number is inside that range, this uncertainty tells the system that probably this number doesn’t make sense, so it’s just there as an initial condition from which to converge to a more significant value.

On the other hand, showing 20k? doesn’t make much sense because is sort of false.

3 Likes
13

Mostly general, but when I was around 10 - 14k I would always get matched with ? and trounce them or they would trounce me.

It doesn’t seem like you have to “prove” a rank as much as other servers here, it’s a bit volatile in the current system.

1 Like
14

This isn’t a contradiction. You only can do it for new accounts but ranked players that aren’t ? only can play ranked players in ranked games. I thought that was clear.

1 Like
15

It wasn’t at all, but it is more so now with your clarification.

16

This user plays almost random moves but is 4k because of opponents who resign out of boredom.

1 Like
17

But this player has only 6 ranked games… only one game after getting a somewhat “stable” rank.

The account won’t probably stay there forever.

2 Likes
18

This is less an issue with the ranking system and more an issue with people not taking ranked games seriously.

Looks like they’ve fallen back down to an uncertain 10k already :man_shrugging: how mysterious :wink:

4 Likes
19

Several games had less than 10 moves. Maybe it would help if games with less than 10 moves were automatically annulled?

2 Likes
20

It used to be n moves where n was the side length of the board (9, 13, 19) but that was too high and it was petitioned to be lower… it’s now 2 so that each player has a chance to cancel without fault if the other user doesn’t move.

Once the game has started, resigning because you don’t like how the other user plays isn’t really in the spirit of the auto-annul but we do review and annul such games when they’re reported (such as with this user)

4 Likes
21

They don’t need to take it seriously.

It’s math, the more they play the more accurate the ranking will be to how good they are when they play (seriously or not).

If they don’t play…. their rank is irrelevant.