Sandbaggers completing breaking the ranking system and going unpunished

I know a guy that was previously reported but the result was a warning and ONE of his thrown games was reverted.

In fact, he has a long history of losing 9x9 games after just a few moves and winning 19x19 games against lower ranked players, such that his 19x19 rating is 2kyu and his 9x9 rating is 25kyu. People are getting matched against his 8-10kyu average and that is affecting their ratings.

I looked through his game history and picked a random opponent: XXXX

This 11kyu’s ratting went from 1215 to 1189 from one loss against a 2kyu! That is not how the system should be working. The impact of that loss should have been negligible.

I don’t agree with playgo.gg’s “Fox-level” ratings, but OGS is very discouraging for people right now and a lot will (and probably did already) play elsewhere instead or just give up.

Hello,

sorry, I have redacted the nicknames from your post, no naming and shaming please, but we can certainly keep the discussion in general terms.

That particular user has been suspended for days now, and most of the recent problematic games annulled as far as I can see.

We are not always super fast, that is unfortunately true, but reporting is the best way to let us know about these cases. I am not sure if there is some foolproof way to automate the process (happy to hear some ideas!), but at the moment this is the best we’ve got :slight_smile:

3 Likes

I don’t think it’s true, I tested on the Go Rating Calculator which says this

If the 11k loses against the 2k, his rating goes from 1215 to 1214.11 (you can change deviation and volatility parameters, the rating drop remains negligible in any case).

Oh you were referring to the partial rank 2k and not the overall rank 10k.

Sure, that’s how sandbagging works. Moderators do their best to annul sandbagged games and suspend sandbaggers if necessary, but that would be too much work to look at every single game in the user’s history.

1 Like

they mean

overall rank that is used in calculation is lower

so
“This 11kyu’s ratting went from 1215 to 1189 from one loss against” someone who is 2k on 19x19

2 Likes

Moderators do their best to annul sandbagged games and suspend sandbaggers if necessary, but that would be too much work to look at every single game in the user’s history.

100s of 9x9 opponents who beat them, and 19x19 opponents who lost to them still have those games on their records, so I don’t think it’s fair to say that these games are annulled.

But really this example tells me that the ranking system is just broken and urgently needs to be fixed before the OGS decline becomes too much to recover from.

Perhaps no one wanted to look at 100s of games (too much work as I said, sometimes many games are annulled but that’s not systematically the case and you can’t ask a small team of volunteers to look at every single game).

Another technical limitation is that only the most recent 100 games can be annulled.

1 Like

While I proudly maintaint that OGS is mostly a positive environment (all the more for a free public server), I would be the first to admit that we do not have the perfect tools for dealing with some of the more unwanted behaviours of some users.

I also understand that it is frustrating, and can feel unfair. For me (given my current role here) it is probably at least twice as frustrating, so I sympathise. And I (and surely all moderators, and community moderators) would be grateful for any improvements. Hell I would be the first-tenth to benefit from those :grin:

But what exactly are you suggesting? That we should annull all games of any suspected sandbagger? For the .1 change in rank? I am not at all convinced that would be an improvement to the general users’ experience. And that’s disregarding all the intricacies in most of those “cases”.

7 Likes

Jev, is this dude sandbagging?

1 Like

I’m sure you’re right, but this kind of behaviour also transitively affects a much larger set of players to some extent. So it’s not just one player’s rating reduced by 20, it’s actually 100s of players who have been opponents of this sandbagger and then 1000s of their future opponents who are reduced by 2. And if there are lots of people doing this then it all adds up.

Are there though?

Where’s the evidence for that? You’ve found one. How many would you expect to be able to find in a population our size?

While we’re on evidence…

what’s this?

Just my two cents. I believe some people are sandbaggers without wanting to be (Me?). Because the ranking is calculated from all times and board sizes. If one person is good at 13x13 (lets say 12k) does not mean they are good at 19x19 (lets say 17k). Or maybe they haven’t played some board in a while or they are only used to some time settings.

Consequence is that a well intentioned player can have an accurate ranking (lets say 14k) and every analysis will say the ranking is the best available estimate. But from the player perspective on a day to day basis will be on average overmatched for 19x19 and undermatched for 13x13. Both parties on these matches will start to notice this. And no one is really trying to sandbag.

For this not to happen I think the entire userbase should be regularly playing equal amounts of all board sizes and times and their rank should also be identical and identically update for all conditions of play. I dont have access to the real data and maybe this is not correct.

Things that may help:

  1. Per play condition ranks used for automatch. But I see this has been discussed before and put to rest.
  2. weighted global rank corrected by time invested on that match. (So blitz and smaller board sizes are scaled)
  3. A mechanism to make a rank again ?preliminary or otherwise reset the algorithm when that player has not played in a while or when player is going up or down faster than anticipated. then the opponent knows what to expect of the match when seeing a “?”.
1 Like

The evidence is the rank deflation on OGS compared to other sites and regional go federations. This may not be the only cause but I think you’d have to come up with other causes to say that this particular problem isn’t a significant one.

It feels like the number of players on the platform has reduced over recent years, particularly at the mid-high dan level.

1 Like

The main cause identified so far, with a change recently made, is bot-games and the way they were ranked.

FWIW it is not true to say “you have to come up with another explanation in order to say this particular one isn’t significant”.

I think you have to explain the mechanism by which sandbagging would even cause rank pool deflation before you can claim it to be a cause, let alone a significant one.

Note that for each person for whom their rank is wrongly tweaked by playing a sandbagger, the sandbagger themselves goes in the other direction. So on the face of it this would be nil effect on the rank pool.

1 Like

Except that, as per my original point, a user can inflate their rank for 9x9 and deflate it for 19x19. The person I was originally complaining about was 2kyu on 19x19 and 25kyu on 9x9. The “average” is 9kyu, which is how they are matched with others.

It doesn’t average out when the rankings are calculated this way.

The idea that sandbaggers (a generalization) are “going unpunished” is simply untrue. Rather than fuzzy assertions, I have receipts. Nine or ten months ago, the Thrown Game report category (an internal sub-category of Sandbagging) was assigned to community moderators (CMs). Since then, I have voted on 2,426 Thrown Game reports, with a 99.7% consensus rate for my votes (three votes in agreement are a consensus). All the valid reports (most of the total) resulted in the issuance of a warning, final warning, or suspension, as well as annulment of the game. Other CMs have similar records in this category, since three votes are required for a resolution, and none of us is active for 24 hours a day.

I always look at a confirmed sandbagger’s record for other thrown games, and file reports on those found, as do at least two other CMs to my knowledge. Just today, 15 of my recent Thrown Game reports were resolved with annulment and issuance of a warning. Also, sandbagged games (those won by sandbaggers with false ranks) are indeed annulled by moderators when practicable. In one case I know, a moderator annulled 27 sandbagged games in one fell swoop.

4 Likes

As a side note, I’ve gone my entire life thinking it was one foul swoop. “Maybe Conrad’s using voice to text” I think. Well that may be true, but nonetheless it’s one fell swoop.

2 Likes

Now I’m thinking the bird isn’t using its talons but cloaca for the attack.

1 Like

What if a sandbagger repeatedly abandons accounts and creates new ones? How does that affect the rating of active players?

Good point, but even without that complexity Eugene’s quoted post is false in general, because rating changes aren’t zero-sum in glicko2 (unlike simple Elo) due the rating deviation scaling the size of rating change for each player.

3 Likes

Even when it’s zero sum there can be issues.

With Elo, if every account, and every new account starts at the same starting rating R, the average rating of all players is R. Because it’s zero sum the average rating of the pool never changes, and even adding a new player doesn’t change the average rating.

However a new account that joins, gains rating and then abandons the account siphons away rating that ends up locked up in that account, because it’s zero sum. The overall average of the pool doesn’t change, but the average rating of active players can decrease.

There’s a chance to counter the above specific effect in Glicko, with it not being zero sum can potentially help with that. But there can be other effects