Exploit in Scoring Mode timer and how to solve it

OGS Development
1

From unpleasant experiences with cheaters who refused to accept the score in both the old and revamped OGS Scoring Mode, I have some points I would like to make and suggestions on how to fix it.

In the old system when people refused to accept score or only accept their intentionally messsed up scoring (e.g. marking live groups as dead), the only way to solve it afaik was to call a mod AND leave the game open. The moment you closed the game first, it would indicate to the system that u left it which would trigger a loss for you.

In the current system, there is a Scoring Mode Timer to prevent non-results from unwilling participants, but this forcing of result can also resort to exploitation by forcing your opponent to constantly edit the scoring to keep the timer moving to prevent the opponent from getting his desired erroneous scoring. So even if you call a mod, you no longer can simply leave the window open till a mod arrives (usually an hour or more), you have to actually play 15/20 second (forgot the exact duration) Scoring Mode Byoyomi with your opponent for an unreasonable amount of time till your mod arrives.

When a mod does arrive eventually, from past experiences in both pre-revamp and current systems, the mod cannot (or claims to) give you the win but at the most annul the score. What this means is that, when a player does not want to lose, he can simply use such cheap tactics to at the very least “draw” and not affect his precious rating while the other deserving player, who devoted at least half an hour or more on average to play the game, is cheated out of a win.

For those lazy to read, in point form on scoring timer problems:

Old System - Have to not leave game before opponent leaves, while waiting for mod to prevent auto-loss. At which point the best you can get is an annulment from mod.

New System - Have to play sudden-death “byoyomi” Scoring Mode Timer game with the opponent till mod comes. Can no longer leave window open and do other things. Annulment is still the best you can hope to get.

Suggestion: Lock the screen with a BIG pop-up stating that the game has been locked till moderator review. I know this could potentially mean a huge backlog of reports to clear, but I don’t see any other way to remove this exploit. A warning that wrong reportings that waste the mods’ time would result in bannings, should cut down the downside to this though.

How to verify claims: Simple, just give the win to the player that actually has the bigger territory. If the one reporting is the one with the bigger territory, no issue. If the one reporting is the one with the smaller territory, ban him, for inconveniencing everyone with false claims.

9 Likes
2

hmm, to me the best solution would be if mods could actually overturn the game score. (as I thought they could and it just seems logical)

There has been a remove stones timeout for as long as I remember, but never even close to 20s (even in blitz). Is that really so? That would really be too much and should be talked about if that was true.

But whatever the case is the single victory really worth so much to people to play a clicking contest with a cheater for an hour? That’s crazy :smiley: Just report him/her and spend the saved time playing someone nicer :slight_smile:

5 Likes
3

I think I have better solution:

If players disagree on scoring for 3rd time freeze the game, make both players do scoring as they think it should be scored and wait until it is judged.

Judging can be done by users too (limit to dans & SDKs?) - once three judges casts their votes (agree with player A / agree with player B / disagree with both) three possible scenarios may happen:

  1. all three judges unanimously agreed with one player - in this case game is resolved with that players suggested scoring.
  2. at least one judge disagrees with both - in this case he/she should state why and game will be continue.
  3. all jusdges agreed with some player, but were not unanimous - in this case it should be passed to moderator.

I believe this would solve the problem, reduce moderators workload and also discourage users from trying to intentionaly mark groups incorrectly in scoring.

5 Likes
4

I think there are a few issues with that suggestions, the system might introduce more abuse/issues:

  1. People who intentionally misjudge the score. (or something worse, like intentionally misjudging the score with two of his friends). There are incentives to do this, people who dislike this system/who’ve been victim of scoring issue might protest like this. Also, It takes time to judge a score correctly but it doesn’t take much time to just randomly assign a score so the system is vulnerable to this type of attack.

  2. I am not sure if SDK is strong/careful enough to judge the score but there’s just not enough dan players here. I’m 4k but i rely on score estimate heavily(which is sometimes not accurate) and I get things completely wrong. I still totally misjudge the life and death of my group occasionally (not necessarily reading issue, just I’m sometimes oblivious to the death of my group). I’ve personally scored my own game incorrectly by misjudgment at least twice in my sdk era and my sdk opponent agreed on the wrong score (We noticed that issue in review 10 min+ after the game ended).

  3. (and most importantly) I don’t have a clear vision on how the system should be implemented. We can’t assign the judging duty randomly to players cuz who knows if they’re willing to do it/if they’re online. But if we just rely on volunteers, we need to prevent the situation where If a use call for moderator, his opponent/the player himself won’t just come into the game and score the game using his 3 alternate accounts.

Anyway, I think the locking idea sounds more feasible.

6 Likes
5

ad 1) yes, there might be such cases, but its much more managable - they can be banned from judging or there can be additional criteria like 50+ games and month+ old account if needed. Also mod can always adjust result if needed.

ad 2) afaik ~90% of cases are trivial things where someone marks all opponents stones as dead when he lose etc. Majority of rest will still be mostly trivial L&D. Judges would just verify L&D in final position, no score estimations needed. For quite rare cases when you are not sure you can and should simply choose to not vote.

ad 3) all dans and SDKs should be eligible (with adjustment as in response to first part) I would love some icon in top bar (near friends online icon) if judging is needed somewhere. First three votes counts, everything voluntary, noone forced to do anything.

2 Likes
6

In my experience with other “on-line” games (including games played through, rather than over, the internet) I have found that getting fair judgement from the players can be problematical. In my culture, if a friend is in the wrong you do not support that wrong, even though he is your friend., and this does not jeopardize the friendship. However, in some other cultures I have encountered, the bond of friendship is different and friends will never fail to support each other, wrong or right, and failure to support is viewed as betrayal. Since OGS is a truly multi-cultural experience this attitude will be found. Having other players weigh in risks having friends weigh in with the sole purpose of supporting their friend. Moderators, however, are honor-bound to be neutral (which in my experience ours are).

The locking idea would avoid risking the “wisdom of the crowd,” and if the lock could stay on comfortably until it can be resolved with without messing with the site or future games by the complaintant, it should work better than the other alternatives currently available.

If registering with the site requires real names & addresses (hidden from other players) then a player ban could not be as easily circumvented by just changing the handle and e-mail account.

Nothing is 100% fool-proof, so whatever is done should be approached as “the highest odds for accomplishing the greatest good.”

Mogadeet

4 Likes
7

We could just go the Wbaduk route and auto score all our games, that way no one can abuse the system. Then we would need a really good scoring system though.

We already have this in place when playing bots, sometimes the score is marked incorrectly though. So maybe have a way to dispute the score if marked incorrectly(which is pretty much just telling a mod.) but make it a button like the call mod option.

4 Likes
8

Right even for locking, just like the reporting of any other issues, requires some degree of trust by the mod…

If for e.g. the winning player, who typically has no motivation to cheat since he is winning, decides to report his game as attempted exploit of the scoring system, his word will be taken for it especially since logically he shouldn’t have the motivation to cheat. But if, the reporting player who was winning, had some grudge or just sick hobby to sabotage the other party, then the flaws of locking (or at least the enforcement of proper reporting) is apparent because then you don’t know if the losing player really did as the reporting player claimed since the mods can’t see edits in the scoring phase so immediate bans might not be feasible to avoid banning innocent players, however unlikely their innocence. Though this kind of treacherous behaviour should be in the 1% of reports.

1 Like
9

Agree with @spatula that users might simply call their friends, Go community group or alternate accounts to verify the score. Wouldn’t be surprised if some twisted individuals banded together to form a group solely to exploit this suggested voting system.
Even if everyone was fair and just, how would you get people to do it willingly? A pop-up at the bottom of their screen for a minute to random users alerting them to possibly help? Force users to help score or be unable to proceed with the rest of the site? Unfortunately anything that is not forceful might result in large delays since people might be unwilling to put in the effort to help. Too forceful however and you have players leaving the site for being too troublesome.
Ultimately I do think @Adam3141 's suggestion is the most practical and simplest of them all. To overturn the loss to a win. But I am unsure if the mods are unwilling, or only the devs gave themselves the right to over turn the score other than annulment, or this is a technical impossibility in the current system.

2 Likes
10

If we take into concideration 9k+, 50+ games and month+ old account it would take tens of hours of preparation for individual to pull it ONCE month later. I highly doubt anyone would put that amount of time into it. Same with calling friends or forming some kind of twisted band - they can do it ONCE. It wouldnt be so hard for mod to find intent here and ban them from judging ever again…

I would be against any mandatory participation, but I believe a lot of users would be willing to help when between games or so. With ~50 users involved it could reduce response time from hours to few minutes, which is huge improvement. Practicaly everyone hates when opponent start marking all your stones as dead to make you resign and I believe a lot of people would be willing to help victims of such behaviour, especially when it would take almost no time and effort to do so. I dont think getting 50 people involved is unrealistic.

2 Likes
11

While I kind of like the idea to me it still sounds way too overly complicated and uncertain… And does not really solve the original issue of people wanting to cheat. There are tons of other ways of cheating or being annoying and indeed while maybe not practical it would still provide some other ways to be annyoing.

If there was confidence that mods could just overturn the score afterward, people would just close the game, do something productive with their time and know that it will be fixed. It does not matter that it will be in one hour. Not humouring such players at all is the best we can do IMHO.

I find it really sad if mods do not have such powers :confused: and would love confirmation/correction from someone who knows for sure.

8 Likes
12

I think we should just send any disputed games to the mods. It shouldn’t be too big a burden, because whenever this sort of exploitation takes place, they get called anyway.

4 Likes
13

Too bad there isn’t a beer/wine fund to help reward the Moderators for all their hard work.

2 Likes
14

On balance, I think I like best the idea of empowering the mods to adjust the outcome appropriately. And on a personal level, one can block the offending player, as you have long advocated. However the latter is closing the gate after the cows have wandered off. I mentioned in a another thread that I have long considered starting a thread called “Troll Watch,” where people could report trolls and cheats. This would make it possible for players to avoid, in the first place, verbally abusive trolls and scoring cheats. Of course, this in turn might be abused and would doubtless be controversial even in principle. Hence, I have never done it, but I’m putting it on the table if anyone wants to discuss it.

3 Likes
15

This is what devs want to get to asap. I suggested the idea of locking games and let mods count games, but they don’t want to assign these kind of tasks to moderators (there will be lots and lots of reports to attend to).

3 Likes
16

Could you maybe have a combination of both? Players get 2 chances to successfully negotiate the score, and if they can’t agree, it defaults to the best available auto score?

2 Likes
17

I can’t speak officially on the matter, as I am neither admin nor mod, but I suspect that such a thread would be shut down almost immediately. We don’t need witch hunts or blacklists.

2 Likes
18

I advocate it but I see little point. Most of the frequent cheaters would have been reported enough to be banned pretty quickly so their accounts will likely be invalid soon. Those that get banned quickly probably also have less to lose and can create another acc.

The ones that pose bigger probs imo are those that cheat multiple times but infrequently enough (as a way to avoid detection or not) as a percentage of their total games so that the mods would have to scroll through their hundreds of games to find a few and wouldn’t be able to find a pattern at a quick glance to confidently judge that the person is a confirmed cheater and deserves immediate ban. Those users would likely receive a warning from time to time and be allowed to continue their cheating activities from time to time. These users are the ones that would feel more pain from losing their account.

2 Likes
19

Bleh. Mods not detecting patterns? Which mods do you think you are refering to? :smile:

3 Likes
20

Troll watch sounds like a good idea but people will get banned someone will have to maintain the list people change their names make new accounts etc etc.

I think the biggest thing of all though would be getting people to actually read it xD.

1 Like