Use AI scoring to eliminate score cheating

Feedback Suggestions
32

Summary

This should be the process: Explain for beginners — Autoscore — Correct the markings blindly — Resolve any disagreements — Resume play of necessary

Context

This type of cheating also came up in https://www.reddit.com/r/baduk/comments/18ru5ae/this_ai_scoring_is_bs/ , where, sadly, the OP wrote “My opponent (13k) marked my upper left territory as dead in attempt to score a win off me. But I wouldn’t expect the go community to have people like that”. This was, however, in the Sente app, and there may have been a race condition leading the victim to accept the position just when the malefactor had changed the markings. There, assuming the app receives the updated markings, I first suggested that the app could either:

  • notify the user of the change and repeat the question, or
  • change the marking back to what the user accepted and await a reaction.

I then went on to consider the rationale for a sensible design, saying:

It seems evident that:

  • The user must know what they are accepting. I.e.:
  • The score markings on which the user bases their decision must be those OGS considers accepted. So:
  • A change by the opponent to the markings shown to the user must be explicitly acknowledged by the user.
    • Otherwise it is impossible to be sure that they have registered the change.
    • Anything else is like letting my opponent change their move while I am making mine!
    • The problem is basically a race condition.

I do not recall gathering whether OGS requires a user to acknowledge a change by the opponent, but I fear not :disappointed: If not, OGS ought to change, which would also benefit browser users. In the meantime, you as app developer could make this problem a little less likely for your users by:

  • Not displaying changes to the markings while the user considers them.
  • If the user accepts the markings, warning them of any change before you forward the acceptance, so they have a chance to correct them and/or reconsider.
  • If the user makes the same change(s) as the opponent before accepting, silently use those changes.

There is still a danger that your Accept crosses a change, but the above makes the time window for that smaller.

An issue that cannot be addressed is resumption of play to resolve disagreements — IIRC it is a limitation of OGS that that is not possible, although it is the ideal solution.

(I gather from the preceding discussion that it is possible.)

You are understandably unhappy with the crude measure of blocking the Accept button, but, given the dodgy scoring behaviour of OGS (and the inherent difficulty of proposing a score for a very wide range of strengths), some inconvenience is inevitable. I feel that my suggestions reduce that to a necessary minimum.

Proposal

What I feel should happen after both pass is:

  • Explain: An explanation of the ensuing procedure is offered to the players.
    • This should be tailored to beginners.
    • For more advanced players it should be available but hidden / tucked away.
  • Autoscore: The position is shown with Autoscore markings (improving that is a separate issue!)
  • Correct: Both players make any changes to the markings they feel necessary, then Accept.
    • These changes are only shown locally until both players Accept.
    • It should be possible to repair any weird Autoscore nonsense by toggling the status of vacant spots between Black/Neutral/White, and that should be propagated to all other connected vacant spots.
  • If both players accept the same markings, the resulting score stands.
  • Resolve: Otherwise each player is notified of the other player’s markings and offered an explanation of the next step.
    • It should be easy to compare one’s own markings with one’s opponent’s.
      • This could be done by letting one go back and forth between the two sets of markings, with the differing spots highlighted, perhaps surrounded by red lines.
      • It could conceivably also be shown statically on a single screen, but it might be hard to design a clear presentation of that.
  • The players choose among:
    • Accept opponent’s markings
    • Stand by own markings Edit: optionally amended
    • Resume play
  • Again, if both players now accept the same markings, the resulting score stands.
  • Resume: Otherwise each player is told what the other chose and is offered an explanation of what happens next, and play resumes.
  • If both immediately pass again, use the Autoscore (however absurd!)
  • Otherwise (at least one player moves), continue until both pass again, then repeat the above.

I have not addressed timeouts for the players’ choices, network / escaping / stalling issues, time allowance for resumption of play, whether/how to implement the Japanese rule that the score be based on the position before resumption of play — I presume that a lot of that is already in place.

4 Likes