A compendium of OGS's terrible scoring system confusing beginners

ah, I see the issue:


By “agreed-dead”, I mean

agreed by the 2 runs of the AI score estimator
(1 with White to move, 1 with Black to move)

, ​ ​ ​ ​ not ​ agreed by the 2 players ​ ​ .

1 Like
3 Likes

All this stuff of 2 runs of KataGo AI auto score is missing the point. If the 2 players: a human beginner and amy-bot(?) think white lower right is alive, then it is alive and territory. KataGo should stfu!

5 Likes

Another case:

By the way, I clicked on the score estimator at move 80, and it said B+73.2

1

Then I clicked on the white stones F7, and the score estimator became B+53.5.

2

Is this a known bug, or is clicking on a group of stones supposed to have a predictable effect?

The score estimator generally gives different answers every time you ask, even when the position doesn’t change. It is deliberately inaccurate. But the autoscore is supposed to be right. The large issue in this thread is that the autoscore does not score things as they stand, as in the examples shown, per the players’ judgment. Even worse, in examples I have posted before, the autoscore occasionally substitutes multi-move tesugis, which is a form of outside help, prohibited by the rules. You must beat the autoscore, as well as your opponent.

3 Likes

Yes I know that, but the score estimator may be another source of confusion. The behavior I described (about 20 points difference after clicking on a stone) is reproducible.

I note that on my first image, white stones have a small black square inside, and in my second image, white stones have a large black square inside. What is the meaning of this?

I don’t know, except to note that White filled in his own eye on that spot at move 80. Who knows why. Beginners do lots of bizarre things.

1 Like

Here is the algorithm as I understand it:

  1. The algorithm guesses at dead stones (that’s why the white stones in the left have larger squares in your first screenshot)
  2. [Monte Carlo simulation] It then looks at ~1000 random playouts and uses that to determine who owns which territory. If stones are marked dead, the simulation treats them as if they are not on the board at all.
  3. the user may manually mark stones dead, and step 2 is repeated.

If the stones die in most random playouts (step 2) the group will appear dead in the final result regardless of whether you marked the group dead or not.

I don’t think this behavior is considered a bug, although I personally would prefer it to be done differently (naive deterministic algorithm based on distance instead of Monte Carlo step). I think @stone.defender prefers a distance-based algorithm, not sure how other folks feel :slight_smile:

I still don’t see how the SE could calculate B+53.5 in my second picture.

Ah sorry I missed that part. Not sure, but @Feijoa might be able to offer insight (he’s done some work in making the scorer respect rulesets)

If you see different sized squares it’s using the strong AI estimator, and in that case I have no idea what the effect is of clicking on groups.

2 Likes

Oh I didn’t realize this is the strong one (I should have guessed from the ai review) plz ignore all my previous comments!

I will note though- there are 21 stones that get “toggled” so this bug is likely the algorithm trying to (or forgetting to) resolve those

A bit that does not implement the dead stone estimation protocol (within a given grace period) may still be worth having but deserves a lower rank even if it succumbs to cheating. So admins should not need to intervene.

What if the disable analysis feature turned off auto-score?
Then if the stones are on the board, the auto-score marks them alive and the players need to click the dead ones.

An OGS client scores correctly, OGS server scores incorrectly, confusion abounds!

Because those points were marked dame, this is not an autoscore mistake. One of the players marked the dame, so this is either score cheating or a mistake which the proper winner did not know how to correct.

1 Like

I think it’s this. It seems that the app, which the proper winner was using, does not expect territory completely surrounded by stones of one colour (with all the other colour stones marked dead) to ever need to be shown as dame.

The OGS score marking tool should not allow an area which is adjacent to live stones of only one colour to be dame. That is against the rules of Go. If black had naughtily marked e1 group as alive rather than dead then marking the left side as dame would have been legal, though naughty.