Autoscore is still overriding players’ skill level

After the long discussion that occurred in another thread when the new autoscore was introduced, I thought it was agreed that the algorithm would be fixed to prevent the kind of thing seen in the game cited above. It is obvious that neither player saw the tesuji in this game, due to their skill level. This is not like the common case of groups that have false eyes and are counted as dead because one doesn’t have to capture dead stones. This is a two-move tactic, i.e., a variation. Where does it end? Will SDKs need to see some dan variation or lose their games to this kind of scoring?

In this case, the weird scoring did not change the outcome, but it certainly could have if the score had been closer.

The players are playing each other, not playing against KataGo’s autoscore.

5 Likes

no single opponent stone in territory below. Just add area fill algorithm in such cases already!

2 Likes

I don’t get this point.

I think both players agreed with this weird score, was it because beginners or was it because “ok, you won, whatever, I don’t care to fix the score, it doesn’t change anything”.

This automatic score can be seen as a hint (“watch out, the border isn’t closed yet!”) but how could autoscore decide on behalf of players?
They have the “accept score” button, don’t they?

3 Likes

that’s why there shouldn’t be unpainted territory except in seki or something without completed borders

and there should be more clear instruction that players have to click the board, not just accept.

3 Likes

The scoring should respect the players opinion. If they delimited territories like here, it should not contest their agreement and calculate the score as it is.

4 Likes

One choice is: leave players alone. Let them score the game as they will.

:tired_face: oh no, it’s boring!

Another choice is: get an extimator that does the boring work.

:tired_face: oh, no, it’s wrong, it’s misleading!

Right. So let’s take a strong AI that won’t be fooled by easy tsumego.

:tired_face: oh, no, it’s giving tesuji hints!

I fear we really can’t make everybody happy. Can we? :innocent:

Shall we make it an option?

  • yeah, please, do it all for me
  • please, do it but let me check and confirm
  • I’ll take care of it
2 Likes

Yes, but there was no point in rejecting the result since it did not alter the outcome. Also, many players (and not just beginners) don’t know how to manually adjust the scoring. This is beside the point, however. If the scoring had changed the outcome and White had rejected the score, Black (whose turn it was) could have played the variation and won. If White adjusted the scoring, Black could reject it and play the variation to win.

It is not a question of making everybody happy. The autoscore is violating site rules by providing outside help (indeed, bot help!) to the players before the game is formally concluded. Furthermore, as I said, the players are playing each other, not KataGo. The game should be scored as it stands, not on the basis of some hypothetical variation that neither player knew. The previous version of the autoscore did that.

4 Likes

From the log we have, it seems that one of the players selected the stones as dead, and then both players agreed with the score.

Autoscore wasn’t used at all.


[Edit:]

I tested it, and my above statement is simply false…

3 Likes

At first, I read this as exterminator, and thought what does that mean? Like a pedant that takes rules very seriously?

3 Likes

Well, that is a relief. However, the statement,

is ambiguous. Do you mean it didn’t show the variation, or it no longer kicks in automatically when both players pass, leaving players to score for themselves if they want?

1 Like

I was under the impression that players always have to select the stones themself. Let me test it.

Hmmm, no, you’re right after all. I thought autoscore wasn’t used, since it says the players changed something, and it specifically says “force_autoscore: false” as well.

But here’s what the scoring stage looked like without touching anything:

Since how long do we have scoring procedure on internet? A few decades. Is scoring a huge and reccurent problem? Not really, problems arose sometimes which induced a call to moderation.

Now we get the help of AI. Nice but Let’s keep it as a powerful help, called on request then. Because It’s powerful, the scoring should become definitive, under the control of an admin because the AI is still not perfectly working. In case of disagreement, not for replacing the well working side of the scoring way used since long ago (mark yourself boundaries and prisoners)

This last game scoring is interesting

If both pass, mark boundary and prisoners then a standard light algorithm give the result.
If one player continue play let see what happen.
If both disagree, then one can call moderation as now. Game result is given by AI but stay hidden ‘under process’ msg until checked by admin. (Clocks frozen too ofc). Final decision given by admin (with AI help) and definitive.

That’s not a problem related to AI but to the call for moderation itself which existed before AI.

Calling for a mod before the end of the game (scoring done) breaks the rule of no external help but is legal as an exception.

For not being too intrusive, should it be definitive decision under admin control called by disagreement between players?
Offering the option to continue the game provide opportunities for unfairness and eager but that could remain a choice of decision by the admin maybe afterall, with the other alternative to cancel the game.
It could be interesting to have the opportunity for the admin to ask to resume the game without having the analysis from the scoring tool given to the players. In that case the boundary/prisoners/territories marking had to be made at 100% by the players without any help.

2 Likes

The sentence from me that you quoted was referring to help in game play (prohibited), not help in scoring (allowed). Mods don’t give help in game play the way the autoscore did in this game.

2 Likes

I completely understand the difference and the care an admin may have to respect it.
There are indeed good reasons to have this help legal. Now this is not like there is a clear cut because there is potentially help in game play during help in scoring (which i guess can make the admin job sometimes difficult )

1 Like

So in this specific case does anyone know what the actual score should be under Japanese rules? Is that right side group actually dead and therefore it’s a giant seki? Seems like white can choose which group to give up after the cut so I don’t know why one would be called dead and not the other.

The white wall with cutting weaknesses neither player notices is alive unless one of them asserts it is dead, so the whole lower area is white territory.

4 Likes

But if Black asserts that a group is dead, as they sort of did here, what then? It seems like White could save any particular group if necessary.

No Black did not assert that the group is dead. Vsotvep’s edit indicates that the autoscore really did do it.

Well, it asserts that the autoscore may have done it, but truthfully, we cannot know.

What the log does definitely confirm, is that both players accepted the score.

1 Like

But supposing that they did actively want to, would it be correct to assert that any groups are dead here under Japanese rules? I suspect that the answer is no, and the score estimator is just completely wrong, but my limited understanding is not deep enough to be sure.