One more case (with no unfinished boundaries)
Here the game is perfectly finished, the assesment by the player is right and the result by the scoring processor is absolutly wrong.
1 Like
I do wonder if it was the scoring processor or a timing problem in which my opponent changed the result at the same time as I accepted the original result. the original result was correct.
I don’t think that game involved a problem with the scoring system per se. As I said in the cited thread:
"Going back years, the scoring clock has been irregular in its function. I have mentioned this several times in different threads. In 2016, when I first joined, the documentation said that the scoring clock had 5 minutes on it and that it would reset every time the board was altered. That was still true when I began moderating in 2018. When AdamR and I and others rewrote the documentation around that time, I verified the 5-minute reset with the developer. However, in the course of dealing promptly with score-cheating reports, I found that the clock often timed out much faster than the supposed 5 minutes after the board was altered. I have long suspected that the scoring clock does not reset as it should after each alteration.
“Bottom line: your opponent probably thought the position was seki, and the scoring clock failed to reset.”
1 Like
Seems the case (check in the other thread).
Sorry for false alert.
1 Like
Here a few more for reference (i think kids go server is built from ogs)
1 Like
Seems like a position that I would guess @Feijoa’s autoscore v3 to handle easily. Testing autoscore algorithms - #51 by Feijoa
2 Likes
My autoscore thing only attempts to determine life and death of the stones on the board. We need to hook it into your seki detection code to check whether or not there might be three random dame points scattered around Black’s territory!
1 Like
To be fair though, when playing with a bot, unless the bot knows how to argue, you might get answers you don’t expect if you’re not sure how to make groups alive.
I’m not sure what a good score would be for this game if both players disagreed, but I suppose one could call it a black win even if the white corner was marked alive.
I think you’ve mentioned before that bots could argue how to score the game, if the capability was enabled - but I’m not sure that’s a better experience unless it just agrees something like everything left on the board is alive (which’d be ok with area rules anyway).
1 Like
Yes, there is a well-established procedure in the GTP protocol bots use to mark dead stones and come to agreement on them, but OGS doesn’t use it.
I would guess the bot also thought the lower left was alive, given it didn’t attempt to kill it and doesn’t seem like the sort of shape a weak bot would incorrectly think is already dead.
2 Likes
Just saw this beautiful example of an autoscore failure:
9 Likes
Why did White accept that? Did Black manually mark that to score cheat?
It was blitz, they were in a hurry and probably didn’t notice. And no, I believe this is actually the autoscore result.
Doesn’t it pause the timer and replace it with the scoring timer during counting? I don’t play much blitz
Ah. imo autoscore ought to have respected the players’ clear beliefs about the position and marked that point as White: it reaches only white stones
2 Likes
That is one of the themes of this whole thread: the autoscore often substitutes it’s own AI knowledge for the players’ knowledge. So, you have to beat the autoscore as well as your opponent.
6 Likes
Specifically this is “beautiful” since even though it’s truly terrible scoring, the autoscore coloring echoes the complicated status of that corner shape. It is all based on an AI ownership estimate, which looks something like this:
The raw numbers for the important part of the corner are like this:
3 -0.556 -0.557 -0.550 -0.595 -0.617
2 -0.556 -0.049 0.602 -0.479 -0.162
1 -0.563 -0.235 0.565 0.525 0.264
E F G H J
One thing that’s surprising right away is that it seems to be only about 60% confident that the White group can stay alive at all. I don’t understand rectangular eight in the corner myself, but shouldn’t KataGo be able to know for sure at this point in the game?
Anyway I’m guessing it’s calling F2 dame because its ownership value is really close to zero. That sounds kind of reasonable, but the unfortunate implication is that points that will likely be owned by Black aren’t marked as dame, so instead (with only white stones around) they count fully as white territory!
1 Like
There are multiple sequences that lead to seki, and there’s also the possibility of a thousand-year-ko (that would likely result in seki anyway).
So I’d say it’s no surprise that AI is unsure about the ownership of points. Say e.g. that there are two optimal and (point-wise) identical seki sequences, but resulting in different points being occupied (= owned) by white / black, then I guess the confidence in their ownership would be low.
3 Likes
But shouldn’t it at least know whether the optimal sequence kills the big group? Right now it looks like it only gives it 60% probability to live, and the score estimate is around B+14.5. It shouldn’t be too hard to figure out any kos - Black has no external ko threats and White has (I think) just one.
1 Like