The new score estimator seems to be more certain about not fully resolved areas than it used to be. What I mean is that even if you click “estimate” several times in a row, it will assign an unresolved area more consistently to one player whereas before, it would sometimes assign it to neither.
In terms of accuracy, on average I feel like the estimator is now better at analyzing relatively open positions. In compensation, it now seems to mis-read almost resolved positions more often (and be certain about it).
My observations so far
Awesome thanks for the feedback, if you have any links you can pm me to games it’s not doing right, I’d really appreciate it
Here’s one: https://online-go.com/game/14859396 (move 18 if it changes in the meantime) By my count I should be ahead by some 5.5 points. The estimator gives the match to the opponent by 25.5 points I know my shape is not that good, but it’s quite hard for me to imagine how I could not make any territory on the left side…
I also noticed a recurring pattern. In the position above for example the stones at E2 and F2 are marked dead. This is hard to believe. If you press the estimator a few times, even the stone at E8 shows up as dead which is definitely peculiar. That happens a lot: the estimator misjudges the diagonally connected stones even when they are close to the edge.
One last point: in the same game above, in the second variation I shared in the Malkovich chat (a fully resolved position), if you press estimate it gives J1 as black, when in fact it should be dame:
Score estimator works very well (as far as I can tell).
I found only 1 case in which it assigns the state of a group wrong.
In this game
(move 176, not over yet) it states that the black stones (marked red) are dead. If I read it right, the black stones should be alive (or at least marked unsettled).
I let the SE run several times (more than 20) and all times they are assigned dead.
The 2 white stones (marked green) instead are either unsettled or assigned to white (depending on of the SE run) although they are already in atari.
PS: SE seems to like to assign the neutral points D8, J7 (and H19) to one player (white or neutral, never black) while Q9 and R12 are always correctly neutral.
PPS: Fun fact. If I mark the red marked stones as dead by clicking on one of them, the SE marks the black group above as dead.
when 2 eyes are not enough. SE is getting creative there.
@anoek I think I may have an idea why this is happening. I reckon there is no code to prevent a player (in this case black) from filling their own eyes. In this case the random player fills J9 as black and white capturs on G8. I think we need to have a provision against this situation.
Well that’s interesting. No there is code to prevent self filling eyes, but apparently it’s broken in some cases!
The first incident report I handled yesterday was both players infilling eyes. I sent a pm to both explaining how the game is supposed to end. So yes, infilling still occurs.
Not sure if you are still looking for issues with this. I recently looked at someone else’s games and found one where the autoscorer sees a seki where I am seeing death:
That is a bot game, and I believe the bots use their own scoring algorithm. In this case, the bot does not recognize the dead shape.
Are you sure ? I thought they are scored by the server - when I looked into bot protocols, I did not see anything about scoring…
Yes, there is some misunderstanding, all games are scored by our server and any mistakes are our fault While sometimes bots make similar mistakes in thinking a dead group is alive the end result is always “our” calculation.
https://online-go.com/game/15444091 After move 151 bH17, the score estimator invariably regards bG15 and bG16 as dead.
https://online-go.com/game/15666689 After move 36 wH9, the score estimator invariably regards bH8 and most of the time bJ7 as dead.
This is nice too: to be (alive) or not to be (alive)?
If black’s dead, white should live. Shouldn’t he?
Not both dead anyway.
The game is unfinished and still ongoing. Happy to say more after it’s done.
i sometimes use analyzer to play forward to let the dead stones die so estimator gives me a little more accurate score estimate.
then of course there is this option to count manually, which is always more accurate and needed skill
It’s done. Not actually a ko: black takes one stone, white takes the black group, black fills.
But the funny thing was the sort of schroedinger groups: they where both alive in the game and both dead in the estimation.
Using a variation to complete the game(see game chat), the SE got this basic two-shared-liberties with outside liberties filled, position wrong. I was unable to correctly adjust the assessment with click and shift-click.
A prophetically named game as it turns out. ^^
The following game was scored wrong. The players made the mistake of trusting the auto-score but I was surprised to find the SE consistently got such a simple situation wrong when I tested it:
SE gives the four Black stones as dead instead of the two White ones (in the bottom right).
Have we reverted to an earlier version of the auto-score and score estimator for some reason?