The new score estimator seems to be more certain about not fully resolved areas than it used to be. What I mean is that even if you click “estimate” several times in a row, it will assign an unresolved area more consistently to one player whereas before, it would sometimes assign it to neither.
In terms of accuracy, on average I feel like the estimator is now better at analyzing relatively open positions. In compensation, it now seems to mis-read almost resolved positions more often (and be certain about it).
Here’s one: https://online-go.com/game/14859396 (move 18 if it changes in the meantime) By my count I should be ahead by some 5.5 points. The estimator gives the match to the opponent by 25.5 points I know my shape is not that good, but it’s quite hard for me to imagine how I could not make any territory on the left side…
I also noticed a recurring pattern. In the position above for example the stones at E2 and F2 are marked dead. This is hard to believe. If you press the estimator a few times, even the stone at E8 shows up as dead which is definitely peculiar. That happens a lot: the estimator misjudges the diagonally connected stones even when they are close to the edge.
One last point: in the same game above, in the second variation I shared in the Malkovich chat (a fully resolved position), if you press estimate it gives J1 as black, when in fact it should be dame:
In this game (move 176, not over yet) it states that the black stones (marked red) are dead. If I read it right, the black stones should be alive (or at least marked unsettled).
I let the SE run several times (more than 20) and all times they are assigned dead.
The 2 white stones (marked green) instead are either unsettled or assigned to white (depending on of the SE run) although they are already in atari.
PS: SE seems to like to assign the neutral points D8, J7 (and H19) to one player (white or neutral, never black) while Q9 and R12 are always correctly neutral.
PPS: Fun fact. If I mark the red marked stones as dead by clicking on one of them, the SE marks the black group above as dead.
@anoek I think I may have an idea why this is happening. I reckon there is no code to prevent a player (in this case black) from filling their own eyes. In this case the random player fills J9 as black and white capturs on G8. I think we need to have a provision against this situation.
The first incident report I handled yesterday was both players infilling eyes. I sent a pm to both explaining how the game is supposed to end. So yes, infilling still occurs.
Not sure if you are still looking for issues with this. I recently looked at someone else’s games and found one where the autoscorer sees a seki where I am seeing death:
Yes, there is some misunderstanding, all games are scored by our server and any mistakes are our fault While sometimes bots make similar mistakes in thinking a dead group is alive the end result is always “our” calculation.
It’s done. Not actually a ko: black takes one stone, white takes the black group, black fills.
But the funny thing was the sort of schroedinger groups: they where both alive in the game and both dead in the estimation.
Using a variation to complete the game(see game chat), the SE got this basic two-shared-liberties with outside liberties filled, position wrong. I was unable to correctly adjust the assessment with click and shift-click.
The following game was scored wrong. The players made the mistake of trusting the auto-score but I was surprised to find the SE consistently got such a simple situation wrong when I tested it:
SE gives the four Black stones as dead instead of the two White ones (in the bottom right).
Have we reverted to an earlier version of the auto-score and score estimator for some reason?
I know my shape is not that good, but it’s quite hard for me to imagine how I could not make any territory on the left side…
While sometimes bots make similar mistakes in thinking a dead group is alive the end result is always “our” calculation.
