Score estimator improvements


#21

Oh if it’s all Chinese already no worries then. I mostly only think about the bots. :wink:


#22

i think accuracy should be priority until end game. if close in end display .5 loss to each side.


#23

The new score estimator seems to be more certain about not fully resolved areas than it used to be. What I mean is that even if you click “estimate” several times in a row, it will assign an unresolved area more consistently to one player whereas before, it would sometimes assign it to neither.

In terms of accuracy, on average I feel like the estimator is now better at analyzing relatively open positions. In compensation, it now seems to mis-read almost resolved positions more often (and be certain about it).

My observations so far :slight_smile:


#24

Awesome thanks for the feedback, if you have any links you can pm me to games it’s not doing right, I’d really appreciate it :slight_smile:


#25

Here’s one: https://online-go.com/game/14859396 (move 18 if it changes in the meantime) By my count I should be ahead by some 5.5 points. The estimator gives the match to the opponent by 25.5 points :open_mouth: I know my shape is not that good, but it’s quite hard for me to imagine how I could not make any territory on the left side…

I also noticed a recurring pattern. In the position above for example the stones at E2 and F2 are marked dead. This is hard to believe. If you press the estimator a few times, even the stone at E8 shows up as dead which is definitely peculiar. That happens a lot: the estimator misjudges the diagonally connected stones even when they are close to the edge.

One last point: in the same game above, in the second variation I shared in the Malkovich chat (a fully resolved position), if you press estimate it gives J1 as black, when in fact it should be dame:


#26

Score estimator works very well (as far as I can tell).

I found only 1 case in which it assigns the state of a group wrong.


In this game (move 176, not over yet) it states that the black stones (marked red) are dead. If I read it right, the black stones should be alive (or at least marked unsettled).
I let the SE run several times (more than 20) and all times they are assigned dead.
The 2 white stones (marked green) instead are either unsettled or assigned to white (depending on of the SE run) although they are already in atari.

PS: SE seems to like to assign the neutral points D8, J7 (and H19) to one player (white or neutral, never black) while Q9 and R12 are always correctly neutral.

PPS: Fun fact. If I mark the red marked stones as dead by clicking on one of them, the SE marks the black group above as dead.


#27

:face_with_raised_eyebrow:

when 2 eyes are not enough. SE is getting creative there.


#28

@anoek I think I may have an idea why this is happening. I reckon there is no code to prevent a player (in this case black) from filling their own eyes. In this case the random player fills J9 as black and white capturs on G8. I think we need to have a provision against this situation.


#29

Well that’s interesting. No there is code to prevent self filling eyes, but apparently it’s broken in some cases!


#30

The first incident report I handled yesterday was both players infilling eyes. I sent a pm to both explaining how the game is supposed to end. So yes, infilling still occurs.


#31

Not sure if you are still looking for issues with this. I recently looked at someone else’s games and found one where the autoscorer sees a seki where I am seeing death:


#32

That is a bot game, and I believe the bots use their own scoring algorithm. In this case, the bot does not recognize the dead shape.


#33

Are you sure ? I thought they are scored by the server - when I looked into bot protocols, I did not see anything about scoring…


#34

Yes, there is some misunderstanding, all games are scored by our server and any mistakes are our fault :smiley: While sometimes bots make similar mistakes in thinking a dead group is alive the end result is always “our” calculation.


#35

https://online-go.com/game/15444091 After move 151 bH17, the score estimator invariably regards bG15 and bG16 as dead.


#36

https://online-go.com/game/15666689 After move 36 wH9, the score estimator invariably regards bH8 and most of the time bJ7 as dead.