Not sure whether this is a known error, couldn’t find anything.
In this irl tournament game, recorded on SmartGo and imported to OGS via the SGF library, the count via “score estimate” at the ending position is about 5 points off from the correct score (as shown by the non-score estimate AI eval, or by a manual count). This is not an issue of AI finding some aji, ruleset issues regarding dame parity or seki, etc. Does anyone know what causes this? My opponent noticed the score discrepancy after I sent him the SGF on OGS and we had to recount a few times to verify.
This thread seems to describe the same issue but the replies are not relevant to the described problem and it veers into a tangent as is tradition for these forums.
Every AI makes trade-offs between speed and precision. A score estimator should be fast, so it will lack in precision as a result. Go is a much harder problem for computer than chess for example. Yes, AIs got much better, but it still takes a magnitude or more more computing to get to an equivalent result.
The score estimator shows which points are counted in which way. When I open the OP game intersections in the center are not marked, that are definitly terretory. For both sides. I did not count it, but it could definitly be in the 5pt region.
I’m not sure if AGA as a ruleset is propperly implemented in the score estimator. This could explain the linked one point swing for sure, especially when dame is not filled completely.
I am aware that what you describe is an issue with any score estimator, or with differences between rulesets (both of which I explicitly note are not what’s happening here in the OP). But it is clearly not what’s happening in the linked game, where Feijoa’s explanation seems to be correct. Please use this thread to discuss this particular score estimate issue, and not to bring up unrelated properties of score estimators.
In case there are reproducibility issues, this image shows what’s going on:
It’s requesting the score with Japanese rules instead of AGA. The estimator returns W+4.2, and the front end adds 7 to account for the non-standard komi to get to B+2.8. However captures were not taken into account in accordance with AGA rules, and that’s where the error of six comes from. (That happens around here in the code.)
Digging in a little more I noticed this in the console:
Maybe someone can now figure out why the ruleset for this particular game is “AGA (Territory)” instead of just “AGA”. That string does not appear in the public source code as far as I can tell.
That score estimator estimates only from the current position without accounting for any history. When you score from move 136, the ai thinks black can take the ko. You can think of it as “do a one-time reset of all kos, then play from here with the ko rule active”. The ai review doesn’t have this issue.
If you estimate from right before a player takes a ko, and never right after, you will get the correct result. I think it’s reasonable for the score estimator to take into account only the position on the board.
From my experience this is not correct. The post-game score estimator seems to just query katago and works just fine for unsettled positions (obviously the in-game score estimator is different). It is useful for exploring variations when reviewing the game.
estimator is way to see painting on the board itself
user would expect estimator to work same as those numbers, its how AI Sensei and other Kata viewers work
user would not expect for estimator to suddenly miss something that those numbers have, no information about it is written anywhere
When we talk about “The OGS Score Estimator”, we (? which “we”) are talking about the in game estimator.
After the game: sure AI is used, and it’s an AI assessment of whatever position you’re on, that is specifically answering “what does the AI think the outcome will be”.
However in game the estimator is not AI, and it is only really capable of answering “what is the score that is on the board?”.
Which means that if the board is not even scoreable, we can’t expect the answer to resemble “what will the outcome of the game be”.
We still haven’t really “got the communication about this right”.
The thing is that users tell us they don’t want their opponent to have an accurate estimate in-game.
It’s like “allowing AI assistance”.
This is the reason why when AI came along, the in-game estimator was not updated.
It is deliberate.
The remaining problem is how to make this obvious to every new person
Note: it is so confusing I even could be confused which score estimator each post is referring to
