Score estimator is very wrong in handicap games, and also very easy to fix

Feedback Bug report
1

Context:
From previous forums post, and by carefully counting a few games, I know that the score estimator always uses “area counting” (Chinese-rules style). One might consider such a thing a bug when the game uses Japanese rules, but such a thing would only be a “small” bug because for the vast majority of games the difference is only one point, and “fixing” it is not easy (Chinese rules estimation programming is much simpler).

The bug:
However, by carefully counting some handicap games, I find that the estimator is always wrong (when estimating the final position of the game) by N-1 points in an N stone handicap game (even when using Chinese rules). That is: In a 4 stone handicap game, it always gives 3/4 extra points to black, and in a 9 stone handicap game, it always gives 8/9 extra points to black.

The reason for this is obviously that the “handicap stones” are all being counted as points by the score estimator. It is well known that if “area scoring” is going to be used (“chinese counting”), then N-1 points of additional Komi are given to white when playing an N stone handicap game (so that “Chinese handicap” and “Japanese handicap” are the same, otherwise Chinese handicap stones would give a slightly larger advantage).

Furthermore, searching previous forum topics, I discovered that the score estimator once did not count komi at all. This seems to be fixed now, but such precedent makes it clear that the “core” of the estimator just counts area and ignores komi at all, and komi was added later. Most likely, this extra N-1 points were not added there as well, and they should. Simply adding N-1 points to the white score estimate would fix this issue.

Also, I would like to note that, as far as I have seen, this N-1 points are correctly added when actually scoring a game that comes to an end: it is only the “score estimator” feature that forgets to count this points. This is specially important in 9 stone games, because the difference is quite large there, and also because such discrepancies are confusing, specially for novice players.

Additional Suggestion:

Since the score estimator:

  • Always uses “Chinese rules”
  • Is only an estimate

I suggest to add in the UI a text indicating so, in order to prevent misunderstandings. For example, right now a text like

“White wins by 9.5 points”

Is displayed. A message in a smaller font, like “(Chinese rules estimate)” right next to it / on top of it / below it could serve to help avoid misunderstandings.

11 Likes
2

Bumping the post as the issue is always reproducible and the suggested fix is very easy to apply.

3 Likes
3

this issue should be resolved shortly with a soon to come score estimate upgrade

7 Likes
4

SE being wrong by 9 points got me in a few recent games.

2 Likes
5

The score estimator is still very wrong and easy to fix for Japanese rules handicap games:

image
image514×742 176 KB

It’s confusing since Japanese rules specify no handicap compensation, but as long the counting is done with Chinese rules, the SE should apply compensation to get a count that corresponds closely to actual score.

Note that it does do the correct thing in Chinese rules games, so I think this fix just wasn’t applied correctly to all the rulesets.

I filed a bug report on GitHub too.

2 Likes
6

Nope. Under NZ rules, which use area counting, there is no compensation for handicap stones. I have in recent years played numerous handicap games against people from China and they don’t know about this compensation rule either.

3 Likes
7

A lot of players don’t have a full comprehension of the rule sets. And rules related to handicap returning stones most of times are only necessary when players of various different strength are required to compete in the same bracket.

To simplify the scoring, tournament I participated in the past that included handicaps usually have added lines in the competition rules

  • “段差一先，同段分先黑185勝；差一段讓先黑181勝；差二段讓2子黑182勝；差三段讓3子黑183勝；差四段讓4子黑184勝。”

Translated as "Every one rank(dan) difference will be given one stone difference. Players of the same rank, black wins with 185 area (the score is counted only with the black side); Players with 1 rank difference, black wins with 181 area; Players with 2 rank difference, black wins with 182 area; Players with 3 rank difference, black wins with 183 area; Player with 4 rank difference, black wins with 184 area.

And for competitions below dan ranks, the score difference is usually much higher, mistakes would cost more than the difference of several stones. So for simplicity, they would sometimes omit any additional rules.

4 Likes
8

I learned that bias late myself

9

Historically, dan rank is indeed related to elitism and privileges. And they would be charged more as well for their diplomas. Kyu rank diploma usually has the same basic fee regardless being 15k or 1k, but dan rank diploma is charged extra per rank up.

1 Like
10

You are right, my original wording was quite sloppy. At least I used quotation marks, to hint that the terms shall not be taken too precisely XD Apart from the N vs N-1 detail, the main theoretical point stands that unless you add some compensation (some rulesets do not), then area-scoring handicap will be larger in practice than territory-scoring handicap, for the same number of handicap stones.

Different area rulesets specify different compensations for handicap games. Notably NZ does not use compensation at all, so each NZ handicap stone is a larger advantage to black than in AGA/Chinese/Japanese, which for nine handicap games adds up to quite a lot of points. Chinese rules use 0 points for a 1 handicap game, but N points for games with N >= 2 handicap stones. AGA uses N-1 points for a N stone game, which is the one that mathematically allows to count territory (with passing stones) but still get the same as if area scoring + correction is used. That is why I mentioned the N-1 number.

So, in this regard, it is like the current score estimator always uses NZ score (no compensation), thus if your 9 stone handicap game uses any other ruleset and you rely on the score estimator, you will see quite a relevant difference. Either N or N-1 points of compensation would bring the number closer to all other rulesets. The ideal solution would probably be to change the estimator so that it actually subtracts the number that corresponds to the ruleset in use, that is N for chinese rules, N-1 for AGA rules, either N or N-1 for Japanese rules (it will not be exact, but both are a much better aproximation than not correcting), or 0 for NZ rules.

2 Likes
11

No, I think it got improved a while ago (as @BHydden promised) to have different behavior for different rulesets: Chinese uses N, AGA uses N-1, but Japanese still wrongly uses zero.

4 Likes
12

Oh I did not know that. We are getting close! :stuck_out_tongue:

1 Like
13

No the original plan was to use the KataGo engine for score estimates in game too, but much to my (continued) dismay it faced very strong opposition, so AFAIK we pretty much have the same SE we’ve always mad for players during the game…

1 Like
14

So @elsantodel90’s initial report was completely wrong? Or am I wrong in thinking that it scores AGA and Chinese games properly now?

3 Likes
15

Honestly, it could have had some minor adjustments and corrections that I have forgotten about. I was mostly clarifying that when you quoted me saying it should be resolved soon, the path I expected did NOT happen.

3 Likes
16

Maybe the handicap adjustments were added as part of that change and stayed in place after the AI improvement was reverted?

By the way, there’s this annoyingly wrong (in the other direction!) score breakdown under Japanese rules too:

Screenshot_20220118-132820
Screenshot_20220118-1328201440×644 124 KB

2 Likes
17

Just ignore the four and it’s fine :stuck_out_tongue:

I do agree having it there in Japanese rules is confusing. I feel like that could’ve been like that a long time

3 Likes