Use KataGo for Scoring

Feedback Suggestions
22

I think this is a good solution, but KataGo scoring is on the way, which should also solve this problem.

6 Likes
23

Keep in mind that the score estimate and territory estimate are somewhat independent in KataGo, i.e. counting up the 99% certain white squares and 99% certain black squares does not always add up to the score. I’ve seen it wrong by 1.5 point in fairly normal situations before.

2 Likes
24

Just wanted to cross-reference this recent announcement into this thread as well (for people who might stumble upon this later).

Glad to hear that is happening! Thanks @anoek

5 Likes
25

@anoek @sanderl
Noticed this while browsing over this old thread:

Yes, it is definitely true that the score estimate and ownership estimate are a bit independent. You’ll need to be a bit careful and not trust KataGo at face value if you want accurate scoring. There are a few different ways to do it, you might need to play around with it. Here are some things to consider:

  • If you’re trying to use it for marking of live and dead stones, you probably want some thresholding on the confidence level of ownership on each spot, or perhaps the average confidence level within each connected chain of stones (in case there is “noise” and one stone in the corner of some group happens to be more or less confident than others). Ideally try testing various kinds of complex and messy/bizarre sekis (perhaps also double-ko-seki, and such) to find the right thresholds.

  • You probably also want to run it at least twice, once with each side to move. If the ownership prediction varies greatly with each different side to move, that’s a sign that the region is probably unsettled, and then your dead stone or scoring algo should do whatever you want to happen in that case.

  • You may want to implement your own logic to override what KataGo says in some cases. For example, if there is a subtle weakness in someone’s territory that allows cutting and killing a part of it, then such regions should show up as differing by player to move and therefore unsettled. But if both players have passed and there aren’t any dead stones or anything near the unsettled spots, and the region is clearly enclosed, then in some sense, both players by passing have “accepted” that the territory there is settled regardless of the weakness, so probably you’d still want to fully count the territory for the player for scoring purposes.

  • As an aside, KataGo’s own internal gtp “final scoring” function in Japanese rules (which just tries to directly find the score, rather than by doing anything with ownership or dead groups), uses binary-like search to find the komi such that the winrate is as close to 50-50 (“draw”) as possible, as determined by a low-playout analysis for each different komi. Although not perfect, this seems to be a bit more accurate than just querying the “lead” output directly.

7 Likes