Auto-score improvements

For what it’s worth, there are also those who think it’s fine to seal the edges here and there. And definitely fine to mark obviously dead stones as dead. These features make scoring a game of go easier in the vast majority of cases.

But sure, the more consistent and understandable the system, the better. And there are some really difficult cases where automatic scoring can still be surprising and inscrutable.

4 Likes

I see, AI estimator was replaced with that pseudo-scoring algorithm


Bring back normal AI estimator!


When game ends by resign or timeout I wish to instantly see how board would look like if game was 100% completed and then scored.
I don’t need those nonsense empty spots.

3 Likes

Ok @Vsotvep et all, new plan is to attempt to clearly mark points that need sealing and prevent score acceptance until they’ve been resolved

image

It hasn’t been a quick fix this morning, so not sure how long before it’ll be ready for production, but hopefully this week at some point.

16 Likes

It was unintentional, it’ll be restored to it’s previous behavior

8 Likes

@anoek If it’s useful at all and you haven’t addressed this some other way yet, once life and death is determined reasonably then GitHub - lightvector/goscorer: Territory scoring in Go with seki detection may be useful for omitting points in sekis when doing Japanese rules. It also does similar-to-KGS omission of points in obvious false eyes that will need to be filled (although, if you prefer not to have that behavior and only want the seki behavior, you can also set score_false_eyes to be True). However, the scorer does expect strictly surrounded territories.

KataGo’s raw neural net does have a seki prediction, but as far as I’m aware the normal ownership prediction exposed by the interface doesn’t normally help tell apart seki from independently alive groups.

If there are any weird corner cases discovered in the seki detection algorithm, happy to look at and try to fix them.

10 Likes

Very cool. I think our seki detection scoring code is working as expected, though false eyes we basically rely on katago to mark those as dead for us.

Edit: I was remembering some very old code, we used to (optionally) apply seki scoring logic, but at point a few years ago we disabled it as hardly anyone preferred it and it caused a lot of headaches because unlike hexahedron’s fancy scoring system, we relied on the users to appropriately mark dame and dead stones, and the whole process was far too hard to get right for the majority of players.

I believe hexahedron was referring to giving Black only 2 points for

  B B   B B
B - B B B - B
B B B - B B B
W W W B W W W
W - W - W - W
  W W W W W

​- not 3 points - rather than to when falseness matters for life-and-death.

4 Likes

Do you have a game link where something is off @hoctaph? That’s some old code that does the seki stuff, it was my understanding that it was working fine so long as everything else is scored correctly, but if it’s not I would very much like to know.

1 Like

I forked an old 7x7 game ( 友谊赛) with some “big” seki

It’s definitely not correct markings

If I recreate it on a 9x9 board say

I’m not sure if the way it’s (auto)scored is correct in any ruleset.

In Chinese/AGA rules I’d expect J1 to be a point for black, but in Japanese rules, there should be no points for anyone, which means G7 shouldn’t be marked dead, and there’s no points there for black (which includes G9). Black should actually remove it from the board to gain a prisoner in Japanese (and presumably Korean) rules.

Edit: Maybe this is slightly complicated because it’s a dead shape for white, but actually killing it opens up a ko that black can’t win, it’s not like a bent four, hmmm. It’s seki because you can’t demonstrate the white stones are dead in hypothetical play.

6 Likes

Let me get this straight. So basically the whole OGS community came together for a month and discussed ways to improve the scoring system, including many different ideas and two different polls.

Only for OGS to then proceed with further work on the “AI-assisted approach”, which has almost unanimously been deemed the worst possible method for scoring.

I hope this improvement was at least a very low hanging fruit.

I made another case that might be an interesting test. Black vs. White. But I don’t know how to test OGS on it. How do you fork a game to play against yourself to test a case? Do you need multiple accounts? I tried the ‘goban’ repo as @anoek suggested above but I got an error like:
Error: autoscore_test_files/game_64848549.json correct_ownership field contains "?" which is invalid

Maybe someone else can run it to see how OGS does?

Anyways, for the game I linked, here’s how the territory scoring algorithm at Territory Scoring Test handles it once dead stones are marked:

  • A19 - this is a (false) eye in seki and so is not a point for black.

  • M19 area, T19 - these are also eyes in seki and are not points in territory. As a deliberate choice, M19 and N19 are still counted as points for captures because they are marked as dead even though the territory underneath the stones is not (and should never be) counted. This is a slight departure from the Japanese rules that mandate that you physically capture such stones during play before ending the game. Gven the realities of server play and interfaces it’s a deliberate choice from the scorer:

Behaving more strictly could require careful corresponding UI work, since if stones that users explicitly mark to be dead are sometimes not counted as points, it may be confusing for users, and it may not be easy to see or know when they are and are not.

This choice is also more “AI-resistant” in that AI ownership markings will likely always mark such stones as dead (because future play will trivially be able to kill them), and so we should just count them as points rather than confusing users with a magically tricky exception that in this one special case, marked-dead stones somehow don’t count as points.

Obviously, in this particular case black should capture them and force white to throw in once more, gaining an additional point, but it’s not the scorer’s job to determine that, so we deliberately score only 2 points here for the marked stones, not 3. (And still zero for the territory).

  • S12 area, T8 - similar thing going on here. No territory, but a point for the dead-marked stone. This is a seki. Part of what also makes this interesting is the bamboo joint here - the solver is good enough that it doesn’t screw up recognition of the seki.

  • T4 area, T1 - also a seki. No territory. It doesn’t matter that the seki is “collapsible” if black plays S2 and can kill white’s stones, because white can kill S5 and T5 in return, and that’s enough for white’s stones to be decreed alive and this to be a seki if scored as-is.

  • A3, C3 - also false eyes in seki, no territory here.

Okay, that’s the sekis. Next, we have optional false-eye non-scoring, which is a convenience that KGS and most other online servers offer. It’s not mandatory to have these - it’s also a consistent choice to decide to count all of these as territory and have players fill the dame to force the protections. And the algorithm I implemented offers an option that can do that, if desired. But otherwise, by default it will not count the points:

  • E5, L4, N13 - the algorithm detects correctly that these can never be a point of territory.
  • A8, A9 - the algorithm detects correctly that these both can never be points of territory, due to the presence of the throw-in stone, but still counts 1 point for the marked dead throw-in stone.
  • L8 - the algorithm correctly detects that this can never be a point of territory, despite the bamboo joint above that a naive algorithm might think it could reach the upper group (If it could actually manage to connect out that way, then the false-eye-point would become scorable, like so:)
    image
  • D9 - the algorithm intentionally does NOT detect the necessary protection here to future double-atari. Neither does KGS nor (as far as I know) other popular servers. And deliberately so - the convenience to exclude territory points is only offered for “obvious” false eye spots, and is not meant to replace player judgment in any more interesting cases where protection is needed.

Anyways, if you like, seki detection algorithm at GitHub - lightvector/goscorer: Territory scoring in Go with seki detection can still toggle this.

8 Likes

Does this only affect territory scoring? There should be no question how to score when area scoring is in effect, right? That’s why I prefer it, at least online.


Ian

1 Like

@hexahedron that’s some great code you’ve got there, FYI I’m in the process of incorporating it. Thanks for putting that together and putting it out there :slight_smile:

11 Likes