Improving OGS' scoring system

I assume someone will mention this in the next 29 replies, but wasn’t someone working on a seki-detection algorithm (given correct marking of strings)?

Agreed. Also hitting accept score should send the string status the client is displaying to the server, and if it does not match what the server has, it rejects the acceptance and waits for the player to hit it again, or something. Some sort of race-condition safety; someone else can probably think of a better way to do it

As long as it always requires 2 passes to reenter scoring, instead of just 1 if the game was just resumed

Yeah, seems like a good algorithm to me

1 Like

It came up here: Score Estimate vs Result - #11 by gennan

Also, someone mentioned seki detection was implemented in GnuGo, which is open source: Score Estimate vs Result - #22 by SouthernGoPlayer

I don’t know if anyone is working on porting it to OGS, but I assume nobody is.
I did consider to try it myself, but I’m sort of always lacking time. And if I would, I’d pick my favourite programming language for it, which isn’t PHP or javascript.

1 Like

Yeah, that was the conversation I was thinking of

Oh, I forgot that @hexahedron (the creator of KataGo) already did it!
Open source territory scoring with seki detection

1 Like

Oh yeah, that was the one. Hexahedron is one of those people who it feels like has been making some of the most impactful tools for a long time. If I’m not mixing him up with someone else, he also created my favorite Arimaa notation (I don’t know an official name for it, so I just call it “Lightvector Notation”)

1 Like

The trouble is that this isn’t defined. The autoscore, to take just one example, currently marks hanging stones as dead even when they haven’t been cut (i.e., when neither player recognizes the necessary play). Even worse, it sometimes marks stones dead because they can be captured by 2-move plays even though neither player sees it. This has all been documented in the “bad scoring” thread (I don’t remember the exact title). In IRL play, these things don’t happen. If neither player can see how to capture, then the capture doesn’t occur. This is primarily a problem for beginners, but the more complex tesuji seen and scored by the AI are sometimes beyond experienced but middling kyus as well.

The hanging stones case is especially bad because it is the most common, and it causes all the adjacent, interior points to be nullified due to the now-open boundary. In a real game, a player might make the cut capturing the stones, but the opponent would then close the boundary, saving the interior points.

When the AI substitutes its scoring knowledge for the player’s knowledge, it has become a third player in the game.

3 Likes

Yes, I wasn’t talking about OGS’s (OGS’?) current autoscore algorithm, but a hypothetical much improved autoscore algorithm.
I expect @Feijoa’s autoscore v3 algorithm to come a long way in the direction of improvement. He guesstimates it to be correct in 99% of cases now and expects to be able to improve to 99.9% when given more attention. But perhaps it won’t become as good as 99.99%, which I think is the lower limit for fully automated scoring on OGS.

1 Like

Since the code hasn’t changed in a while, I removed the testing-in-progress notice from GitHub - lightvector/goscorer: Territory scoring in Go with seki detection and added a v1.0.0 “release” tag for the code as it stands now. If anyone wants to use it as part of any new better scoring logic here, or incorporate it into a client, or whatever, obviously feel free. Poke me if any bugs are found. :yum:

5 Likes

I estimate that our current autoscore is already correct 99% of the time if the borders are closed, just based on my own experience with it. And mine seems to handle any actual failing cases just fine so it could be an order of magnitude better. But I haven’t played around with different rulesets or really tried to break it much.

Here, maybe this will be a controversial case:

image

Autoscore v3 marks both bottom groups dead - since if White plays first, the lower-left one dies, and if Black plays first, the lower-right one dies. Without any unquestionably live black stones in those corners, it can find no justification for keeping the groups alive.

But in a real game, if the players passed, of course they didn’t see the weaknesses, right? A while ago I brought up an almost identical position here (that original one actually turns out to work fine) and everyone but me thought the groups should be scored as alive.

It seems hard to “fix” this kind of situation - a 1 in 10,000 kind of game - and I’m skeptical that it’s even clear what “correct” would be in this case. Or if you make a clear rule that covers this case, it’s probably going to break some other weird 1 in 10,000 kind of situation in a bad way.

what about marking something as dead ONLY if its dead in ALL futures?
and everything else as alive by default

Like, a dark mirror of pass-alive? would still be dead even if given arbitrarily many moves with which to defend? is “pass-dead” a term?

That’s a pretty weak definition of dead, might as well make it fully manual.

3 Likes

in example like above 2 simulations is not enough. Black can choose which group to save and white can choose which group to destroy. So at least 4 different simulations are needed. And if some group is alive in at least 1 simulation, it should be counted as alive I think.

1 Like

One problem is that a single abandoned stone might surprisingly become alive if it’s close to a surprise weakness.

Great idea! What about the option to go full auto resolve? Seems like there are not too many fans for that idea here but it’s what the big guys (fox, tygem) do and it works great, is fast, it’s easy to implement (I’d assume) and it removes the possibility to “score cheat” as well as to stall the game.

The worst possible outcome of this thread would be the creation of additional game settings. There are already way too many so this should be avoided at all costs even if that means not changing anything at all.

Wouldn’t be too sure about that. Question is if OGS wants to implement something like that.

1 Like

Has this changed recently? Following the link, it looks like they’re marked alive. And that’s what I’d expect. If both players pass with unsettled stones on the board, then someone didn’t know how to kill them, so they should be scored as alive? (Adding a question mark because I bet someone will come up with a counterexample to that principle…)

I see it marking as dead (:black_large_square:)

1 Like

If some players strongly feel like marking all their dead groups manually, and some want quick autoscore, why not give them the choice? I don’t see why it would have anything to do with all the other choices.

It’s possible for the results to be somewhat random, but here’s what I always see in this case (square means dead):

image image

Finding a reasonable answer to that question is the whole problem. Maybe they didn’t try to kill them because both assumed they were already dead? Like in this case for example:

image

You would leave those five random unsettled stones alive?

2 Likes

Ah, sorry, I was misreading the icons. I thought black plus sign = black territory!

In the scenario where it has to be scored with no human intervention ever, no. But for my taste, it’s the type of situation where the system should be leaving stones unmarked and asking humans to click on “we agree these are the dead ones”. (Yes, I realise we’re going round in circles now.)

1 Like

I admit I also tend to start with a strong negative opinion on increasing even further the OGS cockpit… but in this case how would it even work? Would it be some setting somewhere that, if somehow independently toggled on by both players, triggers auto scoring?

1 Like