It would be a game setting, like disable analysis, Canadian time, Pause on Weekends, or Japanese rules - all of which seem way less interesting to me than this, if we’re talking about having too many features.
2 Likes
I suppose such problematic cases are not that common, but perhaps when KataGo detects unfinished business like this (group status depends on who plays first), the autoscore might continue to investigate the situation by letting one side passing twice or more times to determine some measure of “deadness”.
In the 1st case, that might reveal that both of black’s groups are in fact half-dead, instead of one or both of them being fully dead.
Perhaps autoscore can also use some topological measure of “enclosurement” by a half-dead group.
In the 2nd case it would notice that black passed while failing to close off any area with his group, so it can be inferred that the defending player (black) considered that group dead. In the 1st case a measure of “enclosurement” would be in favour of considering both black’s half-dead groups alive.
I suppose there will still be situations similar to the 2nd case where the unsettled situation is so symmetric that there is no way to make a sensible adjudication, even by a committee of expert human referees. In such cases, perhaps a fully automatic autoscore should be able to annul the game. Either way, I wouldn’t consider any adjudication by autoscore a mistake when even expert human referees would not be able to reach some adjudication consensus. There will always be actual edge cases, but I expect that those will be rare.
Educating users on the scoring process and the need to manually mark dead stones will not be the easiest, but I’m afraid it will be extra confusing if sometimes it’s automatic and sometimes not. 
4 Likes
Fragmentation aka OGS’ biggest enemy.
OGS may need to add one more dimension for used scoring method to the rating breakdown, in addition to dimensions for time control and board size.
1 Like
Not sure if everyone is aware, but there are two “score estimates” available during the game. There’s also the third one available to observers and at the end of the game.
- Click on the player cards, around where the number of captures/komi/handicap is shown. This one is very simple: just counts stones/prisoners plus fully surrounded territory. It has no knowledge of life and death.
- Click on “estimate score”. This gives lots of hints about what’s alive and dead. It also seems to be wrong often enough to be annoying.
- Use KataGo to get a “best play” score. Not available to players during the game. But I believe this is what “estimate score” does for observers of the game? Not sure.
- Feijoa’s new algorithm.
Honestly, I’m not sure which of #2 or #3 is used for autoscore. Maybe it’s different between the button available to players during human vs. human games and the always-used one for bot games.
(Are there more scoring systems than this?)
IMO:
- #1 is a better scoring system for players, both during the game and during the scoring phase.
- It can/should be augmented so that clicking on a chain of stones marks the chain dead.
- For territory scoring, it can/should be augmented so that clicking on fully surrounded empty/dead points toggles between matching the surrounding colour and empty (to model seki).
- We should change the “estimate score” button available to players during the game to use #1.
- We should remove autoscore for human vs. human games and instead give them #1.
- For bot games, where OGS currently (for “reasons”) always uses autoscore, we should use Feijoa’s improved autoscore.
4 Likes
I believe it’s using the strong AI (3) to mark dead stones, but a more traditional algorithm (1) to score the board once dead groups are removed.
I think so.
I wonder — is the accuracy of the SE available to observers dependent on their supporter level?
AFAIK no, it’s always Katago.
1 Like
The first poll only included three similar approaches. I think it would make sense to also include the other major approaches to then conclude this thread before we discussed all of this for nothing.
Poll closes on sunday at 10:00 CEST.
How should the OGS scoring phase work?
- Auto scoring: The game just ends without a scoring phase. AI determines the result. Players do not need to do anything. Basically the approach that the biggest Baduk server Fox uses.
- Manual scoring: The territories and score resulting from the ongoing marking of dead stones by the players. No hints about group status, but open borders would be obvious.
- AI supported manual scoring: The current OGS approach. Players get AI feedback on the current game during the scoring phase. They might procede with the game and utilize the newly gained information. This system is good for people who like to play with AI support.
- The Undecisive approach: All of the above. Add another plethora of options, for the preferred scoring method, to the already option laden OGS game menu.
0
voters
3 Likes
I mean… I voted but the different options are presented in a very biased manner. 
5 Likes
By default: manual. If opponents can’t agree for 5 minutes in Live 19x19, forced full-auto.
4 Likes
I’m also for full autoscore when the players can’t agree on which stones are dead, but they don’t really want to continue playing.
Like when a game is resumed and both players pass straight away without playing any move on the board, finish it by full autoscore.
That way, manual scoring is the default, but if that doesn’t work out for one reason or another, there is a fair and easy way out of the game.
4 Likes
Yep, the wording seems a bit leading. I think the current approach is basically ok-ish, though if it was possible to tweak it to only decide on the most obvious cases, then I would like it even more. I would not ever call it “playing with ai assistance”.
2 Likes
Agreed. Full manual… but If they restart and can’t agree to move, full automatic.
2 Likes
My ideal, not in the the poll, is fully automatic scoring, but with the option to resume if you disagree. Does Fox really not allow resumption?
Fox does not allow it afaik. The game just ends and a pop up appears with the result. I personally never had an issue with the scoring. Everything happens automatically, instantly and correctly. No arguments over score, no scoring phase, no accepting scores, no score cheating, no time wasting, no menial labor, no CMs needed etc. etc.
There is even an option to force auto scoring during the game when an opponent keeps stalling at the end. That’s a cherry on top. Their system is impeccable.
2 Likes
No arguments over score, no scoring phase, no accepting scores, no score cheating, no time wasting, no menial labor, no CMs needed etc. etc.
That’s the positive side. The downside is that the score may not reflects the player’s understanding of the game (i.e. it may score as dead a group that both players believed was alive).
There is even an option to force auto scoring during the game when an opponent keeps stalling at the end. That’s a cherry on top.
We already have that too.
4 Likes
In practice, what would be the difference between that system and the poll option “AI supported manual scoring”? Improved accuracy of dead group premarking? Not being able to toggle group status (so premarking is definitive)? Or perhaps both?
1 Like