AI based score estimation and auto-scoring is now live

Yeah but here the actual rulesets come into play. Players can resume the game after both have passed if they cannot agree on what is dead and what is alive.

If they are using japanese or korean rules, the one who plays first after resuming the game is the opponent of the player who clicked “resume”.
If they’re using any other rulesets, the normal b/w/b/w is used, so the opponent of last pass gets to play next.

Since most games on ogs are played with japanese rules, usually you cant know who would play first until someone disagrees with the score and resumes the game. (well you can… its the player who would lose if they would accept the score…)

In @pdg137’s example here’s some screenshots assuming pdg didn’t realise they could break in and pass.

Black connects

White passes unlike the game

Black passes

I think the unsettled stones, namely G9, F8, F7, G7 and F6 should all be marked alive since they’re not pass alive, otherwise one runs into the problem of trying to distinguish “dead” stones in a territory from actually dead stones which assumes katago played on from the position.

It would mean that the players would have to manually mark F6 as dead, but that’s okay, that’s what you want, the players to agree on which stones are dead and not gain any new info from what katago thinks of the position.

One could argue maybe a beginner thinks G3 might be alive, but it’s likely they would’ve tried to save it before passing. It’s hard to think certain stones would be alive anyway…

That sounds awful. Some people want to finish the game with the result they agree upon which makes sense at their level.

If both players don’t know one can invade and get a seki in the corner that’s fine if they agree, it shouldn’t just be called a draw.

3 Likes

But if you do this you give the players a big hint that the position is unsettled. If both players pass, they are convinced the position is settled, right?

The problem with auto-score right now, is that it doesn’t mark stones as dead that should be dead. This results in a lot of badly scored games, especially in the DDK ranks. Not marking the stones is similar to the old-fashioned score estimator.

1 Like

At the end of the day either the auto-score provides absolutely no help and should be completely manual, or it does and it potentially gives something away.

I don’t see an alternative.

The majority in what though? If I have massive group that dies with a 3,5,6 point eyespace I might be in the majority locally so would it mark me alive? Should beginners then accept it as alive, assuming that the scoring phase is giving something away?

I do, though:

What’s wrong with my suggestion?

The majority in number of stones. Try to come up with an unsettled group that looks alive to weak players, where the “alive” group has less stones than the attacking stones. I can’t come up with a situation like that.

If you leave behind a bulky 5 without securing it, and your opponent passes without trying to capture it, my personal experience with ddk players reporting games, is that both opponents think it was alive already.

If white passed first and black passed after him, in chinese/AGA/NZ/Ing rules it would be whites turn next, so those b stones would/should be dead. But if the game has japanese or korean rules, and white after seeing the SE and realising what’s happening decides to resume the game, it would be blacks turn and b probably fix the shape.

If we make any kinda automation its important that it can apply the differences in rulesets.

2 Likes

Quite common if there’s fake eyes involved, especially with the cornergroups. Just a theoretical situation for an example
Screen Shot 2021-05-14 at 19.46.57

But here white would be probably regarded as ‘settled’ by the SE, right (depends a bit on the rest of the board)? So black has the majority, and if both players pass, probably both players think the black group is alive.

Also, even if the white group was involved, it’s kind of proving my point: black has more unsettled stones …

For the first one I’m not sure if you mean to depend on katago to decide that, which I’m not sure it’ll distinguish the reason for a point being unsettled (it can reach stones of both colors, or there’s a move to be played here). Or maybe one needs an additional pathfinding algorithm to figure out which points could reach alive stones of both colors and mark (all?) those points as dame?

The second is unsettled groups right? I think something involving a seki will do the job eg below

image

I think attacker and defender are vague concepts when a group can live in seki, but definitely Blacks one stone in the corner at S19 is “dead” but can live in seki with one move, and has much less stones than the surrounding group.

image

Maybe I’m misunderstanding what you’re saying, it’s not clear to me.

I don’t really know how to pick this apart. I need an unsettled group with less stones that the attacking ones? It has to look alive to weak players? Like should I think of a fully surrounded bent four or fully surrounded 6 point eyespace in the corner

image image

The “dead” stones can have hundreds of stones less than the surrounding ones?

So, I mean the following algorithm:

  • KataGo checks the position with White having next move
  • KataGo checks the position with Black having next move
  • Any discrepancy between these two is marked as “unsettled”, the rest is marked as “settled”.
  • Chains of unsettled area (open intersections + unsettled stones) are considered as a whole.
  • If a chain has empty intersections bordering on settled stones of both colours, this is a dame
  • Else, if the number of unsettled Black stones is larger than the number of unsettled White stones, mark the territory as Black
  • Else the number of unsettled White stones is larger than the number of unsettled Black stones, mark the territory as White
  • If the numbers are equal, leave the territory without mark (so like dame)

The Black stone has the majority in their area, since the white stones are unconditionally alive (either in seki or just in general) and thus would be counted as settled.

Still, this would be an example where the Black stone is marked as alive, while the players probably think it’s dead.

This is exactly the kind of thing where SE could give away that the group is killable or can become seki, to White. If both players pass with these shapes on the board, either neither of them knows there is something to do (likely) or they both forgot about it by accident (in which case they probably see it marked as Black in the scoring stage, but then can resume the game to play it out).

3 Likes

Finally, I don’t mean to claim that this catches all scoring problems, but I believe it will prevent the majority of games with an unsettled group at the scoring phase from being spoiled by the AI.

2 Likes

So with that seki example, it might give a hint still that maybe black can live here.

Similarly in this example

I don’t think it would be unreasonable that some players might think the black stones in the upper right are dead: they can’t partition the area into two eyes (if they see a ko it’s unwinnable). But of course they can throw in and create a Oshi-Tsubushi at Sensei's Library .

I think the algorithm will still say that black has more stones in the unsettled area so give it to Black, which if you have an idea how it works and that it’s katago might still tip you off to think again and try to live.

Then what about the bottom left corner and spaces that are big enough to try live in but it’s unclear When is 3-3 alive?

I’m guessing there’s some fractional point values being assigned that determines the size of the squares? Suppose it’s something like +1=black and -1=white and the values of territory go anywhere between +1 to -1 and 0 within some error tolerance is unsettled.

It’s not clear who would have the most stones in the “unsettled area” in such big areas of the board.

In any case my idea gives far more away most of the time, so it’s not a good idea to rely only on pass alive areas from katago. I still think it’s hard to design an intelligent score estimator that won’t give something away.

I have not yet read everything that has recently been discussed, but I just wanted to chime in on a few things.

If the unsettled position is worth enough to swing the outcome of the game, then the Japanese rules would actually declare such situations as “both players lose”, see Odd Cases 🤔 in the Japanese Rules - #2 by yebellz

@_KoBa’s post clarifies as to why this happens:

Due to this, if there is an unsettled position that would shift the outcome of the game, then neither player would want to resume, and the game is put into a sort of limbo state, since the players cannot otherwise agree how to settle the position.

Thus, Japanese rules declares such a situation as “both players lose” to resolve this limbo status.

6 Likes

Imagine if both players lost rating for that :smiley:

4 Likes

How does scoring work in real-life tournaments? Would it be normal for a random Dan to walk by, point out a weakness, and nullify the game? That’s what this feels like to me.

3 Likes

Players in real life tournaments are usually strong enough to agree on the status of each group and score the game without any fuss (except maybe in beginner tournaments).
And there is usually a referee to handle any disputes.

That was my expectation too. But are you implying that players in real-life tournaments are strong enough that they don’t leave any weaknesses on the board? I’m talking about a situation where there would be no dispute unless a stronger player points out the problem.

1 Like

Once more I see cheating being treated as “strong players problem”.
I guess the lower levels don’t really count? Everyone is focused on catching that elusive cheater who shouldn’t be a dan, methinks, not on preventing cheating per se…

2 Likes

Nah. I disagree and even encourage the weakest to join in these IRL tournaments.

I would rather say that players do their best to finish well their games but sometimes need a referee to help.

2 Likes