Bug in KataGo 9x9

Hey,

when using the new feature of AI analysis for 9x9 games, passing apparently screws up the winning probability graph. It shows something like 100% for the player whose turn it is after a player passed. Also, there are no moves printed on the board. Take a look yourself at this example: https://online-go.com/game/view/20952322

1 Like

@anoek I think itā€™s not just 9x9 - the passes in other games Iā€™ve looked at cause this as well.

eg: https://online-go.com/game/20534247

1 Like

Thanks Iā€™ll be working on that. Pretty sure those values are just what the engine is giving me, if so I canā€™t really fix that pesay, but I can fix the issue where that causes it to be a ā€œkey moveā€

7 Likes

Maybe terminate the analysis at the first passā€¦

That wouldnā€™t work when there is resumed play after passing though

@BHydden you beat me :wink:

Hm, then if one player passes out of ā€˜inexperienceā€™ and the other still plays a move, the analysis is incomplete. I think this must be a bug in KataGo that handles passing incorrectly.

If player A passes, and B would win this way, KataGo should output passing as best possible move for B, shouldnā€™t it? Maybe it does this already? As of now there is no UI to indicate this, is there?

1 Like

Clearly thatā€™s the case - the question is whether itā€™s an OK price to pay.

I guess another approach on the same thing is to simply not show the last two moves ā€¦ unless the game ended in resign.

Nothing is simple :smiley:

Itā€™s not exactly a bug. Like other ā€œzeroā€ bots, KataGo basically uses Tromp-Taylor rules (https://senseis.xmp.net/?TrompTaylorRules), which are the simplest possible form of rules to implement. Under these rules, you get only score for the space you surround that has no opponent stones in it. So if there are ā€œdeadā€ stones, you have to actually prove them dead by capturing them.

If you end the game without doing so, then technically under Tromp-Taylor, that area is not your territory yet, and passing would indeed be a fatal mistake (much like it might be a mistake under human rules to pass when your territory has an unfinished border, regardless of whether you would be winning if the border was finished). So itā€™s a little less of a ā€œbugā€ and more like ā€œmismatch between computer rules vs human rules, computer is analyzing correctly under one particular ruleset but that ruleset is the wrong one for human analysisā€.

@anoek For now, you can fix this just by skipping analysis on any move that follows an opponentā€™s pass. You can resume analysis on any moves after that pass if they are not pass moves, because on any moves that are not immediately following a pass, there should be no danger of the game ending.

Iā€™ll add a hack in the next release of KataGo where you can configure it to assume that passes cannot end the game for the purpose of analysis, which should address the rules mismatch here.

4 Likes

Iā€™m not aware of any rule that closes territory borders in scoring phase. Humans have the tendency to not take the rules to the word, but handle them rather loosely.

Just wanted to add that this ā€œpassing bugā€ also occurs at my local KataGo at 19x19. Even on finished boards, sometimes, not always, at the very end when a pass occurs, the graph swings from black to white or vice versa, only to swing back in the next move. And if you run the analyis at that point, it will give the correct result.

Yeah - I find this very weird. I wish we could sort it out - it looks silly on our AI analysis graphs :s

1 Like

I didnā€™t know this was still a thing! :open_mouth: thought it was resolved age agoā€¦ :confused:

I have the latest version of Kata and it still does that. Quite often, actually, but not always. Havenā€™t found the reason yet, but werenā€™t really looking too hard to find it eitherā€¦

1 Like

Itā€™s still a thing - totally messes up the idea of using the AI analysis to see who really won, because the darn thing flips from side to side with the passes.

No it isnā€™t? At least in my gamesā€¦ Am I missing something?

1 Like

Thatā€™s sucky to hearā€¦ Iā€™ve not seen it happen in months, but Iā€™ll take your word for it

Do we have a recent example? Itā€™s not uncommon for players to overlook a L&D which may cause this when passing.

1 Like

If you have a sgf for which it happens on the newest KataGo version ( 2020-04-03T22:00:00Z v1.3.5) on your own installation, please submit it to the KataGo issue tracker. Maybe they have an idea why this happens. At least they should know it happens, so they can fix it some day.

Itā€™s usefull to check first if you havenā€™t overlooked a L&D.

2 Likes

Will do. Not sure I want to recheck all my recent gamesā€¦ there were a handful out of a whole lot. Iā€™ll upgrade to 1.3.5 today (mine is 1.3.3 so not actually the very latest) and see if this will happen again.

4 Likes

This is the most recent example I remember seeing the ā€œalternating 99% winnerā€ on pass.

It didnā€™t leap out to me that there is an L&D although I admit I didnā€™t look carefully for one.

1 Like