I have noticed that both katago and leelazero have distorted win percentages when i upload games from other servers. For example, in the final move of a game i won by 20.5 with white, both give black 98%. Is this a bug? Should i think that the colors of the analysis are inverted? And don’t get me started, when there is a handicap game.
I’m not much help when it comes to AI analysis but the people who do know this stuff are probably going to ask for a link to the game in question. I think Katago has been acting oddly at the end with passes.
1 Like
and basically every game from IGS
2 Likes
Looks very odd: for Leela after first move win rate is 91% white. Katago starts 90% white even before first move
Sorry about that @dsatkas, very weird.
@anoek the review seems to be confused by the HA[0] property, without it it reviews just fine.
as a workaround for now, you can just manually open the .sgf in any text editor delete HA[0] from the file, save and upload that. The HA attribute is a way to define handicap and is not needed if there is none, so there is no real change to the record. But hopefully it should be fixed soon so you do not need to do that 
. Thanks for letting us know.
7 Likes
Yes, more info here, it will be adressed in the newer updates
Just as a heads up, LeelaZ is unable to cope with handicapped games. It is just the way it was designed and there is nothing we can do about that. KataGo on the other hand should be able to, but seeing your above link, it is possible the system does not yet interpret every .sgf file correctly, as every server seems to output them a bit differently. If you have some examples of handicapped games that were not reviewed correctly, it would be a great help if you could share those links. Thanks again, and pardon the inconvenience.
3 Likes
handicap game
for example, katago starts with 54% for black with 6.5 komi, while leela is fine
LeelaZero is only able to play with 7.5 Komi. KataGo takes Komi into account, so in this case KataGo is fine and LeelaZero is off by design.
so, wait a minute, in japanese rules 6.5 komi is good for black? I thought it was like chinese with 7.5 which is good for white. Am i missing something?
Also, no matter what you do, leela will always assume 7.5 komi?
Thinking about it:
at the moment (our) KataGo is able to play under various area scoring rules, but isn’t trained for territory scoring. Under area scoring a komi of 6.5 is (almost) equal to komi 5.5, so it’s possible that it overestimates blacks winrate, while LZ overestimates whites.
I don’t know the current state of the ideal komi discussion for territory scoring, but
territory komi of 6.5 is not the same as area komi of 7.5. Under area scoring all (most) games end with odd score difference, therefore a komi of 6.5 has the same effect as 5.5 under area scoring. Under territory scoring there isn’t such a bias. That’s the reason for chinese to increase komi to 7.5, while japanese only raised komi to 6.5.
Yes.
Most (modern?) AIs are exclusively trained for area scoring with 7.5 Komi. Their inner logic isn’t aware of scores, but only of win chances, which they learned to estimate under their training conditions (Komi 7.5 area scoring).
3 Likes