Ultimately, if we really wanted to create a rating system that could assign you a rank based on how you played and not by your wins and losses. We could probably take the KataGo trained networks, and fine-tune it to predict ranks based on players’ moves. This system would have the same benefits as stated above, but should be immune to the caveats too.
A contributor to katrain has been working on something similar, but mostly going by policy values.
There’s probably something interesting in these threads and repo, including datasets with a few thousand analyzed games.
It’s a hard problem though, you can get a reasonable average, but also pretty big errors. I would not recommend ever basing a ranking system on this. Our goal is to identify whether someone should focus on improving their opening/mid game/end game more.
I would really hate if the whole sites ratings would be determined by opinions of one bot. So instead of playing good moves to win and get into higher ranks, you would have to play like the bot does. Would playing traditional fusekis/josekis from 1900’s automatically make you ddk in katas eyes?
Also good to keep in mind, that losing points can be a winning tactic in some situations after your opponent has done a major screw-up. If you’re up by 40, you can throw 20 points on slow and risk-free yose on purpose, and it doesnt make you any worse player (on the contrarily, losing points for avoiding risks is usually a sign of a strong player)
Absolutely agree here; I think that would be a misapplication of this cool technology. I think the greater possibility for this sort of thing is for cheating detection, as it is used in chess. Of course, the chess community has developed this concept far better than we have, but that’s why we’re having discussions like this, and why some teams are working on this problem. I think that a time will come in the near future when we can use AI to detect cheating with a high degree of accuracy, as it is done in chess.