Wondering what the rank is for KatoGo
2 Likes
That’s a great question but unfortunately it is pretty hard to answer.
It depends in large parts to how much calculating power it’s allowed.
If you use Katago on your own computer with a user interface like Katrain or Sabaki, in the settings you can change how much you want it to think about each move, and that can have a huge impact on its strength, maybe from as low as amateur 4kyu to higher than professional 9dan.
If you’re talking about the game analysis on OGS I don’t know, I would guess it’s about amateur high dan, but hard to get a confirmation.
1 Like
KataGo using a relatively modern network and using the absolute minimum possible amount of computing power with that network - 1 visit - has in the past been able to maintain 8d on KGS when someone ran it there. So, if you disregard situations very specially engineered to be exploits or blindspots (like adversarial attacks would find) and consider just “general purpose” play, KataGo is always at least the very top amateur dan, and probably gets into superhuman rapidly once you have even a little bit of search.
People perceive it often to be weaker than that at low playouts providing analysis, but I think this comes down to a big mix of things.
-
The amount of compute power to accurately and non-noisily quantify the best move is often easily 5x-10x larger than the amount of compute required to usually just choose the best move or a good enough move.
-
Yet more compute needed if you also want accurate judgment of multiple alternative moves. When playing, you can easily avoid a bad move even when very wrong about how bad it is because you haven’t spent much time thinking about it, so long as you still are “sus” enough of the move out of pure instinct to not play it. Same for bots.
-
People’s expectations are, quite reasonably, very high when asking for analysis that they want to be able to trust, and higher than what it would take to merely play a given level, although they don’t always realize this. (I’ve seen this before when working with AI for other games too!)
- Perhaps some of this is due to people having higher standards of consistency for analysis than play. We reasonably want the analysis on every move to be accurate to a given level and will hone in on any point where it looks like interesting things are happening. And to almost never make a mistake of a given “level” requires being vastly stronger than that level.
- Perhaps some of this is that as soon as actual numbers get displayed, it’s easy to presume a level of precision, and so you need another big multiplier on compute power to actually reach the precision that people presume that numbers have.
- (Minor note: people usually interpret a number like -2 as some objective claim like “the bot says the player is behind by 2 points” but the interpretation should usually be “if you give the player 2 points for free, the bot is not sure which side it prefers”).
-
Bots at a given level have a hugely different strength/weakness balance than humans of a given level. A mid-pro-level bot will make many mistakes that even amateur dan players can perceive, because it’s compensated by places where it is massively beyond human pro level in certain areas of instinct and holistic direction of play. A bot’s overall “play” strength needs to be much higher before its weakest spots are also are beyond what people can usually perceive.
Not applicable to OGS, but also there’s the detail where Lizzie github has been dormant for a few years, so if you download Lizzie it comes packaged with a very old network that is still very strong in absolute terms but also much weaker than any modern nets, and a user might not realize this.
17 Likes
Ok, so as a 21k player at best I can’t expect to win many games with a 9 stone handicap.
with 9 handicap stones you can expect to win against 9 ranks stronger opponent
21k with 9 stones as black
What is supposed is
you have
50% chance to win a 12k
40% a 11k
a huge chance someone lower as 13k.
Almost no chance over 9k
If i understand well your first question, you have like no chance, even with handicap to beat katago until becoming a sdk (single digit kyu player, between 1k and 9k)
1 Like
many → any
The general point of the 2 posts above that you have no chance of beating KataGo as a 21k is correct, though it is worth mentioning the ranking system and handicaps are based on human play, and bots aren’t generally as good at giving handicap as humans. So although as a 4d human KataGo would destroy me in an even game, it’s possible that I could give 25 stones to @hughb and beat him as I am experienced in giving high handicaps to beginners and understand human psychology and can trick them into making typical human beginner mistakes, whereas KataGo might not be able to win at 25 stones (it does have the asymmetric playouts option, I’m not really up-to-date on how it performs at high handis, maybe @hexahedron can elaborate).
7 Likes