Ranking system is BROKEN

[shinuito]

@Eugene

also find it interesting that this little exchange/conversation suggests that we’d rather spend development resources trying to minimize humans-who-might-be-using-AI than bypassing humans-who-might-be-using-ai and then developing new features and fixing old bugs…

It’s the same reason you don’t use a sledgehammer to put in a screw. It might work, but you won’t necessarily be happy with the results.

People on average like playing ranked games against other people, that’s why we go to tournaments in the first place. People want to feel they’re improving vs other people, not vs bots which they may or may not be able to exploit (remember even Katago has blindspots like the adversarial attacks, and we had a user that could consistently exploit it).

I also don’t think it’s equally likely for your opponent to be using AI at all ranks, I would assume you have a far higher chance once you get closer to the dan ranks than in the lower half of the kyus for instance.

So that particular argument for scrapping human ranked games doesn’t make much sense for most of the kyu ranks I would say.

---

We also have some real examples of “ranking vs a bot” in the form of the HJJ school and the feedback was that it wasn’t good at all.

@BuriBuri

The so called smooth progression ended abruptly when I was asked to win a few games against the AI in order to finally get into the next level.which I assumed I had earned.

In fact the level the AI is -calibrated- ? is so high relative to my ability I simply get beaten to death in every game. I play reasonably well for my level in quite a lot of the games, but the AI unerringly out thinks and out counts me while demonstrating an infallible ability to play random looking moves with a directional sense that allows it to acquire huge areas of territory with a minimum number of stones.

In fact, I have lost 97 games against it so far Since there is no access to new lectures and puzzles while one is working though a level there is essentially no smooth progression at all. I doubt if I am learning much although I am religiously playing through the rather limited review pointers which range from very useful to something the AI understand but I cannot fathom.

This would be in the context of a course where you do puzzles before eventually playing vs AI to rank up.

I don’t necessarily believe even the human-like Katago nets or nerfed thinking ones, or ones just randomly being forced to make mistakes will necessarily be good to fine grain humans into individual Dan and kyu levels like we have now.

But I also would expect it to be not very fulfilling and probably people will cheat vs AI to get to the rank they want, for the same reason people always cheat: they have something they think they deserve and they do whatever to get it. Similarly players do cheat in unranked games, so it doesn’t even solve the idea of people cheating vs humans. It only solves having to annul those games.