I have just played a few games against a bot rated as 15k (Master Mantis). This bot is quite a weak player, and I beat it easily a few days ago. Although it is just as weak now as it was a few days ago, this bot has now increased its score to 9k by winning hundreds of games against even weaker (25k-20k) human players. Its score is therefore hugely inflated. Would it be possible to tweak the algorithm to prevent a bot (or an unscrupulous human player with far too much time in their hands) from racking up a huge positive score simply by avoiding playing stronger players and repeatedly beating newbies?
I was just thinking about posting about this. Master Mantis should be at around 20k level, but is currently 8k. A few things that I noticed that is causing the inflation:
- As OP mentioned, it has tons of winning streaks against ~25k players. I think I see somewhere that the new rating algorihm has the problem that, when you get winning/losing streaks, even though the opponents are way under/above your level, you still get significantly remoted/demoted.
- It played a lot of newly registed players, who start at 12k. There are newly registed players whose level has been stablelized (the uncertainty value becomes low) at 12k by only playing with this bot.
From 18.6kyu on Oct 2nd to 9.5kyu on Oct 3rd ! I think it’s fair to say that something is amiss. Perhaps people should only be allowed a certain number of Ranked games against bots per week?
I wonder if this ties in with https://forums.online-go.com/t/i-think-the-13k-default-rank-is-doing-harm
Master Mantis got its rank inflated again, this time it’s even above stronger bots such as Kugutsu and GnuGo.