Hi!
Why all the bots say a rank but when I play the rank is another one? The Amaranthus bot says 25k, but when I play, its rank is 17k! Way beyond my league…
Regards
Ernesto
The rank in the name is an estimate the owner was aiming for. The other rank is a real ranking based on their ranked games.
1 Like
You can get a sense of the general rank by looking at the history over the hundreds of games the bot played - if you click “play computer” and then you choose a bot and go to “show custom settings” you have a little link next to the bot’s name - there you can see what level the bot has been averaging. For instance, Apaganthus is on average around 24k-ish, Amaranthus is more like 21k, Bergamot probably around 18k, etc. Hope that helps.
2 Likes
I don’t understand… If a bot is intended to be 25k and its rank is 18k, then what means that “25k”? If the bot wins the matches, and increases its rank, then it is playing better that it was intended, right?
There is only one real ranking which is the result of the games the bot is playing. You can see it in its profile with all the games it played.
The mention of a ranking in its profile description by its owner has no such value, just a word written by its owner.
1 Like
That’s exactly correct. That bot is playing better than intended or imagined by the person who wrote it.
If only we could do the same with people 
Ranks are relative so it’s impossible to say concretely a bot’s skill level is 25k. What 25k means could shift from time to time.
25k is the lowest available ranking on OGS, but if for example 10,000 complete beginners suddenly join OGS and play games, original OGS 25k players will likely beat them and rank up to say 20k.
Now 25k would mean a lower skill level, because original 25k players have ranked up.
So my guess is as ranking shifts the bot turns out to land somewhere at 17k instead, even if the creator meant for it to be “25k” level.
It may be a good idea though to display a bot’s current rank, instead of expected rank, to avoid confusing players?
1 Like
Actually, there’s more to this than I first realised:
You can see here at the green arrow that what is being displayed is something separate from the bot’s name (the name is truncated).
In the code, it looks totally like it’s trying to display the bot’s actual rank:
<span className="username-rank">
<span className="username">{bot.username}</span>
{!preferences.get("hide-ranks") && (
<span className="rank">
({rankString(bot.ranking || 0)})
</span>
)}
</span>
But the orange arrow is pointing at a number that is “wrong”. It’s wrong in the sense that when I look at Agapanthus, I see that it’s current rank is 24k.
This means that there’s something I don’t understand about what bot.ranking is, and there’s no obvious reason why it should not be displaying what we would want to see there.
I have some guesses about what this might be, but it definitely is worth more of a look.
5 Likes
It’s important to remember this bot’s full 5000 game history only goes back 6 days, and in that time it has been well below 25k (currently 32k but it’s been lower) and as high as 12k so it may well be its rank changes as fast as you can check it in one location and then the other.
Also that bots trying to be this weak are inherently unstable.
Also if there is any kind of caching in the display of the bots rank, that will also affect its accuracy more so than it would a normal player.
2 Likes
Yeah I wonder if it’s as simple as caching happening. Because look at my screenshot: it was saying Agapanthus was twenty FOUR k, when on Agapanthus page it was as your shot: 25k.
It’s not as simple as “oh, the rank changes quickly” … it’s been quite some time since Agapanthus was 17k.
This being said NOTHING CHANGED about the bot itself
It is still the same bot doing the same strategies. The only thing that changed is our short-term guess at it’s rank.
If you eyeball it’s longterm performance, it’s clearly a TPK, not far from 25k
It just has massive runs in either direction. This is the price we pay for having “responsive ranking” for humans: bots (who play far more) appear volatile.
4 Likes
Quite some number of games, but time wise it was only yesterday… totally plausible to still be a cached value.
1 Like
I wonder whether bots are so volatile we should display a rank range for them rather than their current rank. Would it be too tricky to pull min and max rank from their most recent 5k games and display that? 
at least that way the cache should be more stable, even over a day or two…
1 Like
I guess my main point is that the number we display “ranking” so far appears to be appropriate, for whatever reason.
It’d be interesting if anyone could find an example where the “ranking” displayed on the Play page doesn’t match the average for that bot.
I personally don’t know (yet) where we are getting that number from, but everything I’ve seen so far tells me it’s not bad.
The main message is: don’t panic if the bot seems to be way different rated right now, it was likely just on a winning streak. It’s still the same old TPK bot that it was before.
2 Likes
Seems like minutes after I took this screenshot now I see noob_bot shows up as 2k for some reason.
Side question, why can’t I challenge a 2k bot when I’m 6k? I was always able to challenge it when I was 4-5k.
Did it deem me not worthy after losing to it for a few times? 
1 Like
If the ranking is so volatile, why isn’t it computed as a windowed average? That is the standard way to deal with rapidly changing data, IMO.
(Almost every DC power supply averages chopped rectified AC into a steady-state DC level. Electronic devices rely on averaging.)
The history of each bot already holds this information and each player can chose its own criteria (like how far to watch back, etc…)
Even if the ranking would be so volatile, I predict some hard time to define what should be the amplitude of a window.
The rating system already is effectively a moving window average.
The thing is that it’s tuned for human players to experience the right compromise between “responsive to change” (so we aren’t stuck on a rank while we’re improving) and “not unstable”.
It’s the same system applied to the bots … but they play a lot more.
That said, I don’t find this explanation fully satisfying, because it’s still not clear to me why a bot would experience a large run in different directions. It’s the same bot each time, the code doesn’t change, the way it plays doesn’t change … yet it’s appearance to our ranking system does.
Is this just statistical fluctuation? 
2 Likes