Why u quoting me? You could quote you 
of user that already has rank
I interpret those issues more as bug fixes, tweaks and finetuning than a rating system overhaul.
1 Like
Are there any properties of the system we would jeopardize by doing ad-hoc fixes (cutoffs, counterweights)?
I do think itās cool we can update the rank from these extreme games, but yeah, linear loss function sounds like it ought to be better.
Clicking on a few of the games, it looks like the user was using auto-handicap. In that case, once the user was winning more than 50% of the games, Iād expect the rank difference between the user and the bot to stabilize. Instead the userās rank kept dropping (increasing rank difference) even once they started to win more than 50%.
For example, in the last couple of games the user has a handicap rank difference of 16, and the observed rank difference between user and bot is 16. This should result in symmetry between the increase/decrease of the userās rank after a win/loss. But thatās not what happens. A loss triggers a bigger change than a win does.
I donāt have easy access to a computer right now to look deeper. But I wonder if there is a bug causing this asymmetry.
Speculatively, maybe the effective handicap for the ratings update is somehow capped at 9 after a win, but not capped at all after a loss.
Or, there could be other explanations.
I wonder what the rating calculator would say for this series of games, plugging in the āhandicap rank differenceā from game info for the handicap.
4 Likes
I agree in principle with @Uberdude, but want to note that extreme or strange handicaps have more chance of hitting the case where the system evaluates the effective rank difference as 50:50 win rate, but for a particular pair of players the win rate is 9:1. Maybe one of the players has a lot of experience with this kind of handicap and the other has none. (This is why I think for rating v6 it might be useful to increase the uncertainty for larger (or stranger) handicapsā¦ effectively decreasing the increase/decrease after a win/loss.)
Regardless, Iād expect a player doing auto-handicap against a bot to have their rank stabilize to a place where they are winning 50% of the games. If thatās not happening then thereās something a bit funny going on with the ratings updates. Maybe they just need to play a bunch more games for it to stabilize, but in the case cited above the final page of games/ratings looks a bit off to me.
1 Like