I think the 13k default rank is doing harm [Closed]

Feedback Suggestions
81

Am I the only one seeing a contradiction here? :smiley:

8 Likes
82

nope. fits perfectly you can never reach the horizon :slight_smile:

3 Likes
83

Has anybody confirmed that? Err, having some math background I don’t see how that could ever be a problem. Yes, it could cause some overall rank inflation or deflation, but since all the ranks are then projected onto the kyu/dan scale, the projection could dynamically fix that over time assuming the flux wouldn’t be too strong.

Wouldn’t it be easiest if someone literate in english (which, conveniently, rules me out) tried reaching out to professor Glickman to ask for his evaluation?

84

I mean… I could do it, but being a generally lazy person I’d probably delay it until the weekend, so maybe someone eager enough would like to beat me to it. How about a community project! Let’s write that e-mail together!

I’ll just keep editing the post as noone seem to be replying to my ideas and I don’t want to leave the impression I’m talking to myself.

Here’s an example implementation of glicko2 algorithm where the very first example uses custom ranks for initialisation.

So looks like we are down from ‘impossible’ to ‘not unthinkable’

85

Who are you going to give the wins to ?

As I vaguely understand it, the whole reason that we “have to” start players at 13k is so that the balance of points in the pool is not adversely affected. Just arbitraritly giving people losses, without giving corresponding wins, is surely mathematically the same as just starting them at low points in the first place.

Isn’t that what I did with the current proposal? :wink:

I said "“We want new players to start at a rating and against players according to the strength they bring here”, and “they have to have glicko 13k at the start” and proposed “therefore let’s have a qualifying rank that determines their get-a-game outcomes, based on the strength they bring here, while their glicko rank settles based on the outcome of those games”.

One aspect of the proposal that I made is that I took it as “given” that players have to start at 13k, because this has been put on the table as “mathemtically true”, and I am not in a position to refute that.

I think that if we are going to open that question up again, it has to be on the basis of informed input coming from the maths.

So there are two choices there that I can see: either solidly refute the statement that we have to start at 13k based on an argument that is informed by the maths, or debate proposals that accept that you have to start at 13k.

I took the latter path.

GaJ

86

Why not assume that players do not have to start at 1500 glicko points and sit on the simple conclusion until someone ‘refutes the statement based on an argument that is informed by the maths’ ?

EDIT: I meant :wink: Completely forgot the smiley. Just a friendly poke to see where we are really heading with the discussion. I’m a big fan of simple solutions. Start simple, if someone proves it doesn’t work, iterate. I do that at work all the time, and rarely ever we get into 2nd iteration.

87

Before all else, I’d see if the ratio “active accounts”/all accounts changed at all (that is, significantly) after the glicko was introduced. I doubt it. A more thorough investigation is possible, but probably not worth the hassle either.

1 Like
88

“Closing” a discussion with hardly falsifiable statement has a name :wink: https://rationalwiki.org/wiki/Escape_hatch

And it’s not that I disagree with you, I’d always prefer conservative approach before making investments in non-proven theory. But at the same time it does seem that the numbers of active games in the main page has not increased for years now, despite of multiple attempts of simplifying the system and whole Alpha Go craze. It does feel like something is pushing out newcomers and I feel it would be worthwhile to run some statistical analysis on the newcomer trajectories for last year or so.

89

As a beginner and new user, my opinion may be of interest. To get my first game I clicked on the “Normal” button and was surprised to be matched against a 10k player, so I cancelled quickly to spare them from a boring game. After that I just played a bot to lower my rank.

A qualifying or provisional rank seems like a good idea. This could be used instead of the rank derived from the rating until the rating deviation drops below a certain threshold. In this way new players should be matched against others of similar ability.

Providing good documentation would certainly help, but not everyone will read it before playing their first game. Requiring new users to play a bot or uneven games against humans to establish a rank is far from ideal.

2 Likes
90

If you require that all statements be falsifiable, you’re setting yourself up for disappointment. Especially when personal opinion is involved. My point is that

  1. I doubt there is a problem big enough to warrant messing with rank calculation for everyone.
  2. Except for a handful of people who seem a little disgruntled but actually have stuck around, there is little evidence to the contrary.
  3. It may very well be that this lack of disgruntled masses is due to survivor bias, therefore we could very well attempt to falsify my hypothesis that the “glicko 13k change” did not make matters worse than they were, as measured by newcomer dropout rate, approximated by the ratio of active/total accounts.
  4. My proposal has shortcomings, I ackowledge them, I could work out a more thorough proposal but that would imply more work than I’m getting paid for so I left the more thorough test to your imagination for now.

My opinion also doesn’t (and shouldn’t) keep anyone from posting their ideas for improvement. All I’m saying is - try to think of things that acknowledge how the glicko system works best and find ways to ameliorate those (hypothesized) effects you seem to find so detrimental, without messing with it.

91

I’m a new player (of about 2 days), and I agree the default rank of 13k is daunting. I think it would have been more off-putting if I hadn’t already seen this thread, so I knew to expect my rank to plummet immediately.

In particular it was a sad moment when my rank dropped so low that even Master Mantis started refusing to play me.

I’ve been thinking about how we could adjust this. At the risk of writing a wall of text, here’s my thought process, which I hope will make it easy to disagree precisely:

Problem

At the moment, weak new players find themselves thrown in with experienced mid-rank players. They inevitably see their rank go down quickly after their first few games, which they predictably lose.

Meanwhile, mid-rank players have to put up with games against much weaker new players.

Goal

We want experienced mid-rank players to get matched against other players who genuinely have a similar rank, so they have better games.

Weak new players should be start by playing against other new or weak players, so they don’t always lose and see their rank drop precipitously.

Strong new players should still be able to earn a good rank fairly quickly.

Constraints

Glicko-2 requires that a new player’s rating must be somewhere in the middle.

Assumptions

The rating uncertainty for new players can be any amount, assuming it’s the same for everyone.

After enough games, a player’s rating uncertainty becomes small.

Solution

  • For the purposes of matching players, use the lower bound of a player’s rating.

  • When displaying a player’s kyu/dan rank, use the lower bound of their rating.

  • A player’s actual rating is still calculated the same way.

  • New players start with a very large uncertainty, so that the lower bound of the start rating is about equal to the actual rating of the weakest experienced players.

Result

For example, if my actual rating is 1500 ± 600, the matching algorithm treats my rating as 900. (I don’t think the matching algorithm uses the uncertainty value, but if it does, it can treat my rating as 900 ± 600.)

I get matched against other players whose lower bound is around 900. My kyu rank is calculated from the lower bound of the rating, 900, so I appear as 29k and seem suitably unintimidating.

After playing a few games, my rating might end up as 1300 ± 300. Now the matching algorithm treats my rating as 1000, and I appear as 25k. My actual rating has gone down, but the uncertainty has gone down more, so the lower bound is higher. I appear to be making progress and don’t become disheartened!

If a new player is actually pretty good, their rating may quickly become 1700 ± 300. The matching algorithm treats them as 1400 and they appear as 15k.

For an experienced player with a small uncertainty, 2000 ± 100, the matching algorithm treats them as 1900, and they appear as 5k. This isn’t very different to now.

There’s a certain logic to this: the site can be (95%) confident that the player has genuinely achieved at least the rank it shows. It shows the humblest rank of which it can be confident for that player.

8 Likes
92

This is a serious quesiton that deserves a serious answer.

The answer is: if we assume that players do not have to start at 1500, then the solution to the list of problems is easy: simply ask players what their rank is, from a small set of well described choices, and set it accordingly. End of problem.

That is what we used to do, and only stopped doing because of the start-at-1500 requirement.

This doesn’t get much discussion because the solution is so obvious and easy. And because every time this comes up (and this is not the first thread on this topic) someone chimes in and says “you can’t do that because maths”.

So now we have two proposals, depending on maths:

  1. If maths allows, let people choose a starting rank

  2. If maths does not allow, let people chose a qualifying rank.

(There may be other proposals as well - I admit I’m focussed on these two, and I believe they are the two that have been most robustly discussed and concensus achieved, my opinion only

Edit: Ooo - while I typed this another proposal appeared, now right above this reply in the thread. I haven’t had a chance to read that one yet, just quickly and belatedly acknowledging it here!).

GaJ

1 Like
93

@Tofoon Now this is still meddling with the system, but I like your train of thought. I appreciate the idea to fiddle with the matching algorithm as opposed to the rating one.

Again, nothing would stop people from directly challenging stronger opponents but automatch would use “humble rating”. Amusing.

94

I appreciate the idea to fiddle with the matching algorithm as opposed to the rating one.

I agree. There is nothing wrong with the rating algorithm. The problem is using the rank before it has stabilised.

1 Like
95

What if a player’s first X matchmaking games were against bots at set rank intervals, to serve as placement matches?

96

I actually like Tofoon’s proposal.

One possible snag is that currently new players don’t get enough uncertainty for this to work.

Currently new players are 12.3 +/- 6.6. This means that the lowest bound is still 18.9 IE no-where near 25k.

I don’t understand why such a small uncertainty is given when we know for sure that a significant number of players with this rank will be outside this range.

So is there some other “maths” reason why a larger range can’t be applied?

If there isn’t such a reason, then Tofoon’s proposal looks good.

(But - would we all be sad to have a display rank suddenly drop by 1-2 kyu? … most people’s uncertainty is of that amount…)

97

I really like Tofoon’s idea. :slight_smile:

1 Like
98

The rating used for matchmaking doesn’t actually have to affect everyone’s overall (shown) rank, only that of provisional players. That is, after all, the entire point of the exercise.

99

Yeah, displaying the humble rank isn’t essential. I do think it would be a better user experience, because then new players would understand that they aren’t really playing a 13k.

100

Yeah - I think it would be an inseparable part of the proposal, honestly, for exactly that reason.

I guess we could recalibrate glicko->kyu so that someone who now is 15k +/- 2 would be 13k +/-2 and thus have 15k showing.

I think this is practical because most uncertainties I’ve seen of active players is around 1.5 to 2