This dude/chick with the indistinct username seems to be making a lot of sense to me. Their attitude may be a little hotheaded, but similar concerns about that ranking range have been voiced before, and their theory as to why we have this issue (and at what fulcrum fixing should happen) seems plausible to me. Id say not to dismiss them out of hand just because theyre a little too passionate.
4 Likes
Actual Glicko2 wouldnât fix the problem with rank instability around 13k. Glicko2, as described in the original paper, starts everyone at the same rating, and the exact same thing would happen even under a more âvanillaâ implementation of the rating system. The same thing happens at whatever the âentry levelâ rating is in most rating systems, if all players are started at the same level. Iâve seen it on chess servers which use Elo. Players at the entry level end up playing a wide range of opponents, because the incoming players have a wide range of skills.
The problem is that new â13kâ players get automatched against 13k players, rather than playing matches at whatever their actual skill level is, and has nothing to do with the ratings system used, or how OGS decided to implement it. There have been dozens of pages of discussion about how to fix this, but to the best of my knowledge there has been no implementation of an effective solution that would allow for more accurate integration of new players into the OGS player pool. If youâd like to code one up, OGS is open source. Iâm sure people would be happy to look at your solution, and integrate it if it makes sense.
On a different note, if youâre openly hostile towards the OGS admin team, theyâll likely be hostile right back. If you want to actually have a discussion, youâll likely get a better response if youâre courteous in your approach.
7 Likes
Thank you for the diplomatic term, my original description was rather less courteous, your term is better. And thanks to @ckersch88 for actually trying to keep the discussion to the point.
I however would like to adress all the claims in the thread just to be thorough.
Let me start by saying that I understand that for some the experience of having to play several people who are actually complete beginners is not perfect. Unfortunately the system will never be perfect to everyone. To borrow a rather overused quote âthe needs of the many outweigh the needs of the fewâ and at the moment this is our best solution. If you have a better one we would be happy to hear and discuss it, as we are honestly happy for every well worded feedback and suggestions for improvement. It is understandable that you want the best user experience for yourself, but for us the job is to make it the most friendly for the majority. At the moment this is the best weâve got. And while you were very vocally against that idea, I did not read a better suggestion from your part. And since I personally know several players who started at the bottom and worked through the 13k without some major problems I would dare to say that the issue is not as hot as you seem to describe it, and maybe I am missing what you are saying, but to be perfectly honest before you resigned all your games the 13k rank seemed to be more or less accurate, or do you believe you should have been ranked higher? Not saying there cannot be a problem, but even if there is one, I strongly doubt it justifies the approach you have chosen.
Before the change to Glicko2 based rating the people had the ability to chose their starting rank and unfortunately the experience was not a great one. Most of beginners who were not familiar with the go learning curve did not know what to choose and then opted for the âmiddleâ approach anyway (because letâs face, we all like to think that we are more clever than the rest, and are used to choosing ânormalâ difficulty for most of other games) and it gave a great opportunity to people who like to cause trouble to create a new account with any rank they want to. So AFAIK the change to singular entry point was conscious and then the middle of the road 13k seems to make sense to me.
Well, you yourself found the paragraph where our developer clearly states that the Glicko was modified and his reasons for it. The rest of the cases is probably mostly my fault and you are right. I will try to remember to add âbased onâ from now on. But honestly I would find it more dishonest to claim it was our system, since it is mostly based on another personâs work.
Well no, we are absolutely not obligated to share anything. Not sure why you would think that. We WANT to share what we can and we try to share and discuss the issues and look for soltutions together, but it is our choice and belief, NOT an obligation.
Since you seem to value honesty, let me be honest with you. You did NOT offer help, you started by basically insluting our developer, said the system is bad without having any understanding how the system actually works, called us liars and overall behaved quite aggressively. I do not think you could have seriously expected to be allowed access into our database or indeed anything similar after you chose such an aggressive tone for what seems to be a rather (letâs face it) globally unimportant issue (the rank is still just a number), and thus have to question the honesty of the âofferâ.
We as a community and indeed our developer are happy for every honest feedback and offer to help. But you have to realize that he cannot respond to every question there is, especially one that would require such an extensive work and documentation for a request by a single person. Thatâs why the forums exist where we first discuss the issue and only then call it to the attention of the man who has work enough as it is.
So to sum this up - if you actually honestly care about the community and want to help improve the site, please try to calmly and politely describe your issues and what you think could be done to improve them. You may have to be patient with me. But if your claims make sense I will be happy to personally relay them onto our developer. If however you would rather keep up the condescending nature of your posts and keep namecalling and insulting people then you are just not welcome here. These forums serve as place to discuss ideas, not other members of OGS.
7 Likes
At last, a statement of the problem.
I can see that thereâs something worth discussing here.
I wonder if we could have a new thread for it, separate from this one that is fully of mis-informed mis-directed invective against an imaginary âtheyâ.
2 Likes
Kinda just starting to sound like tygem though, Play 20 games and you either go up 1/2 down 1/2 or stay the same. Depending on the wins/loses
Great post, @AdamR. Itâs funny how this thread is morphing into the old 13k debate (Iâm not linking it, yâall know where it is, right?), since the 13k entry-point seems to be the only concrete point of contention here. Many here know that I am not in favor of the 13k entry, but I do appreciate the arguments for it (which I wonât reiterate). One argument favoring the 13k entry is worth emphasizing, however. As you noted, the single entry point makes life much harder for airbaggers, sandbaggers, and trolls. This is a non-mathematical consideration.
The primary problems with 13k entry are also non-mathematical in nature. Many SDKs resign when they find they are playing a 13k beginner, thus inflating the beginnerâs rank. We discussed this in another thread; my point here is that this is a behavioral issue, not an issue about Glicko or non-Glicko. Similarly 13k newbies have a problem getting proper auto-matches until they lose a bunch of games, which may discourage them. Again, this is a behavioral issue, not a mathematical issue. Most players from about mid-DDK and stronger seem to get their games from querying the chat, or challenging friends and acquaintances, rather than from auto-match, a strategy that would help newbies find good matches quicker.
Per the foregoing, I donât see that the 13k entry is a problem in terms of the âratings period,â which is what this thread is supposed to be about. Most newbies will drop quickly, though some will rank up due to opponents resigning rather than playing them. Players who rise to the so-called eddy at 13k will rank up undeservedly a little bit when encountering newbies, but get knocked down again quickly. If such players are indeed getting stronger, those encounters wonât matter, because they will defeat some slightly stronger-ranked players and rank up legitimately. In other words, I donât see a serious problem for those players. Like @crocrobot said above, itâs hard to see where there is a problem specifically related to the ratings period.
1 Like
I was thinking about mentioning something similar after the last response to me, but decided against it. So I am slightly pleased someone else brought this up.
I donât think thatâs true.
There is another thread discussing instability of our ratings, irrespective of the 13k thing, and the at-last-clearly -stated problem likewise appears applicable across the board. It would be a mistake to entangle those.
Letâs leave the 13k problem in itâs own thread.
I agree, I have no desire to restart the 13k debate. My point here was that its main problems are behavioral and, in my opinion, not related to Glicko or non-Glicko, which was directly relevant to this debate.
As for general rank instability, I have contributed to that thread too. It seems to me that the general instability, such as it is (I have no idea whether it is really serious or not), is probably caused by four factors, listed here in no particular order: (1) over-ranked bots; (2) the mixing of formats, timings, and board sizes in the overall rank (note, I am not criticizing this); (3) the timeout rule in correspondence games; and (4) the 13k entry point. None of these seems to be related to Glicko, non-Glicko, or the ratings period.
1 Like
But, to be fair, the OP has added a new one to your list. Specifically, it is asserted that continual update of our ranking instead of in periodic batches contributes to instabliity.
However, in typing this, I have just realised that this surely is not the case, or at least only âperceivedâ.
Because while we do get our rating provisionally updated on a per-game basis, actually each âperiodâ the provisional is replaced by the batched evaluation as per Glicko (as I understand it).
2 Likes
Actually I think it works pretty well, compared with some other rating systems like the one IGS (Pandanet) uses.
1 Like
This topic was automatically closed 91 days after the last reply. New replies are no longer allowed.