New Proposed Methodology For Go Server Rank Comparison
With the rise of a multitude of go platforms available for players to play online and the lack of a universal ranking system, the question on how go servers/association players compare to each other in terms of ‘player strength’ becomes more apparent. Several attempts have been made in past, but results are widely questioned by the userbase. The current research methodology hopes to address these concerns.
Why are the current studies questioned / flawed?
Past studies rely on either surveys or a naïve username based matching system to show the relationship between ranks on different servers.
Drawbacks of the survey based methodology:
- Often suffer from a small amount of responses. Especially troublesome for harder to attain ranks or more unique combinations (e.g. KGS / Tygem pairs) to have a sufficient base to get reliable rank relation estimations.
- Surveys take a long time to collect and rely heavily on people answering honestly.
Drawbacks of naïve username based matching:
- Presence of false positive in matching sets. This is especially true for game handles that contain common English words, e.g.: Dream, Master, Winner
- While it is common for people to use the same game handle on multiple servers, you’ll miss a lot of matches at the same time.
More importantly, next to issues addressed above, the current methodologies solely rely on the rank of the participant on the server at a given time and not the performance of player on that respective server. This is especially a concern on servers where people can select their initial rank. For example, a 5kyu OGS player, with 1000’s of games on OGS, decides to try the Fox Go Server, registers as a 5kyu and plays 10 games. In this scenario it is questionable to qualify the player as a 5kyu Fox player next to being a 5kyu OGS player.
New Methodology:
Basic idea: Calculate win percentages on game results between qualified (cross-server / association) players. If the win percentage is in balance (approx. between 48-52%) between cross server ranks and/or real world association ranks, they will be deemed roughly equal.
Qualified server player : Player has played more than 25 games in the last 3 months on a respective server in a specific rank (no rank up/downs).
Qualified association player: Player has played more than 10 OTB games in the past 12 months at a certain rank.
A player stays qualified for at least 3 months, and the qualification can be extended.
An example of the output I’m aiming for (fictional data):
With this methodology I place more weight on the actual games and the results than rank labels. I’m still debating to limit the amount of games per person to control for the results to be biased towards one or two persons in the set.
Prerequisites:
- Substantial amount of cross-server player game handles / association pins (currently collected approximately 3.000 from various sources). I’ve put in a lot of time to collect all of these and to make sure the game handles etc. belong to the same person.
- Game results for largest servers (collecting for Fox, IGS, OGS, KGS, Tygem, AGA and EGF results)
- Time
I will use the old methodologies as a reference point.
I’d like to hear everyone thoughts on the methodology!
