update, we have more than 100 entries now. Let’s start with how to apply the 2018 analysis program with our new survey data format.
It needs jupyter notebook to run right? Is anyone familiar with the technical aspect that can help? The code needs to be modified a bit for the new datasheet I believe. And the GoQuest answer of rank and Elo needs to be separated. And some outliers might need to be manually picked out.
I have to ask, what has actually changed since 2018 that would lead us to get different results?
Because otherwise there seems to be little point in this survey…
(I did fill it in though)
In '18, it had been ages since the last survey and OGS had changed over to Glicko. So we expected some shift.
I think ranks vs strength are drifting up and down quite a bit (can be 1 rank/year, at least for KGS)…
I’ve heard a lot of players from foxwq complaining the dan rank getting much tougher recently since more and more bots of various levels occupying the 9d even push each other down to 8d, resulted in pushing the old human 9d down to 8d or 7d, and the snowball effect pushing every human all the way to low dan.
But some other platforms don’t have this effect where not many bots existed on them or already have fairly stabled bot ranking management. Some with more than 9d ranking platform like IGS have the opposite effect due to fewer bots, and plenty of old 9d gets promoted to 10d (1p+) slowly where high rank players rarely got knocked down.
And with easier and easier access to AI evaluation tools, and more and more free teaching materials and apps help them practice constantly, ddk who are willing to learn ranked up faster and faster to sdk, and increase the total sdk players in almost all the servers, and the ddk players tend to be really bad and fewer in numbers.
Offline ranks should still remain pretty consistent in their ranks within their communities in my opinion, But perhaps when online servers start to get closer and closer in rank difference between each other, online rank might be more reliable in reflecting players’ actual strength in the future (or even now). This is a good reason why I want to find more players with offline ranks and compare them to their online ranking this time.
Actually offline ranks suffer from the opposite problem; those are stable but not consistent over time. Ranks won’t fall back, they only go up or remain where they are. 9P’s are 9P’s for the rest of their life, and the same applies are the down into kyus.
You can stop playing for years, get rusty, and forget half of the things you knew. And when you start playing again, you can lose every tournament game without your rank dropping.
Or maybe someone had some good tournament sometime in the past, or got promoted some other way, or something.
On EGF, players Elo might drop hundreds of points due losing a lot of games, but that does not pull the rank down with it. It is indeed quite common to see 2-3 stone separation on egf tourneys between what someones rank is, and where their elo rating justifies them them to be. And out these, ranks (or macmahon points based on ranks) is used for pairings and handicaps, while elo rating changes based on games outcome. I believe this same happens on other offline communities/rank systems too.
Are you saying you literally cannot lose rank in EGF?
Offline communities are aware of this pro title issue (and for a good reason to keep it as it is). Even though we respect these old 9ps for their past achievement, but competitors and the old 9ps themselves all know their reading and counting are deteriorating as they age (as early as in their 30s). They even acknowledge this and usually pair these old 9ds to compete against their own “league” for the most part. After certain points, the 9p title starts to be just an honorary title, and would usually choose to couch with teaching jobs at that point, and just retire (even though old players are not able to read lighting fast in minutes for each move under pressure at modern competitions, they are still very good at finding divine moves giving them enough time to read slowly as commentators or tutors).
The reason for keeping old 9ps their titles are also more or less akin to retirement insurance or long term financial security, to make sure pros to not worry about when they get older they won’t be recognized, and can still be rotated in competitions for attending compensations (usually still enough to make a living, also pro games don’t have handicap or different komi, just a game with what ever rules they use). The official pro titles don’t reflect their current strength but mostly their highest achievement in their lives (in many countries level to higher pro rank is governed by how many games or titles you’ve won accumulated, not win rate). And some might say pro only has three levels, the bad pro, the good pro, and really top pro.
It is another different story for armature dan and kyu though, and that also differ wildly in different communities. (usually depend on how diplomas are issued and their verification process)
Now i’m curious how other amateur systems work, i’m only familiar with EGF, where we have separate things for declared ranks and GoR progression (Go Rating system).
If you search european go database for active (egf ranked) players sorted by their gor’s (or by egf placement, since thats based on gor’s) and start scrolling down, you can start finding some oddities, like:
Few 1d’s within top500, best one being #325 currently https://www.europeangodatabase.eu/EGD/Player_Card.php?&key=17586624
Strongest kyu ever! #837 being 1k from the Netherlands! https://www.europeangodatabase.eu/EGD/Player_Card.php?&key=18425968
And on the other side of the coin we we have a 4d who is just barely out of top1000 as #1003! https://www.europeangodatabase.eu/EGD/Player_Card.php?&key=10386981
I am myself egf 5k, but i can see 7-8k’s ahead me on our rating, just as i can see 2-3k’s below me.
I get the sense that this was different in the 19th century, probably because players 1) hit their peak later and 2) retired (or died) earlier.
In the typical period of that century, there are two or three 8ds, of which one is recognised as stronger (the “ninth rank” was Meijin, which was abolished after Jowa’s scandal.) Then there are another two or three 7ds, who are also very strong. There were some players who got strong very fast though, like Shusaku – when he was 5d, he was probably playing at a 7d level.
To understand what was like in the 19th-century ranking, we need to understand the origin of the ranking and the background that lead to that. And some of them are still affecting us today.
First, the title 碁所 Godokoro (equivalent to the leader of the modern national Go association) is an official title for an office that governs all manners related to Go, and this job title has “salary” (in feudal Japan would make them small lords that own certain “land” to support this “salary”). Where Meijin is the description of the absolutely best Go player. And the job title Godokoro can only be held by a Meijin (名人, literally means a very famous and good person).
The definition of “absolutely best” in the era before any ranking system, before komi, means someone can always play any opponent as white (定先 josen) and win. And in an era before komi, the result of who is better is usually determined by a set of competitions (consist of many games of even number usually at ten 十番碁). And since a game might last a day to days, 10 games might last weeks even months, so direct competitions of top players happened not very frequent. This custom existed before Godokoro was made an official title,(in the 17th century) and its root can be traced back to ancient China.
The reason for even numbers of games per set is due to before komi, the way to ensure “fair competition” is to play several games and each one takes turns to play as black or white ( TagaiSen 互先), and if the result is when one side has more winning games above 4 in a set of 10 (that is 7 to 3, 6 to 2, 5 to 1, or 4 to 0, if one side wins more than 4 leads there is no need to play out the rest), he will be “superior” to the opponent and demotes your opponent. The next time the two players play against each other it will be 定先 josen, to compensate their strength difference. (if no clear winner after ten games 6/4 or 5/5, it means they are roughly equal in strength, this is the most basic form of win rate). The handicap stones can keep growing to 2 stones, 3 stones, etc (https://senseis.xmp.net/?MatchHandicapSystem). This is the basic form of dan system and also came from ancient China.
The interesting thing about this “raw form” of competing with each other is that there is no global consensus of how strong you are in a list of rankings. Every player has a list of players who he had played with, and what type of Teaiwari (手合割, TagaiSen, josen, or more handicaps) they should use to play each other. Player-A can beat Player-B, and Player-B can beat Player-C, but it doesn’t mean Player-A can use this to establish superiority over Player-C (and realistically Player-C can still beat the Player-A if their actual strength is close). Two players have to start with TagaiSen if they never met before in order to establish who is stronger. No one knew if there are some kinds of rock-paper-scissor type of strategies that can be used to counter each other.
Things always get complicated when money/salary is involved. In federal Japan in the 16th century, most Daimyo (federal lords) are Go lovers, especially Shogun (head of the federal lords, the de facto ruler of Japan), who held a national competition in 1588 and invited all the top Go players (上手 jouzu, literally means top players) in Japan to participate. And the winner Honinbo Sansa who was a monk and the Go teacher of the previous federal lord (who described him as Meijin) won it, and his temple was granted a hefty reward (this eventually led to the creation of the Godokoro job). During Sansa’s time, many of the Japanese Go rules and the system of competitions started to be established, and since Sansa was so strong, those who can withstand to play with him at josen became the first group of recognized jouzu, a group of dozens strong enough players who now became the measuring tools for everyone else.
In 1612, the Shogun established a series of “schools”, and picked top players (including Sansa) to established Go Houses granted with feudal hereditary titles and gave them annual funds to promote more players and trained them (iemoto system). Four later became the Four great Go Houses in the Edo period Japan (17th to 19th century). The house of Honinbo (Sansa’s house) had the most fund out of everyone and trained many good players over the years. The 4th leader of the Honinbo house Honinbo Dosaku had many students and he was a master and Miejin himself (one of the three 棋聖 Kisei of Edo period, the grandmaster of grandmaster). He started to issue diplomas of Go to the young lower dan players (1d to 4d) in the house, measured by his strong students. He also held the office of Godokoro, and he promoted the official diplomas to everyone in all the houses. And he devised a more refined way to determine strength, start with good players who can play josen with his top students - the high dan (5d in today’s rank), where six of the strongest students are the jouzu (7d). Dosaku himself as the Meijin is the measuring standard (9d in today’s rank) and can play josen with jouzu. At the same time, the rank between the 5d and 7d were established as who can play with 7d in sen-ai-sen. that is playing a sequence of two games in black and one game in white out of 3 (black white black, black white black, and so on). And this pattern goes on toward weaker and weaker players up to a difference of 4 ranks (1 rank difference is sen-ai-sen, 2 rank difference is josen, 3 rank difference is sen-ni - one game plays white followed by one 2-handicap game, 4 rank difference is jo-ni, which always play 2-handicap games, essentially each rank is 1/2 stone stronger than the next). Pro players start at 5d, those below them are mostly still students in the houses and not pros (1d to 4d didn’t get to have a salary so to speak). At the time, no one can play with Dosaku more than josen, so the hereditary title’s future successor (usually the top student or the head of each house) should be the second strongest to Meijin, became the first unofficial 8d (just referred to as those between jouzu and Meijin, later on was called half Meijin or Jun-Meijin, there were only 16 of them in the whole Edo period). And the title of gaining the title of Meijin was also established with an annual royal game played in the castle in front of the Shogun, and they knock each other down in Oshirogo. Only the leaders of the Four Houses and their Katoku (successors) had the right to attend this game.
This system continued all the way toward the end of the Edo period when the Shogun system collapses and the Meiji Restoration started (1868). All the old annual findings, titles no longer applied in the new government. And the dark period of the Go began. Jowa was the last Godokoro and last Meijin at Edo period, but not the last one to hold the old Meijin hereditary title. The Meijin as an independent honorary title without salary just as the original meaning of the best Go players existed all the way in the Oteai system to 1938 when the 10th and last old Meijin Honinbō Shusai retired and no longer wish to use the title. But the 9th and 10th Meijin although no longer officially recognized by the government, still held weight in the communities. The Oteai system established in 1927 is a modified competition system based on the old dan level system, only that the difference between 1 rank now reduced from 1/2 stone to 1/3 stone, where pro players of all ranks can compete with each other without komi. And under this system, players were no longer bounded by the jouzu cap of 7d. Oteai last all the way to the early 21st century in 2003 and was replaced by the new promotion system. The model of Oteai had been widely adapted for amateur dan ranking competitions (up till very recently) and some variations of this ranking system still existed today.
The conversion of named titles to numerical titles were a very long and slow process. And a lot of the countries only started to adapt them as late as in the 1970s or 1980s Simply put, in the 19th century there were no 9d or 8d or even 7d as numerical ranking, all the top players are jouzu numbered in the dozens only, and only 5d and 6d are pros and consist the majority of them. The 1d to 4d were usually not considered as pros (although they certainly can challenge top players). One thing made advancing to 5d ~ 7d so difficult in the old system is that you can actually got demoted by your opponent if you lose, but if you want to advance a level, not only did you have to beat better opponents and if someone in the Four Houses disagreed with your strength, they could challenge you in Jubango to see if you are actually that good (also all the Four Houses have to agree upon the decision to issue the official diploma). This process would take quite a long time when you applied for a diploma and waited for all the challengers to recognize your strength, (like a national wide review system, every high dan player should be in theory absolutely stronger than everyone below him), and all the leaders of four houses have to agree for advancing to Jouzu (since they are either potential recruits or competitors). And definitely, for Meijin title where every top player in all the houses (and after Meiji Restoration, include all the Go associations who start to issue diplomas competing with old Go Houses) would issue challenges if someone wanted to apply for it, and he had to beat and demote literally everyone to become Meijin. So naturally, there were gaps of years between each Meijin (thus without Godokoro title holder) since the settlement of all challenges for the next Meijin would take a really long time. Also, since Meijin is a title for life, usually, it’s toward the later stage of the previous Meijin’s life there were half-Meijin appeared.
The amateur dan ranking system in Taiwan work with several levels of “group ranking competitions”.
The 甲組 (甲 is akin to the alphabet A group, this is for the kyu rank) rank to dan group competition usually sponsored by local governments or businesses held by one of the many go associations take place quite regularly where people without diploma can attend, but mostly consist of young kids from various Go schools/collages, who already recognized them above 3k (1k to 3k strength basically, but they are in house rank, and different schools have their own ways of ranking kyu). The group competition works with Swiss-system tournament rules, divide participants to 32 people per group with 5 rounds of games. The winner got to met the winner of the same win rate in the next round. People with 4 wins out of 5 rounds got the 1d diploma. (usually this means 6 people out of 32 got the diploma).
There is another kind of association in dan ranking competition usually don’t rely on the win rate, but giving each win condition certain scores and rank them accordingly, and only a fixed number of the top competitors get their 1d diploma. Also, there is a third way to get a diploma, that is if you can get a recommendation letter from a pro player or an armature 6d (and above) to vouch for you.
The 2d to 4d rank-up group competition also uses the Swiss-system tournament rules and all the 1d to 3d players in the dan 甲組 (the 甲組 group for dan, differ from the 甲組 group for kyu) mixed together. The one with higher rank has to give 1 handicap stone per rank difference. In the end, a fixed ratio of players (used to be 1/10, gradually increase to 1/8, and now 1/6) got promoted (1 rank at a time), or using 4 wins per 5 rounds depending on the competition format. Again you can get recommendation from pros or armature 7d to promote you.
The 5d to 7d ranked up groups each have their separate rank up group competition (no mixing of different rank) with 乙組(B group for 4 dan), 丙組(C group for 5 dan), and 丁組(D group for 6 dan). The ratio for 4d to 5d is 1/10, and 5d to 6d is 1/16. The 6d to 7d are restricted, normally no more than 1/64 with a fixed quota. And they were held less frequently and divided to larger area where people from multiple counties and cities will participate in the same regional competition. The recommendation route is applied, but the letter needs to be issued by associations not by persons, and can only go up to 6d. All 7d has to come through competitions. And there is a special 8d ranking as an honorary title, which no one has gotten yet.
Finally, the kyu ranking competitions are less rigorous. The 乙組 (B group for kyu and beyond) have no fixed rules as to what’s the mixing of ranking per group, what kind of competition format, and even the ratio/winrate are not regulated. Since some schools have their internal kyu ranking and different ones have different range (some only just goes as low to 15k, some goes to 30k, some even goes to 60k). The rank handicap varied a lot, some use 1/2 stone (6 komi, or less per rank), some use 2 stones per rank (for those with very few kyu levels). There are hundreds of thousands of kyu players here in Taiwan, and millions know the rules and able to play like beginners, so it is really not possible to have a unified rating system. Most schools have their internal ranking score for kids to get grades (usually inflated to make the parents happy so they will keep sending their kids to schools 
)
On the other end, the pro levels are also not unified, but since they are only issued by associations, each associations have their own ranking systems. Only the biggest associations’ pro ranks are accepted outside of Taiwan. Interestingly there aren’t many 9p in Taiwan, only 6 at the moment, and 3 8p, the total pros combined in Taiwan Go association is just above 100.
Oh wow i didnt know that Taiwanese amateur ranks, and the entire system, is so well organised! In europe its more like a collective mess of local clubs and randomly held tournaments with guideline-like criterias for rules.
Of course the kyu ranks are impossible to judge properly, since for most kyus go is just not-so-serious hobby that they do for the fun of it, but your B group is still lightyears ahead of what you can see within egf ^____^
@Counting_Zenist I really enjoy your detailed posts, even if I don’t always have the time to read them :3
In europe its more like a collective mess of local clubs and randomly held tournaments
Actually, I like the chaos. It’s cool that you can hold a tournament above a pub, in a company headquarters, underneath a car park etc. – and you can do big boards, small boards, slow, blitz, komi bidding…
The go players per 100 people ratio is pretty high even compared to other Asia Go communities (probably 5% at least in my generation), and do you know the lead AlphaGo designer Aja Huang also comes from Taiwan and he has armature 6 dan. People do play Go as recreations to an extend, amount other chess games, (although probably less so offline nowadays, more online). My father and my brother also play Go, so was my grandfather.
The golden rule is: when there is money to make, there will be organizations to “organize” them and make the best of it 
BTW, the survey entries almost stopped increasing only at 117 as of now. I think we still need to push to other communities a bit more. I’ve already asking lots of players in Taiwan and they mostly filled it. What about other offline communities in the world?
Anyone can help spread the words in the offline communities as well?
Perhaps you could ask a few streamers if they want to spread the message
I finally took the time to properly read this. Great post, very explanatory. Appreciated.
One interesting tidbit about the old system before 20th century is that the stronger player in a game has the right to suspend it (uchikake, and a lot of these games just ended up paused forever. Also Meijin can do it as many times as he want). This loophole was “exploited” by plenty of top players, and some Meijin not so strong compared to everyone else (or getting old) abused this power often. Or they can send “pinch players” to play in their place instead (usually their successors or students), more often they just gathered all the students in the house to come up with moves for him.
Some people, no matter the era, would always try to cheat the system.