On a slightly related note, the Wikipedia article on the Elo system has some really interesting bits. Specifically:
From a modern perspective, Elo’s simplifying assumptions are not necessary because computing power is inexpensive and widely available.
Moreover, even within the simplified model, more efficient estimation techniques are well known.
And then this:
Subsequent statistical tests have suggested that chess performance is almost certainly not distributed as a normal distribution, as weaker players have greater winning chances than Elo’s model predicts.
If this is true, maybe it would be interesting to try out some variations on the rating system and see if there is a noticeable difference.
Are any of you mathematically enclined enough to explain some of those ‘well known’ better estimation techniques?
If some of us were really knowledgeable about statistics, maybe we can figure out a way to get a new, more realistic performance distribution empirically from available OGS data and adapt Elo to use that? Better yet, make it self-correcting over time like the rank points 