A couple of questions about handicap and rating

[Feijoa]

@gennan

I suppose that good matchmaking means looking for opponents against whom the predicted outcome is about 50% winrate. How else would you implement good matchmaking?

@xela

check the prediction accuracy separately per group.

@anoek

We expect to see a white bias in handicap games because white is always the stronger player, and black only gets one stone per full rank difference, so we’re not shooting for 50% here, but ideally we will see something fairly flat as we increase the rank difference and number of stones given, which we do for the most part.

If we’re going to evaluate ranking/matchmaking systems, it’s important to have some idea of what the goal is, and just saying that you are targeting a win rate of 50% (or close) is not enough, is it? Assigning ranks completely randomly gives you a win rate of 50%.

If the ranking system predicts a 50% winrate for some game, does its outcome actually tell you anything about the quality of that ranking system?