That’s a good point. It’s probably the individual games, many of which have handicap, that would be more valuable to look at for data analysis…
… but the rating system can’t look if it’s unranked (since maybe one player was playing blindfold go, or using a computer assistance, or who knows?).
Why does it just work this way for tournaments?
Probably an unintentional loophole in tournaments, since “all tournaments are ranked”, and most players just didn’t notice (until recently) because approximately nobody looks at “Game Information”.
But I think allowing non-standard boards in normal challenges to be called “ranked” would be pretty confusing, since, unlike tournaments, you see “ranked” vs. “unranked” when you select which games you’re going to play and when you set up challenges. I think my somewhat unintuitive (albeit arguably “correct”) definition of “ranked” might cause way too much confusion if we allowed that before the rating system actually looked at the games (to be clear, there isn’t even a plan for that yet, it’s just something I’m thinking about on my own).
I’m not strongly against closing the loophole (option #2 above). Seems like most people disagree with me, so maybe we should just close it.