Would it be possible to do an auto-graded tsumego system based on real OGS rank? For any given puzzle, you would track first attempts only, resulting in “solved” or “failed”, and the user’s existing OGS rank. Though I would not want the puzzles themselves to affect anyone’s rank, of course.
I’m not sure exactly how to take the resulting data and come up with a good system to grade each puzzle but I think it is very possible. Has this been brought up before? What are your thoughts?
Solved or failed is maybe not enough to generate a rating. Firstly players should have the same attitude/interest to solve it. Players may use some help too.
I noticed that players are not so regular in their results because these problems refer to different parts of the tactics and you can be familiar or not. I know 5k very relax on ishinoshita, and 1d who are not trained to read them.
I proposed some times ago to let everyone participate in problem strength evaluation (which is a bit different as what you suggest)