It also depends if the comparison is to the optimal score calculated on the previous or current move. The OGS AI review markings uses the former (so presumably does this new categorisatuon feature?) which means positive numbers are more common because the prior evaluation was using fewer playouts on the actual move so it’s less accurate.
1 Like
What’s one level higher than a blunder? Maybe we should have that for moves that are < -50 and maybe another one for < -100.
1 Like
Nice feature! To me it makes absolute sense to define the categories by points lost compared to Kata Go’s choice and be strict about it. Any other criterium will be much more confusing.
2 Likes
Disaster < -50
Catastrophe < -100
4 Likes
“Oopsie”
2 Likes
Even if it’s measuring points (which KataGo can do), this style of AI is not perfect and can show a positive change since the last evaluation. It’s very similar to an expert being surprised by beginner’s luck.
The only way to totally avoid positive surprises is if you could traverse the whole game tree. Possible in tic-tac-toe, not currently possible in Go.
All this to say, bucketing positive points in the “Perfect” bucket still makes sense to me
4 Likes
It may not be perfect but it’s pretty darn good for something that can be compiled automatically after each game.
This would also be part of my use of the term “blunder”, i.e., not just a big mistake but a big stupid mistake. However it’s probably asking too much for the computer to distinguish between what it thinks is stupid and what it thinks I think is stupid.
1 Like
Certainly in this kind of quick free analysis. However, I think it is plausible with some effort: use one of the KataGo networks trained to play like a human at your level with low playouts so mostly just policy, and then if that network would have played one of the good moves instead, but you played the bad one, we can say it’s probably a blunder for you.
I think that could be ok. If all moves result in equal win rate, then for the purposes of learning how win a game in such a situation, all moves really are equal.
The evaluation graph can toggle between win rate and points, it might be interesting to also toggle the move categorization. Then you could see both of the move made sense locally, but switch to a view of “which moves did I mess up that really mattered?”
1 Like
I think the issue with this one, while it makes sense, is that probably half of the endgame moves will be these sort of game losing mistakes in a close game. Especially in a half point game, I might play something that looks sente and my opponent answers and both us are making ~-3 points mistakes because Katago realises you can squeeze in something else first that’s more sente.
The top 3 mistakes function for non site supporters had this issue when using only the winrate. It would say that this 0.5 mistake in the opening was one of your biggest mistakes because the evaluation went to B+0.3 to W+.2 but the winrate changes like 10% or 20% or something.
While the blunder you made that died with your corner didn’t matter because you were still ahead after it, the winrate didn’t change at all.
That sort of thing doesn’t make much sense to me, and I don’t think it would be useful for people reviewing either looking to improve.
I think this makes sense, and aisensei for example has some kind of slider where you adjust your level and it adjusts what it calls mistakes and blunders.
You could try something like you’re saying and using different Katago networks to calibrate it based on level, or you could take a handful of games at different strength buckets of players and calculate an average point loss per move.
For instance, maybe a 9p is losing like 0.3 points per move on average in a game that isn’t too sharp or complicated, but maybe a 10kyu is losing 2pts per move on average.
You could do something like set the blunder at 5*(average points loss per move), so maybe losing 1.5+ pts is bad for a 9p, but 10+pts is bad for a 10kyu.
Just as an example, of course the “5” is arbitrary. Pick your favourite scaling function instead, or fit it better to real data to make it more useful.
3 Likes
For comparison, AI sensei changes the definition of good move, inaccuracy, mistake, blunder according to the level of the player. For my last game, if I set my level at 1d I get this
I don’t know exactly where is the limit between inaccuracy and mistake, I think it’s about 4 points.
But if I set my level at 10k, the limit is about 8 points.
5 Likes
It depends on level student slider. What this slider does is to set different parameters on what can be considered as blunder, what as a mistake and so on…The thing is, is more important movement value or % winning? or both?
Love this new feature. Thanks for adding it. Perhaps adding an option for identifying unique blunders or errors would be helpful? For example in my games my opponent and I often miss an important cut / connection for many moves, sometimes the entire game. This is identified as a blunder each time it is missed. It might inflate the blunder count artificially, but then again each time is a missed opportunity to make the preferred move? A toggle might be good, but I can also see how it might be difficult to implement. Before this table was added we saw the graph spike up and down many times in succession 
3 Likes
Just brainstorming, not expressing an opinion:
-
What would the subsequent moves in the same position be counted as?
-
Isn’t each blunder different: “Yep, KI0 is still a blunder even with those two extra stones on the board”
2 Likes
Yea, not sure. It just skews the count a little, but still each time the move is missed is a still blunder. I don’t have a good answer, but missing the same move multiple times doesn’t feel as bad as making different blunders.
1 Like
I find that to be true - sometimes. It depends whether you saw the move but decided not to play it yet (usually a corner or side move), or you never would have seen it (tesuji).
I know that I get the up down graph when me and my opponent under value a basic move. In this case, each fresh instance is a bad timing decision.
On the other hand, if you don’t see a tesuji, and so blunder by not playing it, it’s not like you are repeating that “bad decision” each turm.
2 Likes
I’d like to make a UX request in relation to this new feature: briefly, I would rather have the “move type chart” be at the bottom of the side-panel, or else be collapsible.
What I want to be able to view in the side-panel simultaneously when analyzing/reviewing a game are the win-rate and the tree of variations, and the move comments / game chat if possible. Throwing the “move type chart” in the middle of everything makes it impossible to see a the win-rate and the other stuff at the same time. It doesn’t really do anything while you’re reviewing, it’s just sitting in the middle of the panel taking up screen real estate.
1 Like
It’s not quite the same as “being able to see them all, positioned where you want”, but there is this:
2 Likes
Oh wow, that’s exactly what I meant by “collapsible”. Honestly, yesterday I was clicking at random all over the table to see if there was any way to make it go away, and I didn’t notice that slider. Thanks admins!
1 Like
In response to a rank slider, perhaps having a toggle for “show mistakes at your level” vs “show mistakes, for lack of better word, all.” If people REALLY want a slider then maybe, but that’s getting into a lot of UI buttons and leading to too many options to click. I think the default should also be, mistakes at their level.
For some other brainstorming ideas, I personally have been playing around with making my own charts out of this data to find key moves, mistake count, ect… One big thing I did was to filter out moves to be counted only once, basically if a key move shows up 10 times it is only counted as a mistake once per side. You could also point out key moves like “This move was repeatedly missed and it was a big one”