I tried to go back and find a game where this is particularly prominent and I was the loser (didn’t want to be rude and put another person’s loss on display ) but I find this to be the case in most games.
In this game, move 10 (-6%, -0.2 pts) is hardly a game-changing move, and yet it is marked as one of the top 3. Or to re-state that, it may affect the course of the game, but it is not really worth commenting on more than other moves in the opening. The only reason it was chosen as a top move seems to be that the game is so close to a 50/50 toss-up according to the AI.
IMO, move 17 (-8%, -4 pts) was much more note worthy. And generally, high point losses are more interesting than the actual AI-appointed game deciding move.
Anyone else have thoughts on this? I think switching to “highest point loss” would already be an improvement, but perhaps a hybrid would be best, where the current method is weighted more heavily in higher rank games (assumption: high ranked players are more likely to care about the game being perfectly even than kyu players)
@mark5000 Im not sure I follow. Why does move 12 need to be factored into move 10?
@flovo okay that makes sense. What I am proposing is that change in score also be taken into account.
As a thought experiment, imagine I am behind by 5-10 points, then the game is still about even in my eyes. If I were to lose a 30 pt group after that, I would consider that the game changing move. However the AI wouldn’t see this (going off of win rate) because it is likely already showing <1%.
Consider the screenshots below. At move 10, the AI gives Black a 51.6% chance to win, so long as Black plays the suggested path starting at E3. Black did not do this and incurred a loss. To see how big the loss was, let’s check the win rate over the next move or two.
Once Black plays move 11, and White responds (correctly) at move 12, Black’s chance to win dwindled 16.6 percentage points to 35.0%. You can’t stop at move 11, because White’s accurate reply increased the gap. In other words, the AI (at the limited playouts given) could see more clearly how bad move 11 was only after White played move 12. That’s why taking the -5.9 number from screenshot 1 might be misleading.
Yes, changes in score should make more sense for kyu players. I always look at score graph instead of winrates. I also agree that some hybrid metric that gives higher priority to moves around 0 (50%) could be useful. Current system it seems gives too much priority to going over equator though. These mistakes are often hard to comprehend.
Thanks for clarifying. I was a little confused why one would have to look two moves ahead, but really it’s just one move ahead given that move 11 was the real mistake. Also, gains in points tend to be negligible so I didn’t take that into account.
I think this game still represents my point- my 4pt mistake at move 17 is probably more relevant than my 2 pt mistake at move 11, but maybe not as starkly as I thought. A more obvious example would be from a handicap game:
The top three moves are some relatively inconsequential moves played when the game is at the “tipping point”. However one of the big “learning moments” would have been at move 40 (a joseki error that lead to a 20 pt loss).
Yeah I mean if the three moves were supposed to improve ones game, they probably shouldn’t be the last chances you had to save the game, but the reason the game got to these last chances in the first place.
At least two of the moves are supposed to be 5 and 12 point losses which are sizeable but the 20 point one you’d really expect to be included.
Actually if we’re pitching ideas to change which moves are selected, why is it three moves? Why not four moves, and two for each player at the very least. Even if the game was completely one sided, why not still point to biggest point mistakes by each player rather than just the biggest mistakes overall, which could be just by one player in theory. Wouldn’t this also be better than just showing mistakes at the tipping point, where if the ai played on it would win? (Maybe they coincide in some cases)
Since nothing changed in the last 2 months, I want to revive this feedback and add my support to it. The AI review Key moves are not useful right now for the reasons people mentioned: They are often 1-point errors that cross the tipping point, but I don’t care about these. As an amateur player, I’m much more interested in the 5- or 10-point errors even if they don’t cross the tipping point. This is especially true in handicap games, where Black’s winrate is 99% for a large portion of the game, and the errors made during that portion are never key moves.
Thanks for bumping. As far as getting this in, I posted an issue on GitHub.
It didn’t sound like the developer resources are there to get this done now, but if someone wanted to prototype this for site supporters, that is possible to do on the frontend.
Supporters still don’t have top 3 moves on their graph of full AI evaluation. Supporters in chat just opened game of non-supporters and were surprised. They asked “is this a graphical bug?”
Decided to dig into this, and I think I have enough code points that I can prototype this for site supporters since that is a frontend fix. Incidentally, that should make implementation for non-site supporters (which needs to occur on the backend) easier.
I will make a change to handicap games only. For those, we will use Δ(score) instead of Δ(win rate) to determine “worst” moves.
Does anyone have games I can test with? If you could share some games along with what you think the top three game changing moves should be, that would be amazing! @mark5000@S_Alexander@shinuito@frolag
Yes, but I’d need some UI input. Those 6 moves are readily available, but I’m not exactly sure the best way to expose in the UI. Maybe clicking the “+6 more” to reveal more move tiles? Or just more red dots on the graph?
Apologies as I may have missed something, but why would we use ∆(score) only for handicap games ? Wasn’t the whole point of the thread that it is a much better metric in any case ?
a bit of “if it ain’t broke don’t fix it”. After considering @mark5000’s comment and also reviewing more of my games, I felt handicap games are the ones where using Δscore is the most beneficial.
If you have examples of non-handicap games where Δscore would have been better, I can make the change apply to all games without much difficulty.
In my opinion, it is broke! There are certainly rare cases where winrate gives better information than score, but in almost all human games score is more meaningful.
Showing the biggest mistakes in terms of score is by no means a perfect solution either (for instance, lightvector has pointed out that people tend to get too hung up on fixing their “biggest” mistakes, while some “smaller” mistakes may be both more common and easier to fix), but I do think it’s slightly less useless than the current key moves (which are usually just some random minor mistakes that happen to change who is in the lead).
By the way, since we have a switch between winrate and score for the graph, would it be possible for the key moves to change accordingly? That would be very intuitive I think, and it would let everyone use the metric they prefer
) but I find this to be the case in most games.
