Top 3 moves: Score a better metric?

Yes, changes in score should make more sense for kyu players. I always look at score graph instead of winrates. I also agree that some hybrid metric that gives higher priority to moves around 0 (50%) could be useful. Current system it seems gives too much priority to going over equator though. These mistakes are often hard to comprehend.

3 Likes

Thanks for clarifying. I was a little confused why one would have to look two moves ahead, but really it’s just one move ahead given that move 11 was the real mistake. Also, gains in points tend to be negligible so I didn’t take that into account.

I think this game still represents my point- my 4pt mistake at move 17 is probably more relevant than my 2 pt mistake at move 11, but maybe not as starkly as I thought. A more obvious example would be from a handicap game:

The top three moves are some relatively inconsequential moves played when the game is at the “tipping point”. However one of the big “learning moments” would have been at move 40 (a joseki error that lead to a 20 pt loss).

3 Likes

Yeah I mean if the three moves were supposed to improve ones game, they probably shouldn’t be the last chances you had to save the game, but the reason the game got to these last chances in the first place.

At least two of the moves are supposed to be 5 and 12 point losses which are sizeable but the 20 point one you’d really expect to be included.

Actually if we’re pitching ideas to change which moves are selected, why is it three moves? Why not four moves, and two for each player at the very least. Even if the game was completely one sided, why not still point to biggest point mistakes by each player rather than just the biggest mistakes overall, which could be just by one player in theory. Wouldn’t this also be better than just showing mistakes at the tipping point, where if the ai played on it would win? (Maybe they coincide in some cases)

1 Like

Since nothing changed in the last 2 months, I want to revive this feedback and add my support to it. The AI review Key moves are not useful right now for the reasons people mentioned: They are often 1-point errors that cross the tipping point, but I don’t care about these. As an amateur player, I’m much more interested in the 5- or 10-point errors even if they don’t cross the tipping point. This is especially true in handicap games, where Black’s winrate is 99% for a large portion of the game, and the errors made during that portion are never key moves.

4 Likes

Thanks for bumping. As far as getting this in, I posted an issue on GitHub.

It didn’t sound like the developer resources are there to get this done now, but if someone wanted to prototype this for site supporters, that is possible to do on the frontend.

2 Likes

Supporters still don’t have top 3 moves on their graph of full AI evaluation. Supporters in chat just opened game of non-supporters and were surprised. They asked “is this a graphical bug?”

1 Like

Decided to dig into this, and I think I have enough code points that I can prototype this for site supporters since that is a frontend fix. Incidentally, that should make implementation for non-site supporters (which needs to occur on the backend) easier.

I will make a change to handicap games only. For those, we will use Δ(score) instead of Δ(win rate) to determine “worst” moves.

Does anyone have games I can test with? If you could share some games along with what you think the top three game changing moves should be, that would be amazing! @mark5000 @S_Alexander @shinuito @frolag

4 Likes

@stone.defender site supporters do see key moves, but not as red dots!

Are the three dots what you are looking for? If so, it would be relatively simple for me to add those while I am implementing my other changes.

3 Likes

While you’re digging into that code, is it possible to display those extra key moves somehow? The “+ 6 more” mentioned in your picture.

I was thinking about that…

Yes, but I’d need some UI input. Those 6 moves are readily available, but I’m not exactly sure the best way to expose in the UI. Maybe clicking the “+6 more” to reveal more move tiles? Or just more red dots on the graph?

1 Like

Apologies as I may have missed something, but why would we use ∆(score) only for handicap games ? Wasn’t the whole point of the thread that it is a much better metric in any case ?

2 Likes

I don’t have time to do a proper mock up right now but I was thinking of something like

default state: key_move key_move key_move + 6 more

click + 6 more

mv# key_move step step step
mv# key_move step step step
mv# key_move step step step
mv# key_move step step step
mv# key_move step step step
mv# key_move step step step
mv# key_move step step step
mv# key_move step step step
mv# key_move step step step

1 Like

a bit of “if it ain’t broke don’t fix it”. After considering @mark5000’s comment and also reviewing more of my games, I felt handicap games are the ones where using Δscore is the most beneficial.

If you have examples of non-handicap games where Δscore would have been better, I can make the change apply to all games without much difficulty.

2 Likes

In my opinion, it is broke! There are certainly rare cases where winrate gives better information than score, but in almost all human games score is more meaningful.

Showing the biggest mistakes in terms of score is by no means a perfect solution either (for instance, lightvector has pointed out that people tend to get too hung up on fixing their “biggest” mistakes, while some “smaller” mistakes may be both more common and easier to fix), but I do think it’s slightly less useless than the current key moves (which are usually just some random minor mistakes that happen to change who is in the lead).

By the way, since we have a switch between winrate and score for the graph, would it be possible for the key moves to change accordingly? That would be very intuitive I think, and it would let everyone use the metric they prefer :slight_smile:

7 Likes

Any of my games :grinning_face_with_smiling_eyes:

I feel like the point was already made in this thread that ∆(winrate) results in focusing on the few moves that overed around 50% winrate, instead of the great blunders that I would have liked to see highlighted.

For example in my last game (wasuwith vs. qnpnpmqppnp) there was a blunder on both side at -42 point (!) on move 140, and it’s not gonna be highlighted.

If possible the best would be for the key moves to follow the graph : if the graph shows the score, show the key score moves; if the graph shows winrate, have the key winrate moves. But if it’s too hard and only one option is possible, then score all the way.

6 Likes

I get the feeling this would require too much human input though, or an Ai that’s trained to teach humans.

There’s no way, I don’t believe to look at only the winrates, scores etc and tell somebody “move 201 is an endgame move you should know at your level”, “move 86 is a shape move you should know” etc. If they’re just 0.1 point mistakes they don’t really matter. Even if they’re 1 or 2 point mistakes they probably don’t matter unless it’s endgame. Realistically a 10+ point mistake is probably worth learning how to fix, whether it’s not picking the right point for eye space, not connecting a group properly etc.

I would like to see some counter examples to the above points however.

1 Like

I mean I could probably come up with some examples where highlighting a 10 point ish mistake and the ai variation won’t be helpful, but I imagine in the same game, some basic metric on winrates and scores won’t be too simple to conjure up which will find better key moves ( by simple I mean things like first difference as opposed to “weight the last ten moves in such a way, and consider applying some nonlinear function to each score”)

Right, this is the kind of thing that a human teacher can do that we can’t (yet) substitute with an AI :smiley:

But a human without a teacher can still get better at interpreting and using the AI in a productive way. When a move is -1 point, sometimes there’s an easy to understand reason why, and you can figure it out by yourself or by looking at some different continuations from the AI (and if it’s a common shape you’ve just learned something very useful). That’s the kind of thing that you will totally miss if you only look at the “biggest” mistakes, and reason that smaller stuff doesn’t matter at your level (I’m definitely guilty of this myself, by the way - thinking for yourself is hard work, so it’s very tempting to take the lazy route of just staring at pointlosses).

Anyways, the best way to review with AI is a different topic. I basically just wanted to say “point difference is a better metric for key moves than winrate” and then I felt compelled to clarify that using point difference isn’t perfect, but it’s about the best we can do with current technology. Looking at the biggest point losses throughout the game is certainly a good starting point when reviewing, so this feature would be very much appreciated.

2 Likes

I also agree with you that there probably is way more things with smaller point losses where it will be easier to find something to improve on. The problem is that we don’t know out of the likely 100 1 point losses or possibly as many 2 point losses etc which one to randomly choose from for it to be actually helpful for the player to learn from.

It’s just that if one is only giving a top 3 or top 6 or top 9 game changing moments in some way (and only those variations for non-supporters), then I’d prefer it did that (a) for the biggest score shifts even if they don’t change the winrate (for the reasons mentioned before, there probably the game changing mistakes rather than the 1 point mistake that just tipped the score from 0.5 to -0.5) and (b) an equal number for each player ideally.

If someone gets hammered likely they’ll have all the big score losses, but that’s not to say the winning player didn’t make mistakes or have something they could learn also.

Maybe there’s other suggestions though on whether there should be some other subdivisions that could be interesting? Like guarantee a mistake from the first 50 moves, or last 50 etc?

3 Likes

Many thanks! Examples are exactly the thing I need!

I will note though… this is a handicap game, so it still supports my hypothesis that Δ(score) is most helpful in handicap games.

EDIT: But I didn’t have to go back too much farther to find another good example: qnpnpmqppnp vs. kJames :grinning: lots of swings in these games!