I just finished this game https://online-go.com/game/22825772
In AI, move 5 is the actual move, the best move and shows a 23% drop.
I’m confused.
1 Like
I think it was saying that as best it could tell, white’s move 5 was going to make white 23% worse off than before. It does seem odd.
I ran a full strength AI review and it has a more sensible result - I guess there weren’t enough playoffs in the initial review to find the better move.
2 Likes
(F6 makes sense, after the fact. 
)
I guess it’s an engine thing, then. 
Thanks!
1 Like
I wouldn’t say that move 5 is the best move.
I see a “0” at F6, so that one should be the best move.
It’s barely noticeable because it isn’t highlighted.
Looks like AI studied a lot the bad move (hence the colour) and a very little the best one.
Or maybe I still don’t understand colours in AI analysis… 
1 Like
The original AI showed move 5. Eugene run the strongest one and it gave F6.
I tried to dig deeper. 
Here:
I found that
The Blue circle indicates the AI thinks this is the best avilable move.
So I fear there’s a bug somewhere: -23% can’t be the best available move if there’s a 0% around.
Either the colour or the manual is broken.
Omitting some technicalities, best move is the one with most playouts not the one with highest winrate.
To clarify: interestingly, that 0 was not there until I ran the second run. I’m pretty sure that it was displaying only the single -23 option.
2 Likes
Yep, that’s why I asked. 
Now I’m confused again. 
3 Likes
I went again to that game and found something interesting that could be reproduced.
I firstly looked at the basic AI review. Then switched to the (I) analysis and there was only the -23. Moved to the (III) and back to the (I) and the “0” was there.
So, perhaps the display of that zero is just a visualization issue. That doesn’t explain why a -23% should be considered a "best move " though
2 Likes
We probably forget to clean the marks when switching between reviews.
About the board position. There are 3 separate technicalities making this position confusing.
-
The AI suggested move is the played move. For the played move, the displayed delta is not the delta reported by the AI for this exploration run (winrate expolored - total winrate move 5), but the delta between the whole board winrate of this and the next board position (total winrate move 6 - total winrate move 5). We do this, since the winrate of the next board position is better explored (all available playouts used instead of only a part).
-
It’s a low visits blindspot of KataGo. KataGo thinks E6 is the natural move on the given board position and spends most of it’s time exploring variations of the move. At low visits the exploration is driven by what KataGo thinks is a likely move. Even at the 2000 visits run @Eugene ran, the estimated winrate for E6 and F6 are almost equal. Only after analyzing move 6 E6 turns out to be a bad choice.
-
The AI selected move is neither the move with the highest win rate nor the move with most visits. The KataGo uses a value called utility. That’s for some reasons:
- The move with the highest win rate sometimes is underexplored (for example KataGo spent only 1 visit on the move for reasons, but the estimated winrate is the highest of all explored moves).
- The move with most visits could have turned out to be a bad choice.
To avoid both cases, the utility is calculated using multiple properties and a lot of math to find a move KataGo can be most confident of to be a good choice.
There are 3 explored moves for the 300 and 2000 visits review:
| Move | 300 visits | 2000 visits | |
|---|---|---|---|
| F6 | WR visits utility |
42.88% 5 -0.1292 |
11.66% 795 -0.796 |
| E6 | WR visits utility |
9.87% 293 -0.8453 |
12.98% 1203 -0.7623 |
| E4 | WR visits utility |
48.34% 1 -0.0029 |
49.96% 1 0.0443 |
- WR: win rate black, so lower is better
- visits: how much time KataGo spent exploring variations of the move.
- utility: a lot of math, used by KataGo to select the best move. It’s whites turn, so lower is better.
For fans of long lists of numbers, the raw data is linked above.
On an additional note: We don’t show the win rate change for low explored moves (less than 20-50 visits, depended on total visits). They are usually bad moves despite having high WRs in some cases.
1 Like
So, it’s not a bug, due to math it happened to be this. Rare but probable. Right?
Yes.
1 Like