You can see an example of the fast review by clicking the little three line icon thing right above the win/loss graph, it says “Top 4 moves according to Leela Zero”
2 Likes
The fast review is very teasing, but having just a number isn’t much satisfying.
Could it be possible to have also the win-rate graph in a non-interactive version?
It would be enlightening to understand the flow of the game and also the importance of those Top 4 moves.
Another question:
is it normal to have a strongest move (bightest green) without variations?
See picture:
Nice feature 
. In future, I will get a slap in my face after each game 
Let’s hope it will help me improve.
It seems to me that the bar showing the chance to win for both players and the win rate change of that move are using the blue line, not the better purple line.
For example at move 42->43 the purple line is already near 100%, while the blue line is at about 50%. The improvement of blacks move seems to be only an artefact of LZ not reading at all.
Do I get something wrong or is that because I don’t have a supporter account on the beta site or is this a bug?
An improvement of 40% against LZ previous estimate is much much more than I would expect to be possible. At least after enough playouts.
As I understood it’s not the strongest move, but the move LZ explored most. It probably was the best move until LZ explored it in some length. After extensive exporation LZ probably decided it isn’t as good as she frist thought. With more playouts the color would probably be less intense.
explored = spend reading time
letters = strongest moves
2 Likes
Amazing 
Are you planning on crowdsourcing any of the analysis bots, like the way lichess does with fishnet? I’d be happy to contribute GPU time if it was as simple as running a docker container and plugging in an API key.
3 Likes
This is extremely good!
This whole “explored vs strongest” thing is puzzling me, possibly because seeing ‘A’ makes me think “I should play here”.
AIUI, the place that Leela would play at any given time is the one with the most playouts - the most explored. That’s the “blue” circle in Lizzie.
Am I understanding correctly that is the green coloured node in this feature?
It’s not totally intuitive that some other node than the one the bot would play is marked “A”.
I’m I misunderstanding?
GaJ
1 Like
I thought about that but I’m a bit worried of malicious users, we’ve had a few very technically competent trolls over the years… so basically to avoid that headache I figured it’d be best just to keep it all in house
3 Likes
I don’t understand the problem you are seeing.
After black 41 was played, white’s odds were 52.4 and black’s 47.6. Best next move for white is J13. If this move had been played, I guess odds would have remained the same.
Problem is white 42 was not J13 but A16, which is a mistake and changes the board. Hence best next move for became black D13, which greatly improved its odds.
Now it is true that given that black’s best answer to white 42 improves its odds by 41 percentage points, I would expect white 42 at A16 to be rated much worse than just -1 percentage point.
But in any case, Leela would not have played A16 and therefore did not misjudge the previous position.
The odds should be given as “if black and white will play the moves which improve their probability to win as much as possible starting with the given board, what is the probability for black/white to win the game”. With this assumption, neither black nor white should be able to increase their odds, but only decrease their odds by playing suboptimal moves.
So the move black 42->43 which increased blacks odds by over 40 percent points shows a big misjudgement of position 42 by LZ.
I cannot find the feature on iPhone, is this just for the desktop version or am I missing something?
The feature is not available yet. There is a preview available if you follow the link in the initial post.
2 Likes
I used the link but can’t see any reference to the AI review, no graph at all.
Is it possible and is it still a probability? I don’t know how Leela works, but following what you are saying, what should white’s probability to win be after Black 1? If all future best moves are factored in, the probability is not the probability to win the game but the probability to play all the best moves, is it not? My point is that the probability should probably factor in bad moves as well.
After black 41, whiet’s odds to win are 52.4%, with the given board. Whether its odds will increase or not depends on the quality of the next move. If the next move improves the board for white, then the probability to win increases.
So I still don’t really understand the problem that is raised.
The graph isn’t there any more. I don’t know why. Maybe @anoek changed something.
I know get sometimes the error message:
Maybe it’s related.
(My scriptblocker blocks fonts.googleapis, googletagmanager and ravenjs, but this wasn’t a problem before)
I idealized my explanation a little. I assumed LZ knows what the best move is. If LZ indeed would know what the best move is for all given boards, then LZ would give the odds only as black wins, draw and white wins.
LZ gives us only the odds, because it is not able to find the best move, but uses positional judgement and reading to find the probably best move. LZ cannot read through the whole game, so LZ factors in her own uncertainty of the positional judgement, but doesn’t factor in possible mistakes (possible mistakes in the sense of “I know there is a better move, but maybe I will not play it”).
The probability LZ gives are LZs estimations for a game LZ vs. LZ where both LZs play as good as possible.
The given probabilities are used by LZ herself to find the best move to play. To factor in blundering (hope your opponent makes an error) doesn’t do any good if she wants to win the game. I think the neural network factors in if the move leads to a hard to read positions, since the zero AIs tend to play very save if they think they are ahead.
The probability is not a statistical factor for random players, but only for a LZ playing at training strenth (1600 playouts).
In our case we ask LZ after white 42 “what is you probability to win the game as black if you would have to play against yourself?” and LZ answers: “The best move for black I can find has a probability of about 50% to win the game.”
Then we place the black stone 43 on the board and ask again: “what is you probability to win the game as black if you would have to play against yourself?” and get the answer: “with this move my probability to win as black is now 90%. I hadn’t seen that move before you played it. I have to admit, you found a move that is much better than I thought is possible.”
It statistics nerd in me forces me to write this here:
It’s a 41.5pp (percent point) change not 41.5% (percent).
But do like you want. “Percent points” will probably irritate many users because they aren’t used to it.
9 Likes
It should be there? I woke up to some (unrelated) server problems on the beta server, so that might have been causing some errors. Can you check again?
How did I not know this… fixed, and thanks for the correction 
1 Like
Looking fine on Linux (64-bit)/Firefox 66.0.
1 Like
But this would mean that we have in this case many moves (those with letters and no green) that are better than that one but way less explored.
Does it make sense?
1 Like
What you’re proposing is a refinement, which I’m afraid to discuss right now because it may detract from the more pressing issue that the underlying data seems to have errors. Your comment is also not specific to my table. If you’re saying that winrate change should be “curved” before being interpreted as mistake severity, then why not bring this up in the context of the “Top 4 moves” feature?
If that’s true, then you might have found the bug that I predicted exists. If winrate change is based on 1 playout and not 200, then it means that 99.5% of LZ’s effort is being discarded.