AI Review updates 2019-06-19

image

White 100%? But I’ve wooon already…

3 Likes

The exploration doesn’t so deep. The leaves of the explored subtree are just other unfinished board states. For any leave LZ estimates the probability to win. The win probability of a possible next move is then the average over the leaves.

With LZs NN it is possible to do a hunch with 0 playouts, just using the NNs estimated win probability for the current board position without any tree search at all.


LZ isn’t a strict AZ reimplementation anymore. Some minor details may have changed to get better results for the move to play selection.

6 Likes

Known issue, being worked on.

For some reason, the last pass of a game makes the bar flick into the opposite direction. If you click back one move in the analysis, it gets it right.

1 Like

It’s probably a 2x performance gain. Nothing spectacular, but worth doing nonetheless. I know that using one playout per move is bad, but that’s pretty much all OGS can afford it seems, so options are limited. OGS is trying to make the best lemonade with very few lemons, and every trick helps.

At this point I’m thinking maybe we would be better off without the “top three game changeing moves” feature. I know these are harsh words, but hear me out:
Everyone knows AI is superhumanly strong in go, thus many people probably trust the AIs choice of top three moves. People trying to learn from it might come to wrong conclusions because the Analysis is actually lackluster.

Don’t get me wrong, I really appreciate the effort, and I hear the full review feature is awesome now.

1 Like

I was/am being lazy and waiting for the documentation to come out before trying to wrap my head around this feature but there is this:


Thanks again flovo. :+1:

Isn’t the blue move locally better than N6 because I will avoid being captured the 9 black stones. However, I don’t know how effective for the entire game. And why the numbers are so different. Sorry, I’m an old guy who doesn’t know anything about AI, but try to learn. Ignore me if I am saying nonsense.

1 Like

Hello @Ewak and thank you for the question! I am happy to explain :slight_smile:

The 9 black stones are already dead. There is no saving them with any move. I should not have let them die earlier with m6 (-60% chance to win).

I believe that the big difference in numbers (o7: -7.5, n6: +17.3) comes from the different depths to which Leela has explored these alternatives.
Blue o7 is the game variation. Leela has spent a lot of computing time looking at it and she is confident that after black plays there, his win chance will be ~18.4%.

In contrast, she has only glanced at n6. OGS does not reveal the number of “visits”, i.e. explored variations, that Leela has invested in n6, but presumably it was not more than a few dozen. The same is probably true about the various -70 moves strewn about. As a result, the number 17.3 does not mean anything. It is a shallow guess.

This is why I made the suggestion that these shallow results should be removed from the output (and certainly not put front and center in green highlights).
I also have a suspicion that there is something wrong with the “baseline” value of the position (25.9%). Normally, Leela’s move should preserve your winning chances (± 0).

2 Likes

Hi,

I don’t know if someone else has already reported it, I couldn’t find it when I browsed through the thread.

It seems that the diasble AI review doesn’t really do its job in post-game.
You will still see the evaluated moves.

Is there a way to completely hide it?

3 Likes

Got it. Thank you.

This game just ended.

We have the three top changing moves and I don’t understand something.

  • Top changing moves are all worth 0.0pp. That’s quite strange, isn’t it? I mean: I know that “top changing moves” aren’t correctly detected at the moment, but also the same zero value for all three seems strange.
  • The blue ball isn’t visible for two moves. Shouldn’t it highlight the next move?
  • Some pink moves are soooo bad. Down to -50.8pp! Does it make sense?

Also: at move 137, if E16 (starting a ko for life) can be a good move (+1.5) how can be “neutral” D17 (0.0) that forces white to connect?

1 Like

Yep I broke that, I think I’ll be able to get that fixed up tomorrow

In general, a lot of stuff with the top 3 are quite wonky at the moment, all of which should be fixed up starting tomorrow (probably)

6 Likes

I pushed up a mostly graphical update, now with 99% less potato stamps, along with several minor and not so minor bug fixes. More tomorrow.

11 Likes

But Mah stampz!!! :cry:

What does “network size” mean?

Network size means the total number of residual blocks and convolutional layers in the network architecture. (See the AlphaGo Zero Paper.) The AGZ Paper shows that as more residual blocks and convolutional layers are stacked, the network becomes more information-preserving, and thus stronger. Deeper networks require a longer time to generate playouts, however.

5 Likes

Is it still on the list to change the move numbering for captured stones? It still shows move numbers for a capture or a fill in after a capture instead of the earlier stone that’s played there. So there’s some variations where I’m not sure where move 2 is eg jump to move 56 white to play move 57 and choose the blue move:

The move 2 gets filled in at move 23 by the looks of it. Just makes it really hard to follow the variation.

Edit: I should also say thanks for all the work so far! :slight_smile:

2 Likes

I agree that’s difficult, any suggestions on how to display that?

2 Likes

Sometimes there is a big discrepancy in the estimated win rate between Top 3 and full review.
For example at move 151->152 in this game

white to move:
For the Top3 display
image
image
And the full AI review of the same move
image
image

3 Likes

I was trying write some pseudo code but it was looking very messy so I’ll just try bullet points.

Problem: It draws later move numbers over earlier move numbers if the multiple moves of the sequence are at the same board position. Examples where this occurs are ko-fights, captures that are forced to be filled in by atari, or squeezes after a capture, throw ins etc. It can make it hard to follow the sequence because you can’t find move number 2, since it shows move 23 instead which was played at the same point later on.

Summary/Rules to implement, from the discussed cases below:

  • For empty points on the board that will be played on by leela in future, choose the earliest number after the current move number to display. If it’s empty now but was played on before it has been captured and we want to leave room for this to be played on again, and the move number it is.

  • For filled points, choose the latest number in the sequence up to and including the current move number. The idea is that if you’re ten moves into a ko fight, it doesn’t matter if move 1 is in the same place as move 10, you only care about 10 if it’s on the board right now, since you want to know who captured last, and when can you capture again. Similarly if there’s a throw in and a squeeze, you only care about the last number which actually corresponds to the stone on the board (since these could have opposite colors).


Points in the review by leela and what to do:

  1. Right at the start of a leela variation: No moves (barring the first which is necessary) of a Leela variation have been played:
    *Only draw earlier move numbers; if two moves are to be drawn in the same location choose the earlier move number to display. Example: the board and move linked earlier https://online-go.com/game/18286853 move 56 blue move.
    *If a leela move is to be played later in the sequence and there is already a stone on the board there from before this leela variation, maybe don’t display a number there yet. It will most likely be confusing to see a number on top of a stone already on the board but not yet played in the variation. Eg (same game) https://online-go.com/game/18286853 move 205 the blue move. Actually in this example move 1 becomes numbered move 8 immediately, which is even more confusing.

  2. Some Leela moves have been played and there have been some captures:
    *We draw move numbers as before for moves leela has put on the board up to the current move.
    *We stop writing a move number on the board for a stone earlier in this sequence if is no longer on the board. This leaves room for a faint number/stone that may come up later in the sequence.
    *If we’re in a situation where two moves have been made on the same point in leela’s variation we keep the latest number in the sequence for this point, if the latest stone is still on the board. This is because this is the relevant number for our current point in leelas variation.
    *We draw faint stones and move numbers as usual for LATER moves in the leela sequence to be played with the same rules as in point 1. (keeping only earlier numbers if again there are more moves to come which will be played upon the same point multiple times eg ko fight.)

  3. At the end of the leela sequence there’s no more faint moves/continuations to display:
    *Only show move numbers for stones still on the board as in point 2, (It’s just consistent I suppose?) but it probably doesn’t matter what number is here at the end of the variation if there’s no stone on the point in question.
    *Again only show the latest move number for stones on the board. Since we’re at the end these are more important than seeing that the 1st move was at so and so point if it’s been covered over/filled in later on.

1 Like