AI Review updates 2019-06-19

This game just ended.

We have the three top changing moves and I don’t understand something.

  • Top changing moves are all worth 0.0pp. That’s quite strange, isn’t it? I mean: I know that “top changing moves” aren’t correctly detected at the moment, but also the same zero value for all three seems strange.
  • The blue ball isn’t visible for two moves. Shouldn’t it highlight the next move?
  • Some pink moves are soooo bad. Down to -50.8pp! Does it make sense?

Also: at move 137, if E16 (starting a ko for life) can be a good move (+1.5) how can be “neutral” D17 (0.0) that forces white to connect?

1 Like

Yep I broke that, I think I’ll be able to get that fixed up tomorrow

In general, a lot of stuff with the top 3 are quite wonky at the moment, all of which should be fixed up starting tomorrow (probably)

6 Likes

I pushed up a mostly graphical update, now with 99% less potato stamps, along with several minor and not so minor bug fixes. More tomorrow.

11 Likes

But Mah stampz!!! :cry:

What does “network size” mean?

Network size means the total number of residual blocks and convolutional layers in the network architecture. (See the AlphaGo Zero Paper.) The AGZ Paper shows that as more residual blocks and convolutional layers are stacked, the network becomes more information-preserving, and thus stronger. Deeper networks require a longer time to generate playouts, however.

5 Likes

Is it still on the list to change the move numbering for captured stones? It still shows move numbers for a capture or a fill in after a capture instead of the earlier stone that’s played there. So there’s some variations where I’m not sure where move 2 is eg jump to move 56 white to play move 57 and choose the blue move:

The move 2 gets filled in at move 23 by the looks of it. Just makes it really hard to follow the variation.

Edit: I should also say thanks for all the work so far! :slight_smile:

2 Likes

I agree that’s difficult, any suggestions on how to display that?

2 Likes

Sometimes there is a big discrepancy in the estimated win rate between Top 3 and full review.
For example at move 151->152 in this game

white to move:
For the Top3 display
image
image
And the full AI review of the same move
image
image

3 Likes

I was trying write some pseudo code but it was looking very messy so I’ll just try bullet points.

Problem: It draws later move numbers over earlier move numbers if the multiple moves of the sequence are at the same board position. Examples where this occurs are ko-fights, captures that are forced to be filled in by atari, or squeezes after a capture, throw ins etc. It can make it hard to follow the sequence because you can’t find move number 2, since it shows move 23 instead which was played at the same point later on.

Summary/Rules to implement, from the discussed cases below:

  • For empty points on the board that will be played on by leela in future, choose the earliest number after the current move number to display. If it’s empty now but was played on before it has been captured and we want to leave room for this to be played on again, and the move number it is.

  • For filled points, choose the latest number in the sequence up to and including the current move number. The idea is that if you’re ten moves into a ko fight, it doesn’t matter if move 1 is in the same place as move 10, you only care about 10 if it’s on the board right now, since you want to know who captured last, and when can you capture again. Similarly if there’s a throw in and a squeeze, you only care about the last number which actually corresponds to the stone on the board (since these could have opposite colors).


Points in the review by leela and what to do:

  1. Right at the start of a leela variation: No moves (barring the first which is necessary) of a Leela variation have been played:
    *Only draw earlier move numbers; if two moves are to be drawn in the same location choose the earlier move number to display. Example: the board and move linked earlier krishnav14 vs. shinuito move 56 blue move.
    *If a leela move is to be played later in the sequence and there is already a stone on the board there from before this leela variation, maybe don’t display a number there yet. It will most likely be confusing to see a number on top of a stone already on the board but not yet played in the variation. Eg (same game) krishnav14 vs. shinuito move 205 the blue move. Actually in this example move 1 becomes numbered move 8 immediately, which is even more confusing.

  2. Some Leela moves have been played and there have been some captures:
    *We draw move numbers as before for moves leela has put on the board up to the current move.
    *We stop writing a move number on the board for a stone earlier in this sequence if is no longer on the board. This leaves room for a faint number/stone that may come up later in the sequence.
    *If we’re in a situation where two moves have been made on the same point in leela’s variation we keep the latest number in the sequence for this point, if the latest stone is still on the board. This is because this is the relevant number for our current point in leelas variation.
    *We draw faint stones and move numbers as usual for LATER moves in the leela sequence to be played with the same rules as in point 1. (keeping only earlier numbers if again there are more moves to come which will be played upon the same point multiple times eg ko fight.)

  3. At the end of the leela sequence there’s no more faint moves/continuations to display:
    *Only show move numbers for stones still on the board as in point 2, (It’s just consistent I suppose?) but it probably doesn’t matter what number is here at the end of the variation if there’s no stone on the point in question.
    *Again only show the latest move number for stones on the board. Since we’re at the end these are more important than seeing that the 1st move was at so and so point if it’s been covered over/filled in later on.

1 Like

I have suggestion:
If opponent resigned but I was 99% behind, I need ability to annul this game. I don’t wish to get extra rank for no reason.

1 Like

Just report the game, moderators will annul it for you. They will also investigate if the player who did that is sandbagging.

2 Likes

Yep, second what Eugene said.

It’s important that you (and others) report such games. Not just for the accuracy of your own rating, but without reports it’s easy for such offenders to go unnoticed.
There is a big difference between doing something once and doing something 100 times and we need help from people like you to actually report when these things are occurring.

We’re more than happy to annul games when the clearly winning player has resigned.

1 Like

You can just create automatic report of such games.
I don’t talk about sandbaggers.

Ideally would be cool if any game that ended in resign or timeout can be annuled by winner if winner has less than 50% by leela.

In blitz all you need to get dan is to never timeout. Just because most people are not fast enough. You will become airbagger and there is no way to stop it. Reporting every game is absurd. By playing very strange fuseki similar situation happens, people just resign.

If winner has 49% - who is winner is not clear anyway, so nothing changes if winner chooses to annul. If winner has 10% he should have right to EASILY annul this game - or rank will be disturbed. Report system is harder way therefore there will be more rank disturbances than in easy system.

Undo system exists. Why not go further?

2 Likes

Upon reflection, I can see that there is something to be said for this proposal.

However, no feature is “free”. Each thing that gets implemented means that something else did not.

We get very few requests from winners to annul their games, and yet it is EASY to do.

It is just a button click and type a two words “please annul”.

I think that if we got more of these, to the point where there was clear demand for this, then a feature might seem more justified?

EuG

Meanwhile, back on the original topic, the recent release of AI Analysis display appears to have knocked it out of the park - is that right?

It looks good aesthetically, and the results all make sense… I think?

4 Likes

So the stronger the strength, the more sophisticated the analysis. Giving that AI can not explain itself, and even explained, we might still not understand, is that possible that some of us are better off by choosing lower level strength AI on purpose?

I am a 3kyu and thinking to upgrade my support level. But before I do, I want to hear your opinion.

1 Like

I doubt a weaker AI (of the same basic architecture) will play more intuitive moves. More likely it’s just worse at reading ladders and the like. I would guess a stronger AI can never harm. But in many cases it may not help either, because us mere humans (and especially kyus) don’t understand anyway.

6 Likes

When there is only one good local answer, the AI shows how bad the other local moves would have been. I can undertand the value of knowing that the move played indeed was “the only move” locally, but for me it seems bit distracting. Example
31

Handicap games are still looking awful, even when disabling the AI review, the board is still littered with 0’s. Not a real problem, just an aesthetic one.

EDIT:
Also, could you automatically just show the territory when the last 2 moves have been passes? Without disabling the AI review, it’s littering the territories with nonsense 0’s.
31

6 Likes

first of all, thx to the devs for this awesome feature! :smile: all thats left for it to be a chickplayermagnet is for the analysis to work reliably.

it seems the analysis has indeed gotten better since the first implementation, but im not quite convinced yet. in some situations the output seems to be intuitively wrong.
for example, here i got a recommended move worth -16…

…just followed by a recommeded +17 move for white. the two moves would effectively turn the game if that were true.

i tried the sequence on my machine and leela doesnt think anything special is going on here, no significant winrate changes.

5 Likes