Lizzie shows too many moves

So I bought a new laptop with a pretty good GPU Ninvidia GeForce GTX 1650 and it really does a good job of generating possible moves and thousands of playouts. Too good in fact.

My board now looks like it has some sort of disease where every point is coloured red as Leela has considered it as a possible move. That means that at some points int he game analysis it’s almost impossible to see the moves on the board as lizzie has completely covered it.!

I really almost never have to consider the 1x1 point! Anyone know how I can set lizzie to only show the top x moves?

1 Like

ok, after going through the settings for a 3rd time, I found this under the view tab. You can limit the best moves number from 0 which is apparently unlimited to whatever you want.!

And now lizzie looks a bit more user friendly.

1 Like

I actually do not think this phenomenon is too tightly connected with the GPU strength, but with the board position itself. When one side has such an overwhelming advantage all moves are either 100 or 0% the engine will just start freaking out and trying every move.

In a more balanced position you should not see this no matter what the settings. I always enjoy seeing it for some reason. Makes me feel like I made the bot sweat a bit :smiley:

6 Likes

By the way, if you plan to do lots of analysis of positions where the bot is stuck at 100% or 0%, especially handicap games, you might consider KataGo, so you can see score estimates instead of only winrates and get some idea of the magnitude of differences between moves. (Although Kata too gets less accurate when the advantage is extreme… and for less-experienced players, understanding the reasons for the differences can be tricky unless you have a stronger human player to ask for help).

4 Likes

thanks, that’s a good idea. It’s another level of detail because it’s still possible to lose a game at 99% with one bad choice. Having the score will give me some extra guidance on what I could have done differently.

1 Like

How do I get KataGo? I searched for awhile but the only runnable I found was Lizzie :sob:

(Specifically linking the Releases page, which most people not familiar with Github wouldn’t check. This link brings you to the spot where you’d download the release versions, as opposed to master branch source code.)

Open up “assets” for the downloads, and definitely read the directions for new users.

Reading on, this looks familiar I think I have done this before in order to get GUI for KataGo on Lizzie client.

Though I see no score estimation and the “Engine is starting” part takes aeon compare to Leela on Lizzie.

This might be a silly question… but you do know Lizzie comes pre-bundled with KataGo, right? You just click engine up the top and change it from LeelaZero to KataGo

Only the first time

Well I think that is my problem. I mean I have tried the built-in KataGo and also followed instruction to upgrade everything to latest but…

How do I see score estimate? I don’t see anything that looks like score estimation here:

It doesn’t tell you total score, just score difference… above the graph on the left hand side the current score difference is -8.6 meaning white is 8.6 points ahead of black image

On the stones, the bottom number is how much the score will change if you play there, on the blue circle the blue move will improve white’s score by 2.2 image

2 Likes

Ohhhhhhh!!! Tyvm!
Now I have more solid evidence for crying each time I lose!

6 Likes

No problem, happy to help torture :smiley:

3 Likes

KataGo feels pretty weird comparing to Leela. For example in this position it’s suggesting me to kick (it’s the latest network).

Also usually KataGo show a lot more move suggestion comparing to Leela, is this because of the cfg file? (I used the “auto generate config” feature from Readme)

Edit: Oh I just realized limiting nubmer of move as @willow1044 said will only work after reloading. I thought since it’s UI it would be applied immediately lol

Edit2: Not sure if this is because of

# Number of threads to use in search
numSearchThreads = 24

The Readme say higher numSearchThreads will be weaker (but faster), but then in the auto generate benchmark it says choosing 24 will result in +400 Elo comparing to 4. Not sure which one to believe in lol.

More search threads will be weaker holding the same number of playouts fixed, but if more search threads gets enough more playouts per second than fewer search threads, then more search threads is better overall. The benchmark takes all of this into account.

2 Likes

Oh I see, since Lizzie Analyze feature only allow limit by number of playout, I suppose it’s better to set it back to 4?

Why would you do that? More playouts on each move means more accuracy in the evaluations of those moves and less chance of KataGo missing things or making mistakes.

How many moves you want to set Lizzie to show is up to you, but there’s no reason to make the bot twice as slow (or something like that) by using fewer threads than the optimal suggested number. In the worst case just run the analysis for only half as long.

Kick is quite often a good move, if the side is uninteresting. For example:
https://www.alphago-games.com/view/eventname/leesedol/game/1/move/33

Pros also knew this pre-AlphaGo/AlphaZero and played kick at many appropriate times, but post-AlphaGo/AlphaZero we learned that the kick is playable even very early sometimes, and much more often than conventional wisdom suggested before then.

Judging by your screenshot, I’m guessing you have something like “color by winrate” selected. It’s possibly better to either leave it as coloring by number of playouts, or to hide moves with too-few playouts. Winrates for moves with very few playouts can be highly misleading - they might not be very accurate at all. But you can see what you prefer, perhaps.

If you want better evaluations for more moves with coloring that makes sense for wider ranges of moves, in KataGo “analysisWideRootNoise = 0.06” in the config and coloring by number of playouts is a possible choice.

1 Like

Oh, and it looks like you’re hiding the number of playouts. I strongly recommend you not do that. The number of playouts that a bot invests in a move should carry as much if not more weight as the winrate or score reported for the move, in terms of understanding the bot’s “preferences” between its top few suggestions. And it also lets you judge when a move has too few playouts for its evaluation to be trustworthy.

Mainly for its top few moves - if a bot only plays 2 playouts into A and 3 into B while other moves have thousands, it doesn’t mean much, it just means it doesn’t like either of the moves but it doesn’t tell you which one is “worse”. And with that few playouts, neither evaluation will be trustworthy anyways (you’ll need to instead play each move actually on the board physically and see what happens to the winrate, to judge them).

But among the top few moves, it’s a clear signal of the bot’s own preference, and you can take it account in addition to the winrate and score. Indeed, when you try to make an engine play a real game (rather than just analyze), playouts is such a strong signal that get the strongest actual play by starting with “play the move with the highest playouts” rather than “play the move with the highest winrate”.

This is because playouts also takes into account the bot’s “intuition” beyond just the judged score/winrate, and also because it’s possible for low-playout moves to be noisy and get high winrates just by “chance”, such that if they were evaluated deeper, the winrate could actually fall way back down (which is precisely why you shouldn’t trust evaluations with low playouts, but rather actually play out those moves if you want to find out just how bad they are).

1 Like

When you start a game analysis Lizzie ask you to choose a number of playout (10-50000), so I suppose while analyzing it doesn’t matter whether you choose 24 or 4, it will always does 50000 playouts. I’m just confused between the contradicting statements of README and config generator, so I tried to experiment with both cases.

About coloring by winrate vs playout, thanks that was informative, I think I’ll set it back to color by playout.