The integrated AI Review feature for OGS

This will be a truly wonderful addition!

1 Like

Yep :slight_smile: Lots of worker nodes analyzing the games together.

Correct, only after the game is over.

Shoot thanks! I think that’s because you weren’t logged in, I’ll fix the error message.

4 Likes

I honestly didn’t even know that there was a beta site. This is amazing! Do I need to make a new account for the beta site? It won’t let me login with my username/password. That said, congrats on the new groundbreaking feature! <3

2 Likes

Yep you’ve gotta create a beta user, it’s effectively a complete and isolated system from the main site

2 Likes

This sounds awesome anoek! I figured when I saw you post in the LZ github a while back you were working on something neat. :slight_smile: The way lichess has stockfish integrated I always thought was really cool.

4 Likes

I knew it was coming as soon as @anoek posted this :slight_smile: :

3 Likes

Hello! Very nice new tool! I think it’s the first Go server that has one like this, it well be amazing!

I’m wondering what’s the difference between the short review and the full review. Can you tell me?

4 Likes

The short review only does the top 3 moves, the full review does the entire game

5 Likes

Huge thanks!

2 Likes

This is amazing!

My only (totally unsolicited) suggestion would be to consider using current 15-block weights for LeelaZero, such as network edb61bc2 (aka LZ-15b-202). This is because you’re not gonna read any ladders at 200 playouts, and 15x192 weights can go up to 550 playouts using the same resources, improving accuracy at slight cost to overall strength.

7 Likes

Can sfg’s be uploaded upload for review, or does the game need to be played on the betasite as well?

Can’t wait to try it as see how it helps my game improve.

2 Likes

That’s an interesting observation. I was planning on doing more playouts for the production site, though I’m not sure how many are necessary to read out ladders to be honest. Would 800 be enough, and would 800 on the 40 would be better than the ~2k of the 15? Any thoughts on what some good limits / values would be?

No support for uploaded games yet, I’ll be watching our resource usage and seeing if we can afford that.

2 Likes

How can you tell the difference between a blunder and a good move

Yes! I’d avoid using a 40b net at 800 playouts and instead suggest a 15b network at 1000 playouts or more. I set up a classic cross-board ladder below (joseki used: http://josekipedia.com/#path:peqcocqeqfpfoeqgrfrergqhpgofphrh):

40-block network 223 needs 1400 playouts to see that the ladder is good for White (>50% win rate):

40-block network 211 (OGS beta) actually fares better in this test. It needs 1200 playouts:

15-block network 202 (my link) does significantly better, kind of shockingly. It needs 900 playouts:

Based on this test, I feel that anything under 1000 playouts would be unreliable for reading ladders, and it sounds like playouts in excess of 1000 are only achievable using a network smaller than 40b. So I’d go with the linked 15b network (or its successor, when that comes).

3 Likes

H6 was a -10% blunder. I found it by clicking on the graph at a place where the line dipped suddenly, then clicking the left and right arrows till I was precisely on that turn.

You can see on the board that Leela thinks white should have played P10

3 Likes

This is very nice ! thanks a lot anoek for working on this.

Looking at the board from GaJ’s post, I have a few questions.

LZ’s moves are indicated with both letters and background color/intensity; how is one supposed to read that ? (i.e. B looks better than A in the example, but then why not always use label A for the bot’s recommended move ?)

Would it be possible to show the bot’s proposed variation as a branch in the analyze mode panel ? (not sure how much it’d help, but maybe do it just for the few largest missed-opportunity moves in the game ?)

If you click on the bot’s variation it does load into the analyze mode game tree.

I’m not sure why the whole lot isn’t loaded immediately :slight_smile:

Oh, wonder!
How many goodly creatures are there here!
How beauteous mankind is! O brave new world,
That has such people in ’t! :star_struck:

4 Likes

The color intensity indicates how much a position was explored, and the letters indicate the order in which the bot would choose to play.

6 Likes

That seems a bit wierd - I thought that Leela will play the move that was most explored? But A is not coloured at all?