AI Review updates 2019-06-19

Hello OGS!

Our AI review system has been updated! Here are a list of the changes:

  • Review strengths have been increased by about 25%
  • The backend analyzer is now capable of reviewing multiple moves at once, this should speed up the analysis process on average, even with the increased strength.
  • An issue with analysis interpration has been fixed which would sometimes make it look like the move after a blunder was “brilliant” as opposed to the blunder being identified properly as a blunder.
  • The win rate estimation for the final move in the game is now available after all games
  • A bug where you would sometimes receive multiple notifications for a review ending has been fixed
  • Possible moves are now displayed as circles with the change in win rate displayed for each move.
  • We no longer show AI alternative moves that don’t have any further variations (these moves are not very well analyzed, thus shouldn’t really be considered as good alternative moves according to the AI.) The result is that the number of alternative moves may be lower, but the moves you see should have a much higher chance of being decent moves.

More improvements are still in the works, including having the top 3 moves for full reviews listed, being able to see the ghost stones after selecting an analyzed trunk move in a top-3 review, better top-3 move analysis, and some corner case bug fixes.

64 Likes

Cool! Thanks for the update.

Does this apply across all of the review levels? With games that have already been reviewed, can they be reviewed again?

1 Like

Yes to the first question. From https://online-go.com/user/supporter

19x19 network size Playouts Approximate strength
Supporter 15x192 250 Strong Dan level player
Kyu Supporter 15x192 1000 Professional level player
Dan Supporter 40x256 800 Strong professional level player
Pro Supporter 40x256 2000 Beyond most professional level players
2 Likes

Yep, all levels. They can be reviewed again, though it’s a little clumsy at the moment - if you select the “top 3” moves from the little review selector button, and then click “Full Analysis”, it’ll kick it off again with the new settings. I plan on making a better interface for that since the AI’s are still getting better as is hardware etc, but for now that’s the clunky way of making it happen.

4 Likes

I think you’ve missed a 0 off the playouts for “Dan Supporter”

The win rate being displayed makes it much easier to use. Thanks!
Are there plans on including the AI analysis when you start a review? I end up reviewing the game with the original in a separate tab so I can use the bot moves.

1 Like

Great stuff! The output percentages really do match the strong evaluation now. I re-visited my own test case blunder to confirm this. -> -64.5% for move 81!

There are however some leftover issues with the presentation of the data.
(and just to be clear up-front, I love the little circles and delta-pp, it’s very informative! :+1:)

  1. (not marked) The colors: the convention established by Lizzie is that Leela’s preferred pick is the blue move, which is on its way to become an established Go term. You don’t have to copy it, but satisfying user expectation is good UX practice :stuck_out_tongue:
  2. The engine version and network name have gone missing from the information popup.
  3. A lot of stray variations…? I thought these were supposed to be culled from the output.
  4. I still see a lot of these positive-percentage moves. This is a misrepresentation of the data.

If black can gain 17.3pp by playing N6, then the value of the current position is not 25.9%, but 43.2%. It is generally only possible to lose percentage points by committing mistakes.
As a rare exception, it can happen that Leela sees a variation more promising than her #1, but due to lack of exploration in the branch, she has no confidence in the result and thus prefers the “weaker” move. However, I see these positive moves far too frequently, even in the opening.

All these green moves create the impression that one could make gains by playing genius moves, which is just not true.

8 Likes

Thanks !

Of course it might be more useful to make it like that but can we have it as beautiful shades of green as before? Leela Zero is not Lizzie and it’s not OGS. The new AI feature looks like a kid worked on the board with a potato stamp…

Yep, me too. This part does not seem to be solved yet. (And imho figuring out this is a lot more important than discussing colours, though yes, I agree it’s not exactly beautiful atm.)

Dan supporter really is 800 playouts on a 40x256 network. This is a good bit stronger than 1000 playouts on a 15x192 network.

2 Likes

Thank you.
Thank you.
Thank you.
(Three responses because… oh, you fixed that bug.)
Thank you.

2 Likes

It’s an awesome feature that just got even better!

2 Likes

I retract that. I had been looking at an old analysis which still coexisted with the new one. The new one looks fine actually.

2 Likes

Truly an amazing upgrade!

My only concern – visually parsing the board. With the A, B, C, D options from previous version, it was a bit easier to navigate, and find the preferred move/response.

Given a smaller screen size (say, tablet or phone), reading the percentages and finding a few preferred moves is at times quite challenging.

Perhaps a sidebar list of the preferred half-dozen moves, listed in order?

That’s all. The overall AI is absolutely smashing!

1 Like

Ah, OK. I know little about AI and didn’t look at the network stats.

1 Like

I don’t see AI reviews for my 9x9 games. Is this limited to 19x19?

Yes, I believe it’s just 19 by 19.

1 Like

Still only 19x19, 9x9 and 13x13 are still planned

3 Likes

SOunds amazing, I’m glad I donated when you all put so much care into this

3 Likes