New feature: Score-based AI review

Katago supports score estimation, which is often a much more human-understandable metric than the winrate. Ie. a bot will think a 5 point lead is 99%+ while for a human it’s much less of a guarantee. Seeing that the game was within a 5 point margin instead would be useful. Similarly if you’re ahead by a lot, the winrate is unaffected by blunders while they would show up in the score estimate.

Or, in an even game a half point mistake can be a 90% blunder.

I’d assume that this would be a fairly reasonable change or addition.

20 Likes

Yeah,

as discussed in December AI Updates this should be somewhere among the planned additions already. No idea about any ETA though.

9 Likes

It’s fronted stuff, in case someone wants to implement it. The API already has the score data.

9 Likes

Can you advise on where I can find documentation on this API, if I were to contribute the implementation of this feature? (Or else just dump whatever info is useful about the API into this forum :slight_smile: )

5 Likes

The best available documentation is the source code: https://github.com/online-go/online-go.com/blob/devel/src/views/Game/AIReview.tsx. There is no public documentation for aireview api.

/termination-api/game/${game_id}/ai_review/${ai_review_id} returns a json with all available review data. Coordinates start at the top left corner as 0-0.

updateHighlightsMarksAndHeatmaps should be the function which places the marks on the board. A substitution win_rate -> score should work.

The winrate graph is at https://github.com/online-go/online-go.com/blob/devel/src/views/Game/AIReviewChart.tsx

7 Likes

As a noob, my games have win% graphs that sway a lot, and get stuck in the extremes.

I wished we had the option to toggle between a win% graph and a score estimate graph.

Some might argue that score estimates don’t matter, but if I’m ahead by 40 points and make a -20 points mistake, I want to review it afterwards. The win% graph wouldn’t show anything because it assumes near perfect play from both parties, and being ahead 20 points is still a 99.9% win.

edit: Oh my bad. I see there’s another thread here Proposal: Score-based AI review

5 Likes

FIFY :slight_smile: Welcome to OGS Forums! Hope you enjoy your time here…

Hmm, is @flovo suggesting that this would be a pure typescript-react one-file feature? If so, I might actually take this opportunity to learn some front end and submit a PR.

5 Likes

unlikely to only be one file that needs editing, but yes, since the data is already there… displaying it is purely a typescript-react-css feature :slight_smile:

Here is our contributing guide :heart:

1 Like

I checked out the code and started poking around. I found one dev game that contained an AI review:
http://dev.beta.online-go.com:8080/game/6682

However, the json that this api endpoint returned didn’t have any scores, despite being a katago review:
http://dev.beta.online-go.com:8080/termination-api/game/6682/ai_review/586

Two questions:

  1. Is there an example dev ai review json that contains scores?
  2. What’s the best way for me to ask questions as I stumble my way through the code? Github issues or this dev forum?
3 Likes

I started a new review on the game. The new review (KataGo v1.3.3) should contain score data.

If you need a review on another game, just ask.

I prefer the forum.

6 Likes

Took me way longer than I expected, but here’s a tiny proof of concept!

16 Likes

Do you think the winrate in the bar on the top could show the current estimated score difference instead of the winrate when score estimate (mode/graph?) is on.

It’s just because neither the winrate or score estimate have a scale on the y-axis, so it’s hard to tell if it’s 10 points or 5 etc just from the graph.

Or maybe it could be a setting? It looks cool anyway. Especially useful when someone is at 100% solidly for a while.

3 Likes

I don’t think this is how bhyddens profile picture looks like and also says he is a mod so I guess it is a old picture

I think it might actually be his profile pic on the beta site, but not the regular site.

1 Like

Alright alright. Got a reasonable PR up here: https://github.com/online-go/online-go.com/pull/1164

Before

After

21 Likes

Looks really good.

3 Likes

Amazing, I am rather excited about the addition :slight_smile:

If I might offer some tweaking ideas, I think some margin around the toggle would go a long way, so it does not stick to directly to the surrounding letters and buttons.

Also some sense of the “scale” of the graph might be very usefull. Where it’s touching the top, is it +50 points?

Many thanks for the work

6 Likes

50 would have the advantage that those of us who prefer dozenal can just pretend it’s 40 (z) and be close enough for government work.

Updated margins:

image

14 Likes