Proposal: Score-based AI review

Katago supports score estimation, which is often a much more human-understandable metric than the winrate. Ie. a bot will think a 5 point lead is 99%+ while for a human it’s much less of a guarantee. Seeing that the game was within a 5 point margin instead would be useful. Similarly if you’re ahead by a lot, the winrate is unaffected by blunders while they would show up in the score estimate.

Or, in an even game a half point mistake can be a 90% blunder.

I’d assume that this would be a fairly reasonable change or addition.

15 Likes

Yeah,

as discussed in December AI Updates this should be somewhere among the planned additions already. No idea about any ETA though.

8 Likes

It’s fronted stuff, in case someone wants to implement it. The API already has the score data.

7 Likes

Can you advise on where I can find documentation on this API, if I were to contribute the implementation of this feature? (Or else just dump whatever info is useful about the API into this forum :slight_smile: )

4 Likes

The best available documentation is the source code: https://github.com/online-go/online-go.com/blob/devel/src/views/Game/AIReview.tsx. There is no public documentation for aireview api.

/termination-api/game/${game_id}/ai_review/${ai_review_id} returns a json with all available review data. Coordinates start at the top left corner as 0-0.

updateHighlightsMarksAndHeatmaps should be the function which places the marks on the board. A substitution win_rate -> score should work.

The winrate graph is at https://github.com/online-go/online-go.com/blob/devel/src/views/Game/AIReviewChart.tsx

6 Likes

As a noob, my games have win% graphs that sway a lot, and get stuck in the extremes.

I wished we had the option to toggle between a win% graph and a score estimate graph.

Some might argue that score estimates don’t matter, but if I’m ahead by 40 points and make a -20 points mistake, I want to review it afterwards. The win% graph wouldn’t show anything because it assumes near perfect play from both parties, and being ahead 20 points is still a 99.9% win.

edit: Oh my bad. I see there’s another thread here Proposal: Score-based AI review

2 Likes

FIFY :slight_smile: Welcome to OGS Forums! Hope you enjoy your time here…

Hmm, is @flovo suggesting that this would be a pure typescript-react one-file feature? If so, I might actually take this opportunity to learn some front end and submit a PR.

2 Likes

unlikely to only be one file that needs editing, but yes, since the data is already there… displaying it is purely a typescript-react-css feature :slight_smile:

Here is our contributing guide :heart:

1 Like

I checked out the code and started poking around. I found one dev game that contained an AI review:
http://dev.beta.online-go.com:8080/game/6682

However, the json that this api endpoint returned didn’t have any scores, despite being a katago review:
http://dev.beta.online-go.com:8080/termination-api/game/6682/ai_review/586

Two questions:

  1. Is there an example dev ai review json that contains scores?
  2. What’s the best way for me to ask questions as I stumble my way through the code? Github issues or this dev forum?
2 Likes

I started a new review on the game. The new review (KataGo v1.3.3) should contain score data.

If you need a review on another game, just ask.

I prefer the forum.

6 Likes

Took me way longer than I expected, but here’s a tiny proof of concept!

9 Likes

Do you think the winrate in the bar on the top could show the current estimated score difference instead of the winrate when score estimate (mode/graph?) is on.

It’s just because neither the winrate or score estimate have a scale on the y-axis, so it’s hard to tell if it’s 10 points or 5 etc just from the graph.

Or maybe it could be a setting? It looks cool anyway. Especially useful when someone is at 100% solidly for a while.

2 Likes

I don’t think this is how bhyddens profile picture looks like and also says he is a mod so I guess it is a old picture

I think it might actually be his profile pic on the beta site, but not the regular site.