KaTrain: AI analysis/playing/teaching tool based on KataGo

[sanderl]

I’ve just released version 1.4 of KaTrain, a tool based on KataGo which you can use for playing against a wide variety of bots, analyzing your games, and playing teaching games with the AI.
It is aimed at focussing on your biggest mistakes in terms of point loss, rather than how your moves are different from AI style.

Latest release: https://github.com/sanderland/katrain/releases
Github page with manual:

sanderland/katrain

Latest features

The latest update introduces:

• Easy access to all the KataGo versions from within the program, including those that do not require a GPU.
• Visualizing the area of the board where your mistakes lead to a loss in a teaching game.
• An option for detailed point loss on the moves for those strong players who require that level of precision.

Playing against bots

KaTrain comes with a variety of bots based on a weakened KataGo, including a variety of fun playing styles as well as the ability to use KataGo’s maximal strength in handicap games. These have been calibrated by running the bots on OGS and by self-play, so their strength is easily configured. Overall, bots above 10k level play reasonably, for weaker players I suggest playing a teaching game against a bot of at least this level, rather than weakening bots to ddk level.

Regardless of what bot you play against, the analysis is always done using the strong KataGo AI.

Teaching games

Perhaps one of the most unique features is the ability to play teaching games. The AI will analyze your moves, and if your mistake is large enough, give you the chance to try again, with various hints that you can choose to take or not, such as the expected refutation, or area of the board in which losses are expected to occur.

Analyzing games

KaTrain also includes most of the functionality you may be used to from other AI analysis tools, with some extra options that come with KataGo. You can check the suggested moves in a position, improve analysis as needed, see expected territory, score, and win rate.
As the tool is based on the KataGo analysis engine, it is possible to explore variations at the same time as your game is still loading, making it ideal for a quick review after a game.

Feedback

I’ll update this thread regularly for new versions and updates. If you have any suggestions or contributions, I’d be very happy to hear from you.

[ayaros]

One feedback and two suggestions:

• feedback: I consider this sw one of the best out there for game analysis. Great work.
• suggestion 1: it would be useful to have a number or a letter on the green spots indicating the rank (from the best to the worst) of the suggested moves. This option could be actionable by a switch in the preferences and would be shown only during the analysis of a game.
• suggestion 2: it would be nice to have the highlight of the most important moves in the plot (only for post-game analysis) where you can trigger the winrate% or points lost/gained to be considered (like AI Sensei, or something like that).

[sanderl]

Thanks for the kind words ayaros.
As for your suggestions:

1. I generally consider any ‘large green’ suggestion to be as good as any other - with the new points feature you should be able to distinguish even tiny differences them. I’d like to encourage people to explore their own style, rather than push ‘the AI move is the only move’.
2. I think this is covered by show dots/thresholds to show them in teaching settings. Not being a website, the UI is always going to differ a bit.

[Atorrante]

Once I googled what UI means (user interface), it all started to make sense.

a user interface ( UI ) is the space where interactions between humans and machines occur. The goal of this interaction is to allow effective operation and control of the machine from the human end.

[ayaros]

@sanderl

I generally consider any ‘large green’ suggestion to be as good as any other - with the new points feature you should be able to distinguish even tiny differences them. I’d like to encourage people to explore their own style, rather than push ‘the AI move is the only move’.

I agree with your that one should avoid to remain stick to AI best moves. Moreover the clean interface you realized is really nice w/o numbers on the green spots. I consider this just a secondary option to follow in certain situations “the best sequence” for AI.
KaTrain is really fine w/o this feature too.

@sanderl

I think this is covered by show dots/thresholds to show them in teaching settings. Not being a website, the UI is always going to differ a bit.

For this, I must explore better KaTrain’s teaching feature. It’s not so long that I discovered KaTrain and it has been a really (positive) surprise.

[richyfourtytwo]

Very nice tool, thanks a lot.

There is one thing that I am missing though and that’s a clear next move marker in analysis mode. The way it’s done at the moment is very hard to spot for me.

Also I’m not sure I understand the options and features around analysing with different depth (playouts). Could you explain that? I have a pretty weak GPU, so using the time efficiently is somewhat important. Ah, and while I’m at this topic: I think it would be useful to have some kind of progress indicator when doing a full game analysis (maybe even when a single move is analysed). Atm I let it analyse a full game with 2500 visits (at least that’s what I think I am doing), it’s running since 2 hours or so and I have no idea if it is at 1% or at 99%, unless I’m missing something.

[kindaichi]

thanks for the great tool. this is one of the best in the future

[sanderl]

The green/orange light at the bottom shows when it’s ‘busy’ or not, but you’re right in that there’s no real way to see how far along it is. When loading an SGF you can look at the score graph loading though

[ayaros]

Some more comment:

An improvement would be to have portion of the screen resizable, especially navigation three panel and the graph one in relationship with the board. Currently you can hide/show each panel but the remaining space is not used by the others.

Very nice the feature that bring to the position just before the move that you double-click on. However a scroll bar in the navigation three would be useful too.

It would be easier reading the graphs if on the intermediate horizontal rules a small number indicate its value (currently only the extreme values are reported and the scale often change in the score and in the rank estimate graphs).

On my Mac, when I save the game and I reopen it, the AI analysis, the variations, and my comments are not restored.

What about adding a weird sound when you play a red/purple move in a teaching game?

Since it seems that KaTrain is oriented to learning (I like it for that), why don’t add a sort of game in the game when you play a wrong move (red or purple) in a teaching game? You would have a second chance for that move selecting only among pre-defined spots (A, B, C … ) each of which are some of the best AI top moves mixed with other few fake (wrong) moves. This to push the learner to think better to that specific move with some suggestions but not to exploit all the options coming from AI (it would be a more proactive approach). Actually you can do something similar turning off the top moves feature and trying yourself w/o suggestions.

Again all my encouragement to improve the product which is very nice.

[sanderl]

@ayaros

On my Mac, when I save the game and I reopen it, the AI analysis, the variations, and my comments are not restored.

Very nice the feature that bring to the position just before the move that you double-click on. However a scroll bar in the navigation tree would be useful too.

SGF is not a format which really stores AI analysis. Your comments should be saved, but not editable.

The navigation tree is in a scrollview, but kivy (the gui framework) is a bit mobile oriented at times, you should be able to drag the panel around to scroll though.

@ayaros

Actually you can do something similar turning off the top moves feature and trying yourself wo suggestions.

This is the intended way of using it, thinking yourself first, and then maybe using a territory loss hint, but never peeking at top moves, see e.g. the teaching game video.

[shinuito]

@sanderl

SGF is not a format which really stores AI analysis. Your comments should be saved, but not editable.

You can look at how OGS stores AI analysis when you download a game after which ai analysis was run. SGF can store variations (and comment winrate/score), which is more or less what the analysis is right? I would suggest not using the AB and AW options though for the variations, just use B and W with the usual tree structure, because programs like sabaki etc treat AW and AB as more like handicap stones or other board position edits as opposed to variations eg here Smart Game Format - Wikipedia

AB Add Black: locations of Black stones to be placed on the board prior to the first move
AW Add White: locations of White stones to be placed on the board prior to the first move.

Basically they won’t show up on the board the way variations usually would, eg Sabaki shows the next moves with a faint colour indicator/circle of where there next move variations.

[sanderl]

I know how SGF works, but took the comment as meaning the visits/detailed analysis are not really suitable for storing in it.
I normally show the PV in the comments for big enough mistakes, in a format that sabaki and gokibitz can analyze. Adding branches quickly becomes overwhelming (and then on loading, analysis of those branches would lead to more branches,etc.

[ayaros]

@sanderl

SGF is not a format which really stores AI analysis. Your comments should be saved, but not editable.

Dear @sanderl, let me amend what I wrote in my previous post. I discovered only now that the SGF with comments and variations are saved in a different directory and there is a clear link in the Load Game menu to get there . The comments are there as well as all my variations. My fault.

@ayaros

@ayaros

Actually you can do something similar turning off the top moves feature and trying yourself wo suggestions.

This is the intended way of using it, thinking yourself first, and then maybe using a territory loss hint, but never peeking at top moves, see e.g. the teaching game video.

I see. It seems that if I make a wrong move, it is refused and I have the chance to play again that move. If I made another bad move again, I have the chance to change it. The third time KaTrain accepts any move, even a red or purple one. It is fair enough at the end of the day.

I downloaded yesterday the latest version of KaTrain and I have seen that you added the option to show small numbers onto top moves to see the points lost. It is a welcome option imho.

Let me remark again that you are really making a program where AI can be used for learning and improving your Go skills. This makes KaTrain unique. Make AI more human-friendly is not an easy task but you are working in the right direction with fresh ideas.

[sanderl]

v1.5 was just released, some graphical updates and improvements to analysis.

[taatelikakku]

Thoughts about the older version:
I did play some 19x19 games with the calibrated rank opponent set at 1d.
It felt unnatural. A strange mixture of pro strategy and rookie mistakes. In the opening it makes a lot of tenukis and probes. In many games it first crushes me and then blatantly lets me win.
So then I set it at 3d and most of the weirdness was fixed although a 3d opponent is way too strong for me.
I tried the teaching mode. It forces me to play a very territorial style and give the opponent a huge moyo (no surprise, thats how strong bots roll). And then the game stops because there are 1000 possibilities to invade and I can’t find the only correct KataGo^TM magic sequence. Even if I give myself infinite undos it gets tiring to click through every possible empty point. So in this situation it would indeed be super helpful to have it show me options A B C D to pick from.

Edit: by forces I mean encourages, the auto-undo triggers easily when I choose less territorial moves

[sanderl]

I’ve found that the teaching mode can at some points allow you to dig yourself into a hole only katago can dig itself out of, but I disagree that it forces you to. The default settings can lead to all sorts of games.
As for calibrated rank, here is how it works, and you’re right in that it’s far away from mimicing human play.

[sanderl]

The 1.5.1 pre-release has been out for a while now, but not many people have gotten back to me on the new AI options.
If you can give them a spin and leave your thoughts here, that’d be great.

[sanderl]

v1.6 was just released with the new AI option, a katago upgrade, and a whole bunch of smaller features
My favourite one is probably ‘next mistake’ which makes for nice and quick reviewing.

[Keep_Strong_Ukraine]

Thanks for creating this wonderful tool!
I switched from Lizzie and have been enjoying using it for reviewing.

An idea for improvement:
How do I cancel/interrupt/pause the full game AI analysis or set it not to start by default? Would be nice to have such feature if it doesn’t exist atm. Some times when I open a game record, I dont necessarily want the full game analysis, but ie. just to go over some specific moves with AI.

GPU intensive apps like KataGo tend to slow down my window manager so I would like to have some control when to start it, ie. when AFK.

[sanderl]

@They_Live

How do I cancel/interrupt/pause the full game AI analysis or set it not to start by default?

The tool sort of relies on analysis being there (or pending) for every node.
I recommend just setting fast visits in general settings to something low, like 5 if you want really fast loads (and check the box in the load dialog).