KataGo v1.15.x - new human-like play and analysis

Announcements
41

I’m not super concerned about cheating, i think vast majority of players play go because they love the game and enjoy the challenge. Out of the thousands of people playing on ogs every day, only a tiny fraction do any kind of cheating whatsoever.

However, im very hopeful that improved AI’s and other learning tools will increase the overall skill level worldwide. I think most amateur players do not have a teacher, and there is a ceiling on how strong one can become without good guidance. When kata and other free softwares get better at being tools for learning, the benefits are far greater than the risk of cheating ^^

7 Likes
42

Nobody said “anybody” was mocked. It was the concerns that were mocked. The tone of the discussion became largely dismissive of the botting concerns after the comment about “the negativity.” Raising concerns about a potential increase in botting is not negativity, unless one aims to be Polyanna or expects the assent of a host of “yes men.”

A dismissive attitude, in addition to the obvious dangers it entails in the world at large, is a passive-aggressive form of mockery, whether about people, places, historical events, political opinions, and a host of other objects, including the dignity of concerns about botting.

If I am able to draw out claims that people really are concerned about botting, then I will consider each individual claim as a success.

43

Ah i’m sorry, i’m not dismissal about the cheating concerns. I just feel like the positives outweight the negatives here

In general it feels like the level of human players has gone up a lot since 2016 thanks to AI’s, and i’m sure that trend keeps going on in the future. I think its only a matter of time until some western youngsters are able to end the dominance of CJK nations ^___^

2 Likes
44

It’s not that we don’t care about cheating, but let’s keep in mind that the OP has made a huge contribution to the go community by creating Katago, and before saying anything negative we should thank him for his work.

4 Likes
45

I never said you were. You took no part in the discussion between “the negativity” and my first post on this subject (#26), therefore you are not included among the dismissive.

46

Any new technology will be used for both good and bad.

– Change my mind!

4 Likes
47

You shouldn’t take it personally - this is the standard reaction for any new development at OGS :wink: :stuck_out_tongue_closed_eyes:

I mean - look at the massive thread just about the colour of a cross on a dead stone … and here we’re talking about the supreme cheating tool!

Of course my own immediate negative reaction comes from me being in the position of daily trying to weed out the AI cheaters - not something the average player is spending time on.

I am not at all optimistic that this can be used to help catch cheaters.

It is really really difficult to prove cheating even when the cheaters are using AI-like-AI.

While it’s not the only tool we have, the non-human-nature of moves is frequently the “determining factor”.

The fact that a sequence of moves might be “similar to a sequence that a human-like AI suggested” seems … pretty useless for detection.

If this net is something that can “recognise humaness”, that’d be an interesting twist, but it’s not obvious to me “an AI that can generate human-like sequences” this would be a “humanness detector” of any sort. It’s a move generator … and the moves can be more or less human. Honestly: this is the perfect cheating tool.

It’s worth noting that the only botters we “catch” (IE that we detect and are confident to prosecute) are those dumb enough to use the existing AIs that are non-human, and fail to disguise their use.

It is absolutely certain that there’s far more cheating going on that we don’t detect or can’t prosecute - both because of the already existing less-AI-like bots, and because anyone with a mind to do it it can disguise bot play.

Where this leads me is to think: actually we should forget about worrying about AI assistance.

This may be the actual utility of this release: the realisation that resistance is futile.

For a start: if you can’t tell the difference, then who cares?

The only reason I can think of for caring is if you want the “feeling” of defeating a human. Is that what we’re here for?

Or are we here for playing the stones in the best sequence each of us can? And against an opponent who’s track record shows they play the stones about as well. Why do we care how the opponent comes up with their moves?

I think we are being forced to stop caring about that, because we have no way of knowing anyhow…

13 Likes
48

I think consistency of the players level is maybe fundamentally important - for good matchmaking and for enjoyable games. So if someone was checking every move with ai then it would just like playing katago which is something many people do anyway.

But if they play some games with and some without then that messes up their rank. And if they pull out ai when they feel they are behind then that is a confusing experience for the opponent.

4 Likes
49

^^ so true. That’s a real problem for sure.

OTOH, if they have a human-like AI to do this with, and choose a level that is “a rank or two up” rather than “help, dan powers to the rescue” then maybe the experience is less bad (undetectable in fact).\

Also, if they keep doing that then they’ll rank up, and be “out of your way”.

50

Yeah even the functionality of classifying the move as likely to be played within a rank range, and then comparing that to the players declared rank should be a decent amount of information to generate even some stock messages.

Like “well done you found the best?/9d/ai move” or if the move is likely to be played only below a certain level but you’re above that level call it a mistake, and if it’s at or above your level don’t call it a mistake etc.

1 Like
51

Does this imply “we run the position for each rank of interest and see what is predicted”.

I guess that’s a cool idea, although the number of moves necessary to compute to “AI-check” a game is multiplied by the number of ranks to check. But at least that’s a linear factor :face_with_monocle:

A bigger problem is likely to be that for any given position, there is not just one play that is predicted.

It certainly will be fascinating to see how the “heat map of recommended moves” changes with rank.

1 Like
52

That’s a good point actually. If you have to run an instance of Katago at each level, each move, you probably don’t want to run like 30 instances for 30 or more ranks.

Maybe in that case you pick representatives like 20kyu, 10kyu, 1kyu etc so it’s narrowed down. Maybe optimising it where if it already matches 20kyu, no need to check the rest etc?

It’s also something maybe one could consider as like an advanced tier feature if it’s computationally expensive. Like you only get X of these per day at a certain subscription amount etc.

Just imagining what one could do, spitballing something :slight_smile:

53

Heh heh I wasn’t viewing this from the perspective of “an interesting user tool” I was viewing it from the perspective of “OGS’s efforts to contain AI cheating”.

From that perspective, we are forced to run as many of these as it takes to do the proposed detection.

54

Right right, I was more thinking of it in the other mentioned sense

where something like chess com has a way to try to explain the meaning of each move in its review

image

55

I was trying to train a model to guess the rank range from the move and only managed to get it to be +/- 5 ranks when considering the whole game - at move level it was more like +/- 10 ranks. Since most interesting games for cheating are in 20k-1d range then I’m not even sure if it would be better then always guessing the average rank of the player base :slight_smile:

Maybe someone can do a better job, but in general it doesn’t at least sound easy to detect if there are a few moves outside the expected moves for the players rank.

With AI analysis enabled we could maybe somewhat trivially check if a player tends to turn around losing games uncharacteristically frequently?

56

All AI moves are human, in the sense that each given AI move could be found by some strong pro. What is non-human is the ability to find them consistently throughout the game, or to play at much higher level than one’s rank.

It’s already possible to cheat while being undetected: just copy moves that you think you could have found. However at some point, cheaters want to continue crushing opponents, so either they copy higher level moves (like high-dan tesujis, or long fighting sequences with highly accurate moves), or they only cheat every other game (and then their playing level becomes very inconsistent, like bad shapes + misread ladders in one game, followed by efficient shapes + highly accurate reading in the next game).

With a human-like AI, 10k cheaters could also just copy the 5k bot, however the 5k bot can only play at about 5k level, so if cheaters want to win more, they’ll be tempted to copy the 1d bot, and then AI use will become obvious.

Note: the above is based on my limited experience of observing a small number of kyu-level probable cheaters. I’m not able to tell the difference between a cheating 9d and a non-cheating 9d.

1 Like
57

That might be true in some context.

But it’s not the sense I was meaning. I was meaning that experienced go players (Dans) can tell you “humans don’t tend to chose that, it’s an AI thing”.

In this context “human moves” means “a move that humans are likely to play”, rather than “a move that could be found by some strong pro”.

Yes, that happens all the time. Only people stupid enough not to realise they need to do something like this get caught for botting.

No, what will happen is they will rank up and then become 1d botters using a 1d bot.

1 Like
58

This is what I said that sent everyone into fits. I feel vindicated.

59

I think we should still care if users still report it, but I wouldn’t be surprised if we saw less reports.

60

I think very few moves fall into that category, like unexpected tenukis. What do “experienced players” really understand about AI moves? Strong pros also make surprising moves, unless you are pro level yourself, it’s very hard to tell whether “a human would play that”.

The following position is from the latest “Revue Française de Go”.

Capture d'écran 2024-07-22 082457

What is White’s next move?

Answer

This move was played by Ke Jie
Capture d'écran 2024-07-22 082513

If I saw that move in a kyu game I would certainly conclude that White is a cheater. The move is indeed amazing, yet it was found by a human.

Yes they could copy every move of the 1d bot (which would be very boring). Or they could only cheat part of the time, and then cheating will become visible.

1 Like