Why bots which can't read a ladder can play rated games?

I will start posting bot games on here to display how they are strange and more disruptive than useful for ranks. (unless anyone can think of a more suitable thread for it)

Here are some of the games by Amybot that ended in a timeout, today. There are more, even amongst todays games.

2 Likes

In another thread Eugene said: “However, a solid purpose to have bot games being ranked to provide newcomers with a quick way to get a rank. It’s definitely the case that some people like to do that with a bot before getting into ranked games with people.” So, I will also be checking what ratio of games played by these bots are actually against people who are new to the server.

So far, I checked Spectral-4k

As you can see, not a single ranked game against a ? in the last 50 games, while many ? people played it in unranked games. All of the last ranked games by this bot are against established players.

Some of those games are like this one: Partie amicale

3 Likes

That user might be borderline is definitely exploiting spectral, for it’s terrible ladder reading.

1 Like

Something to be considered is the motivation of the people who bring their bots online. If they cannot get rated games, not sure they will still enjoy to do it.

The bots “one can abuse easily” could be provided by OGS itself, if then there is a lack of them (ofc offering not rated games only).

1 Like

I join this idea. The quantity of games has to be well studied to stop quick enough before offering exploit opportunities. Then the restriction could be canceled for the “not that easy to cheat” bots.

On a tangential yet arguably related note, I find it surprising that there is such unanimity on the necessity of preventing exploitation of the bots. To give an alternative perspective, the general feeling of the Arimaa community would be that if there’s an exploit, that is a legitimate way to win against that bot, though players may be encouraged to make a second account for exploitative bot games.

One reason for this may be that for a large portion of Arimaa’s history, and in many ways still today, there was an emphasis on “Man vs. Machine”, and exploitative strategies were thus a means to keep ahead of the bots.

2 Likes

I am curious as to why bots struggle with the concept of ladders at times. It seems like the ladder is fairly simple and I just don’t understand why it can be difficult for even some of the strongest AI engines at times. Can someone provide me with some insight into why this issue exists and why it can be hard to correct?

1 Like

Humans can recognise that a ladder will progress only diagonally until it hits either stones or the edge.

Many bots can’t recognise the ladder shape, and so they cannot just “run the imaginary ladder” – they have to consider other options at every single step, which critically overdilutes their calculation.

3 Likes

To give an example, a human would never consider anything here but A.

A non-ladder-capable bot, though, cannot recognise the shape and will still stop to think about B, C, D etc. and to run Monte Carlo investigations on them.

With all its effort taken up by these distractions, it has no idea where the ladder is actually going to end.

5 Likes

Interesting. Thank you!

1 Like

I’m curious about why this bot keeps throwing games. Is it technical issues? Is it a special case of a troll admin running a sandbagging bot? Either way, it is another example of bots damaging the integrity of ranks.

1 Like

I think if the games weren’t ranked then exploit as much as you want. If the games are ranks you’re funnelling rating from people who lose to the bot to people who exploit the bot.

They’re really long. I also find it tough to read ladders sometimes, especially toward the end. Of course there’s some tricks to speed read if there isn’t any complications at the end.

As bugcat said though, they have to just read it out without tricks, and if they don’t look far enough ahead then they can’t tell its bad and keep playing it out.

1 Like

bugcat and shunuito pretty much said everything, but its mostly because there are so many moves involved. You can read the next 50 moves in a ladder and the stones are still not captured. If you have some sort of upper limit on how many moves a bot will read, long ladders across the board are ofen above that limit. Also if you assume that the opponent might play any legal move on the board at any given time, there are a ton of possible situations where you suddenly get 2 liberties and you can play any of those cuts.

4 Likes

Yep; that is the general consensus here, I was pointing out that it is not in the Arimaa community as far as I can tell. ranked is usually the default, and if a bot gets exploited, it should be ranked so that that exploit is accurately reflected in its rank.

Yeah and I agree that this kind of makes sense. If one put a cap on the number of rating points one can get from a bot for example but no cap on the rating the bot could gain/lose then maybe one could find an appropriate rank for bots including their exploits but not worrying about how it inflates other players ratings.

The only issue is that a broken ladder can really be a game breaking mistake. Like you might give up 30 points across the entire board with massive thickness, or give tonnes of double ataris and again massive thickness, so much so that bots want to resign afterward. The bot will also fall for the same pattern quite regularly so really they would accurately ranked at like ~30kyu if they can’t read ladders and everyone who can read a ladder was encouraged to exploit it. (ok I don’t know what rank you learn about ladders, but its one of the first things in every beginner book/intro).

Imagine putting LeelaZero at 30kyu because there’s some joseki it can’t read the ladder in but would definitely misread the ladder reliably most times and play it.

1 Like

Why is this bot playing ranked and accepting undo? I’ve seen people bragging about airbagging via playing this bot. An oversight?

2 Likes

amybot-ddk has been buggy for a long time, even before your post. Not only does it time out in a few moves sometimes (still doing it too, 親睦戦), but it also has a history of often stalling in the endgame, playing on endlessly (I am not sure if it still does this, however).

I agree with your idea to limit ranked bot games to provisional (i.e., ?) players. Many SDKs and even some low dans use the bots to rank down so they can sandbag, in my opinion, and because it is not provable, it can’t be stopped…

3 Likes

When a bot shows behaviour that would get a human player banned, shouldn’t that bot be banned?

3 Likes

Well I don’t think one has to ban bots or the bot accounts, however what one could do is restrict them from accepting ranked games.

I don’t know if that’s easy to do, but it would be an interesting solution to still let bots played ranked games, but preferably ones that won’t be too heavily exploited because of something simple at least.

It probably could be better to restrict the number of ranked games the same bot can play against the same user within a timeframe though.

2 Likes

I saw gennan mention this thread from last year.

2 Likes