Use AI results to decide Serial Timeout ranking results in Correspondence games

[flovo]

At the moment if a player times out in multiple correspondence games in a row only the first game is ranked and the others are silently annulled. The pros and cons are summerized in this post (some extensive discussions on that topic are linked below).

Now, with the new AI feature, at the end of a game the AI kicks in and calculates the chance to win for both players. So why not use that result to decide correspondence timeouts?

For example we could decide the game for one side if the win chance is >80% or so. If the game is not decided yet, anull the game (or maybe count as draw)

It’s probably not as easy as it seems, but I think it’s worth discussing its pros and cons.

Some pros are:

• The games would count against the rank, which could mitigate (un)intentional timeouts for lost games.
• Player who like to play fast correspondence games (time per move <1d) wouldn’t have to fear to disort their rating with timeouts of players who haven’t checked the time settings before accepting the open game challenge. (If the AI decition is applied to the first game as well).

Some cons are:

• Only possible for 19x19 at the moment.
• Players could time out of a clearly won game if the opponent didn’t resign. (Is this even a con? It’s impolite to let a game time out. But one could argue that not resigning a clearly lost game was impolite in the first place).
• One has to decide what the lower bound for a clear win is. (>70%, >80%, >90%, the AI would resign?)
• Handicap games have to be handled differently. At the beginning of the game the AI would give the game always to black. (Maybe only apply the rule after 150 or so moves?)
• The value of corner territory is valued much more by the AI than its value would be in low elo human play. The AI takes its superhuman ability of whole board reduction into account. In some cases this could result in games decided for one side, which would else end in favour for the other player because the inability of weak players to invade a big mojo.

I’m really interested in what you think about AI decision for cor timeouts (and to see it implemented if it turns out to be a good and doable idea).

---

For a general discussion about the timeout rule, please read the linked threads and join the discussion there if you have a new point to add that isn’t directly related to the proposed change of using the AI.

Escaping by Timeout

Unfair? No, I honestly did not, my appologies if it came out as somehow offensive You said you thought you proved “it would not mess up ranks”, and I merely pointed out what data I still felt lacking in the proof in the hopes that someone capable of doing such a simulation would repeat it including for those eventualities. And since I have serious concerns that we are bound to repeat the same arguments over and over, let me try to summarize the whole thread in simple points: Th…

Proposition: Loss by timeout in correspondence games should not affect rank

I’m going to claim that, philosophically, adjustment of rating is done in order to achieve an accurate estimate of strength rather than to punish the loser or reward the winner. When a player times out due to not playing turns for a days or weeks … that’s not really the same as losing a live game on time. I think the current practice of adjusting rank for timeout makes the OGS ratings less accurate. Once I entered a few tournaments and my rating ballooned up 3 to 4 stones due to timeouts by m…

What is the rule about time out now

I know there were lots of discussion and changes made. Last time I remembered, it is like the 1st timeout game will cause you to go down in ranking but not subsequent games, and if you time out in one game from a tournament you retire from that tournament. And I am not sure about what happen to the winner. Does the win count? Is this still the rule? If not, can someone make a summary ? And what exactly is “subsequent games”? Does it mean life time or games within that tournament or some other …

Escaping by Timeout

Could you please finally stop the “feature” that people can simply time-out their lost matches in bulk? The argument that it would “distort” rank if time-outs count as time-outs, but it does not distort rank if people time-out their lost matches is ridiculous. Some of them may not even do it on purpose, but simply do not know what move to play … because the game is lost. And no, I certainly do not want to complain about individuals, because there is nothing intrinsically wrong with timing out,…

New Correspondence Timeout Rule?

Hey there! Recently I have been picking up some evidence that correspondence games that are timed out no longer count towards the rank. It does not matter if the game isn’t annulled, it still looks as though there is no effect. Does anyone else have this issue or is it just me? Is it a bug? Or is this something that was recently changed that I am not aware of? Thanks for any help!

[_KoBa]

@flovo

• Handicap games have to be handled differently. At the beginning of the game the AI would give the game always to black. (Maybe only apply the rule after 150 or so moves?)
• The value of corner territory is valued much more by the AI than its value would be in low elo human play. The AI takes its superhuman ability of whole board reduction into account. In some cases this could result in games decided for one side, which would else end in favour for the other player because the inability of weak players to invade a big mojo.

Maybe only after 200 moves (when game is usually pretty much decided) AND if the point estimate is large enough. Bot may have 80% trust on its ability to maintain sente and win by 0.5 points, but that’s not the case with us lousy amateurs.

[shinuito]

I’m a bit skeptical about allowing a strong ai decide whether a game is won or lost. For instance I’ve played (live) games whereby I was winning with a probability of over 90% early on in the game so if Leela could play on it would win, but I, being roughly 10 or so kyu would not be able to play any of those winning variations it has in mind. At best (being generous) I’d say I’d have an advantage.

Actually in some correspondence games I would think I am losing (should lose the game), but my opponent makes a slip or something in the endgame or just toward the end and that turns the game around.

Long story short, maybe it’s not bad if both players are strong dan players who won’t make mistakes or endgame mistakes, but I don’t believe it should guarantee a weaker player like myself a win.

Example:

Suppose I want to abuse this and I get to a winning position (according to Leela say) in a correspondence game, but say I’m actually terrible at endgame [I mean I’m still learning so… :)], then I can just let my game timeout and win rather than play out the endgame and possibly lose?

[SanDiego]

I agree with @shinuito. Even at a low dan level, an AI 80% win probability in the middle game is far from guaranteeing a win.

Maybe we could get some stats from OGS, for example comparing the win % at move 100 with the actual outcome?

[Eugene]

I think this is a fantastic idea.

I think some concerns might be mitigated if the AI decided win is only handed to the person who did not time out.

So - if you have a correspondence opponent who bails, you should not be subjected to the AI saying “well you lost”. Actually, you might have been about to win because your opponent could not deal with you in the way an AI could.

However, most often someone who bails was losing in their opinion. And it seems entirely fair to hand the game to the person who kept playing if the AI says the position is winning for them. It’s simply agreeing with the implicit resignation of the person who bailed.

And… in the case where the person who bailed did so for non-score related reason (RL dramas etc) - well at least the result against them was tested. If they were really winning at the time of their RL disaster, then their game will be annulled as it is today. But if they were losing at the time of their RL-induced timeout, with this proposal justice will be delivered to their winning opponent.

Bring it on.

EuG

[andysif]

@flovo

Players could time out of a clearly won game if the opponent didn’t resign. (Is this even a con? It’s impolite to let a game time out. But one could argue that not resigning a clearly lost game was impolite in the first place).

a little fix could solve this problem:
the timeouter could only get a lost or draw game, while his opponent may get a draw or win.

@SanDiego

Even at a low dan level, an AI 80% win probability in the middle game is far from guaranteeing a win.

i total agree that AI is still far from perfect and any probability for winning is nothing more than a guess.

But we don’t need it to be perfect. call it an ad hoc fix to a problem we currently have no solution. any improvement is better than no improvement. yes maybe he could have turn around if he had continued, but too bad he didn’t. tough luck, and he himself is the one to blame for.

let the AI decide if he is losing the game. if he is, then a lost is not “distorting the rank” anymore.

[BHydden]

@anoek there is some great discussion going on here, I’d love to hear your thoughts on the idea!

[shinuito]

With the tweak from @Eugene and @andysif it does seem reasonable. I didn’t know before this post that multiple successive timeouts didn’t all go down as losses. So it does seem like someone playing multiple correspondences could potentially just let a bunch of losing games timeout. So there could be something in it, to let the ai call it but cap the timeout-er at a draw.

[stone.defender]

If you lost by timeout but “won by AI”, opponent should be asked to accept the lose or not.
Because there will be cheaters who escape as soon as they are ahead.
With that system it will be possible to decide winner in blitz too without worry.

[andysif]

@stone.defender

Because there will be cheaters who escape as soon as they are ahead.

As mentioned above, there’s an easy fix for this: the quitter gets a draw at best, even if ai says its a win.

[stone.defender]

I lost a lot of games where I was ahead. For example there was tsumego that possible to solve but I had no idea how to. So I should lose. But with your system I can just cheat draw and lose no rating.

[Eugene]

Only once. As with the current system, if you repeatedly do serial timeout, and get reported, it will be put to a stop.

You appear to be pointing out a small flaw in a broadly excellent solution to a longstanding problem… where the current problem is worse than the flaw you are pointing out.

[stone.defender]

@Eugene

current problem is worse than the flaw you are pointing out.

Then players who resign while ahead should have draw too.

[andysif]

@stone.defender

But with your system I can just cheat draw and lose no rating.

You don’t need to wait for the new system. You can already do that with the CURRENT system.

[martin3141]

I never understood why timeouts in correspondence games didn’t result in a loss in the first place.

EDIT: Especially since there are already a number of ways to deal with an unpredicted predicament:

• talk to your opponent and pause the game
• talk to your opponent and an admin to cancel the game
• use the holiday feature
  etc.

[Eugene]

Please don’t start that debate again here in this thread.

There are plenty of other threads where it is done to death.

This thread is about using the AI to decide them when appropriate.

[MrGorkajuice]

With the suggested tweak that the person timing out cannot be awarded a win, I like the idea. Without that tweak, it’s one of the worst ideas I’ve ever heard.

AI percentages tell us how confident the AI is that it can find a path to victory. That doesn’t mean that doing so is easy. I personally just came out of a game that swung from 98% black to 98% white, then straight back again in two moves (not a glitch), and eventually ending in a white win. As for comparing to the current situation, rewarding the person who causes the timeout with a win, regardless of level of computerized certainty, is really the worst solution I can imagine.

I respect our new robotic overlords and their opinions, but if my opponent times out, I’ll be damned if he gets the win just because Leela knows how to drive his win home.

Edit: Leela is a valuable teaching tool for those who spend the time digging into her findings. But bringing “referee decisions” into Go that supercedes well known rules (you timeout, you lose), and then in a shape that you cannot argue with? No thanks.

[Animiral]

@flovo

So why not use that result to decide correspondence timeouts?

The idea is kind of cool and OGS-unique, but you should still establish why it should be done. “because we can, why not?” is a good start - we don’t want to be stuck in our old ways all the time - but not entirely convincing on its own

What do we achieve? From your pros:

• mitigate (un)intentional timeouts for lost games
• players wouldn’t have to fear to disort their rating

I don’t play correspondence outside ladders and tournaments, or use “fast correspondence” settings. I have no experience if quitters are a real problem or not. If it is, seems like a really contrived way to address it.

In general, some players worry about their rank too much. Do you feel guilty for undeserved wins? We already have a very fast adjusting system compared to all other servers. It’s not a big deal.

The biggest downside of your suggestion is that it complicates the system and surprises people. If you think there are enough uninformed complaints now, then brace yourself for several of these:

I had no internet connection for only 2 days (never heard of vacation feature) and now my rank is down! How dare you give me “losses” instead of timeouts in these games that I could totally have won and did not resign of my own free will. This is an outrage and infringement on my human rights

[Eugene]

We get those sort of stupid complaints all the time. I love it when people threaten to sue the site over infringement of their rights. Yeah - your rights to use a free web site run by volunteers.

I would welcome the sort you suggest, if it solved the problem of “hey, I put all this effort into this game, and now the result was stolen from me”. I totally sympathise with this one, and I hate that we have no decent solution to it - until now.

[teapoweredrobot]

I like this idea and immediately thought that one condition should be that the win rate should never have been above some threshold for the opponent. I.e. if my opponent times out and my win rate at the time is 90+% and my opponent’s win rate has never been above 70, 80 or whatever then I think a win could be given. However if my opponent had a win rate above whatever threshold at any point (or maybe after a certain move number) then I think awarding a win would be dubious.
I notice in games that a comfortable win shows in the ai analysis as a consistently high win rate for me but often the win rate swings wildly where I’ve missed big or urgent moves.