The latest update on Prod also brought what is hopefully a resolution to the longstanding issue of “hey my win got annulled due to my opponent’s mass-timeout-annul”.
The change is that from now on when a person’s games are being assessed for annulment due to mass-timeout, we check the likely result using the AI.
If it is likely that the person was going to lose that game anyhow, before they timed out, then the result is _not_annulled.
Games timeout as part of a mass-timeout are only annulled if the person was likely to win, or if the result was indeterminate.
if during mass timeout rank is able to go down, but not able to go up, then people with a lot of ongoing games will become 25k anyway. Just 2 times more games are needed than if no protection exist.
if games where score is close enough are annuled, but games where score difference is big are not annuled, then rank change would be balanced and it would be difficult to force outcome in your favour.
(Prefatory note: I still use the older term, “serial timeout,” because seriality is the salient characteristic, not the ambiguous “mass.” Two games may be serial, but calling them a “mass” is bizarre.)
If it were implemented right, this fix could be a great improvement over the idiotic, serial-timeout rule. I used to support the rule until I realized how rare its legitimate manifestation should be (death, hospitalization, computer failure, shipwreck on a desert island, etc.) and how common the abuse of it is. The abuse consists of large numbers of cheaters, on the one hand, and large numbers of rude people who habitually escape without thought, on the other hand. Abuse of the rule became rampant when escaping from a correspondence game could no longer be reported as a violation. This happened when rengo was implemented, presumably because vast numbers of correspondence rengo players routinely escape.
I think this fix would be great only if it were semi-automatically implemented. The system detects a serial-timeout string in correspondence games and sends an alert to the mods who then carry out the evaluation. Is that possible? Even this sounds time-consuming, so I wonder whether the evaluation itself could be automated, which would be the perfect system.
I raise this issue, because of an inconsistency. How are the mods going to know when a serial-timeout string occurs? Since escapes (timeouts) are not reportable for correspondence games, few if any such reports will be received. If an exception is carved out for serial cases, I expect that few people will “get the memo,” and therefore the number of reports will still be negligible.
Question: does this fix include resolution of a bug that has occurred in recent years, whereby the intervention of a properly completed, live game fails to break the serial string?
If I’ve understood correctly, this is what will happen.
This is a great point and certainly needs sorting!
My view is that the proposal to determine both clear wins and losses while annulling the rest is a good one. My question is about the criteria. I think there should be a fairly broad definition of “indeterminate”. If we are using AI to determine “clear” wins/losses then there is obviously a significant risk that a game is seen as clear by AI which is far from it as far as the participants are concerned.
I realise this pushes things nearer to the status quo but would still be an improvement in respect of the clearest games. The definition of indeterminate could presumably be tweaked fairly easily in future I hope once we see how things work in practice. (I.e. if we still see a lot of complaints then we can tighten it or if we get complaints the other way then we can loosen it)
Suppose A times out against B.
A was clearly ahead if A still has >95% winrate after passing.
B was clearly ahead if B still has >95% winrate after A plays the AI move and B passes.
I don’t understand the point of this change: the (original, in pre-Nova merge OGS days, has it changed?) purpose of the correspondence time-out rule was to prevent hte rank of someone who disappears for an extended time from plunging, because then if they return they are sandbagging and upset opponents who have to give them unfair handis or get unfair rank changes (5d drops to 3k, 4d then loses to “3k” and loses loads of rank points, fake 3k wins the SDK tournament). Now their rank will plunge, though only about half as much (assuming they are losing about half their games). So as we aren’t getting the benefit the rule was designed to give, why not just ditch the rule entirely (which also eliminates the problems/abuses/complication it brings)?
The purpose of this change is to stop the experience of “whoa - that game I put all the effort into winning is now annulled”, and to also stop the (perceived or actual) use of this “feature” to escape a set of losing games.
Now that we have AI to help decide what “would have happened”, we’re into the territory of using that to solve the “serial timeout” issue.
stone_defender already pointed out that the partial solution to the “whoa” problem is perhaps unacceptable from the point of view of “what happens to folk for whom the serial timeout was a real thing?”, and we’re discussing how to do better.
E.g. after 10 moves, one player is on 60% win rate. In this timeout situation you’d want that have to be called for them? I’m guessing not. So maybe the question is after how many moves is a 60% win rate enough to call it?
Maybe it sounds a bit purist, but you probably shouldn’t be able to win by timing out?
I can understand the point about balance. It’s not like the player actually lost that many games in a row, so the way it statistically affects the rank probably doesn’t make sense. Maybe the half annulled they could’ve won a decent fraction of, if continued, so the streak of losses is unnatural in a way.
Would people be less disappointed if it didn’t say annulled but rather it just became unranked and showed that they still won?
Essentially, annulling on a large scale makes sense even for rating accuracy for the player (if they return), but people don’t like the idea that they won and it’s not being recognised is the point right?
I don’t really know myself where the priorities should be.
This idea of granting a win rating-wise to time-outers (which I predict would cause people being even more pissed than before - not me though) brings me back to a point I’ve made previously in related discussions: The way this is presented. If my opponent times out, I have won. Period. Nothing else is acceptable for me. I want to see that 1:0 (or whatever) in my game history. I want that point in the tournament. If you detect I was far behind and hence the stability of the rating system demands that the rating changes applied need to be as if I lost, sure, go ahead, I have no problem whatosever with that. But don’t tell me that I have lost. Ever!
If you abandon the fight, you lose. I think that’s more “purist” than “it has to go to counting or resignation”.
Otherwise it reminds me of that smartass mandra “you can’t lose if you don’t admit defeat”. Like the opponent is some prop and not an equal half of the game process.
Anyway, a timeouter should absolutely not get a win, under any circumstances. You leave the battle and you get rewarded on a technicality? Ridiculous, imho.
If AI is doing the work, then maybe something like this would make sense?
If AI consistently favours one side with a 90% or above for the last 25% of the game or more (percentage of moves, for example moves 75-100 and we’re in move 100) then that colour gets the win, if they are not the timeouter.
If the timeouter would win, it gets annulled.
Personally I consider timeout = loss, but I understand rank manipulation is an issue.
I understand this will go through lots, but I appreciate that OGS is working on this issue.
Yes, I’d want it called for them if there’s a 60% winrate after 10 moves. If you time out, you have no cause to complain “but I might have come back and won if I hadn’t timed out”
I think that determining a win-rate to call it in favour of the not-timer-outer is kind of easy - it’s not a “big deal”. If anything the question in that case is “how uncertain are we willing to be?”. Right now it’s quite conservative (you have to be pretty clearly winning by win-rate).
The question is what’s the win-rate at which we’ll let the timer-outer be declared the winner?
Honestly: it’s not going to be an algorithm that looks over the history of the game.
Or at least, this is my current mindset: that’s a crazy amount of work.
I’m picturing something like “a winrate >90% is one where humans typically can agree with the AI that it’s a win”.
As opposed to say 60%, where it strikes me as easily in the domain of “the AI may see the win but the humans don’t yet”.
Never imo. Timing out should never result in a win (unless the time out followed cheating by the opponent, in which case I wish the system allowed mods to correct the result to make the cheater lose, but for now we make do with annulment in that case)
