Mass-timeout-annul fix

If the person who timed out wanted to claim “there was still plenty of play left in that position! AI may have thought my opponent had an edge, but it was still anyone’s game at our level!”, then they shouldn’t have timed out

In this case you either

  • Don’t want this current change in place (arguing that all games should be annulled) or
  • You’re arguing that the timer-outer’s rank should be hit by the games they were going to lose, even though this is not being compensated by the games that they were going to win (which you are arguing can never be awarded to them.

Right?

This. The games they were losing should be losses for the one who timed out, and the games they were winning should also be losses, but I’m okay with this compromise of making those they were winning annulments

1 Like

I think this should look back at least a few moves. When a L&D situation is misjudged by both parties (and it can happen to all, not only DDK), is not unusual for that 90% to pingpong.

4 Likes

I find that hard to argue with, despite the work involved :crying_cat_face:

1 Like

Can I clarify the problem we are trying to solve?

It seems there are several in two categories

  1. Problem which the system seeks to address
  2. Problems which arise due to the system

It strikes me that there is no perfect solution so out of the problems which is the most important?

If we can identify a list of problems we could have a poll!

E.g.

  1. People abandoning the site messing up others ranks
  2. People abandoning the site messing up their own rank
  3. Wins being annulled (probably need to clarify if we mean wins where the winner was winning or wins where the winner was loosing or both)
  4. Moral issues about people not following through on their commitments
  5. Others??

It strikes me that maybe we need a whole different solution. Like a new category of timeout. Win by abandonnent or something

2 Likes

The things that matter :+1:t2:

3 Likes

Actually there might be two different streams of problem solving.

One is:

  • given that we are “required” to have some sort of “serial timeout rank smack mitigation to cover real life impacts forcing absences”, how can we best set up that system, taking advantage of AI now that we have it

The other is

  • actually what do we really care about, do we actually need a serial timeout solution at all etc.

I’d prefer to focus questions in this thread on the first one. Past debates tell us that we are required to have some sort of mitigation for serial timeout due to life events: that’s not going away.

So more focussed questions are around “what, if anything, should we do about the fact that the change that just went in dilutes the protection against real-life serial timeout events?”

That’s the problem that this thread is about, from my perspective.

1 Like

In which case, I think my preference would be to have a system as Richy described and with obvious loopholes closed (like logging on to the site prevents it triggering) and with a slightly higher threshold in the run (like the annulments only start after the fourth game for example rather than after the first as now)

I think it’s not right to try and determine hypothetical results using AI. This I suppose was what was behind my reasoning previously that determining results should only be done in the most clear-cut cases.

So my system would be:

Serial timeouts after the fourth game timed out after the last site visit.
And these games would be shown as won/lost by abandonnent.
And this would mean that they just show as wins/loses, are unranked but are worth tournament points etc

This doesn’t really solve the problem of “I was clearly winning and I didn’t get my ranking points” but I think that’s a gap we need to accept because I think doing “better” has other drawbacks which make it worse overall.

That’s my view on

3 Likes

I think 2 is bigger than 1, so the rule should be ditched entirely. And I say that as someone involved in its introduction.

5 Likes

I’d say we want the optimal balance of

  • rank estimation is as accurate as possible (highest goal)
  • games that are quite settled as to result get scored and ranked
  • cheating and abandoning rewarded as little as possible

Say a timed out player was winning with a 99% probability. Can that player be assigned a LOSS while simultaneously the rank is adjusted for a WIN?

Recently someone pointed out that rank changes shouldn’t be a reward or a punishment. Rank changes are an attempt to achieve a good rank estimate. I’m on board with that. And wouldn’t it be cool if the uncertainty on the rank estimates became smaller.

How about for handicap games where the win rate is about 99.9% for black until that last move where white wins…

3 Likes

Just to be clear, I favor the abolition of the serial-timeout rule. Its legitimate occurrence is rare and the abuse of the rule (consciously and unconsciously) is massive. The test that led to its retention, as I understand it, failed to take into account the abuse; consequently, that test was invalid IMHO. However, as GAJ notes, the fundamental rule is here to stay.

Thanks to @teapoweredrobot for suggesting the best solution to the question I posed:

This seems like an easy way to counteract the bug I mentioned, where activity on the site fails to break the serial-timeout string. This is something that can be implemented right now, without waiting for resolution of the other issues in this thread, @GreenAsJade.

Since we are stuck with the rule, the main purpose of the fix, as I see it, is to mitigate the effect of abuse of the rule. This abuse victimizes innocent opponents and does more damage to the rating system than the problem that the original rule intended to address. The problem now is to balance the effect of mitigation with retention of the rule’s purpose.

If we take this balance as the goal, then awarding wins to the time-outer is counterproductive. Not only would it be repugnant to most of the community, but the abusers of the rule would continue their abuse: why should they play on if the system is going to give them the win without any effort on their part?

The balance is achieved by awarding probable wins to the opponent of the time-outer and annulling everything else.

The question is thus reduced to establishing the criteria for the wins. They should certainly include factors other than predicted win percentage, as @teapoweredrobot has argued.

Remember: whatever criteria are set now can always be modified later if some tweaking is needed.

1 Like

So we’ve already got the OGS variant where if you are winning loads and annoyed your opponent hasn’t resigned yet then you can pass 3 times to get a system assigned win. Are we going to add “let the game time-out” to the things you can do to get your win in unusual ways?

3 Likes

Credit where it’s due, this is not my suggestion. I think it was @Gia who proposed this solution.

More generally I agree with the thrust of @Conrad_Melville 's analysis. But then we are left with the problem that @stone.defender mentioned. The original problem is not solved.

So then I’m wondering if we shouldn’t just revisit the need for the rule in the first place.

2 Likes

We could keep the rule but apply it manually only by request of the mass time outer.

This hasn’t been shown. In fact, I have gone through my game history in the past, and legitimate occurrences seemed more prevalent. Game annulment due to extended absence of opponent - #64 by benjito

Edit: to be clear, I do not consider my brief search a sufficient analysis, but I think it’s important to point out that no one has performed an analysis to conclusively show there is widespread abuse of the serial timeout rule.

1 Like

With the Catch 22 that if you ask for them to be annulled then you are clearly able to access the site and therefore they should not be annulled…

2 Likes

I’m saying you would get a loss. And your rank would be adjusted based on your actual play. It’s based on the idea that we want the ranks to be accurate as a primary value. YMMV

Win in terms of gain rating points but lose in terms of words in game result seems like a win to me if I care about rank.

But this is all so unnecessarily complicated. Part of the reason this rule was introduced is old OGS used Elo so it took along time for the 3k timed-out 5d to return to 5d. Now we have glicko with uncertainty parameter that may not be the case. Or just tell the fake 3k to make a new account, or don’t let him enter handicap tournaments until his rank is sensible again.

1 Like