I like the idea of taking previous fluctuations in win estimate into account, I imagine it’s in some way taking into account the players strength relative to leela. My only issue is you’d never win handicap games by timeout in this instance, ie if you play someone and give them stones. Leela usually wants to resign because you “passed” so many times at the beginning so the winrate is probably over 90% for the first five to ten moves I imagine. Might depend on the handicap.
I think the various ideas about how exactly to handle this suggestion should be considered a giant warning in terms of how dangerous ground we’re aproaching. Even under ideal circumstances, determining the winner of a finished game assumes agreement regarding life and death, and the rules include protocol for settling disputes.
Being declared loser of a game by an automated AI proces that we generally trust to make excellent moves, but that we do not always understand, and that we cannot query for a proper explanation, falls as far from the spirit of Go as we can possibly get.
In my opinion, determining actual winner and loser MUST be left to the rules of the game itself, and running out of time is an established losing conditions.
There are secondary considerations though, where I think OGS is more entitled to make their own decisions - the mechanics of tournaments, ladder and rating. Using AI score as a mechanism that rules a timeout loss as being “in bad faith/resignation by inactivity” and rewarding the winner (by timeout) accordingly is something I can get behind.
2 Likes
I think we should try to get the wording straight. A timeout is already causing a loss and I don’t think anyone is suggesting to change that ever. The only thing under discussion (as far as I understand it) is how these timeout losses should be handled with respect to ranking (which has nothing to do with rewarding players).
The original post can absolutely be interpreted as timeout games going to whomever AI believes is ahead. Several posters makes explicit mention of the loser by timeout getting a draw if ahead by AI.
But it’s a good point. We do need to get the wording straight.
In case anyone actually wants to let AI score determine recorded winner, I think this is a HORRIBLE idea, should be dead on arrival.
In case we are “only” discussing whether or not to adjust ranking - the timeout person already lost. Honestly, if Leela thinks the winner is a favourite by even 70% or so and more than 20 moves are played, sure, give him rating and ladder spots and everything that comes with the win. I already don’t agree with many of the mechanics intended to safeguard scores against timeout (sorry for bringing that up again Eugene, but it’s relevant in terms of figuring out which debat we’re actually having here).
1 Like
That’s not exactly the point.
For example look at life and death situations in DDK or low SDK games. The players’ assessment can be very different from the AI assessment, It’s not about turning the game around, it’s about agreeing on the status of the group.
Fair enough.
Wrong.
When part of a series of timeouts from the player, all subsequent timeouts are not ranked as a loss, they are annulled. That is the whole point of this thread and discussion: finding a solution for the unpleasant experience that this rule causes.
2 Likes
You also appear to be unaware that in a series of timeouts, the games are annulled not declared as loss by timeout (as the rules would otherwise dictate).
The “unfairness” of this to the opponent of the escaper is the whole point of this thread.
Please don’t ask why the rule is this way in this thread, please instead go and read how it actually is and why it is that way in all the other threads about that topic. It’s honestly been done to death in those threads.
This thread is about solving the unpleasant experience caused by this rule, by calling the game using the AI.
I have updated the title of this thread to emphasise the scenario under consideration.
1 Like
with all due respect, this idea is totally against the new suggestion.
what we are trying to fix, ie, the problem with the current system is: people have incentive to quit when they are down because they know they can get away with a draw.
with the proposed new system, they will get a loss if they quit when they are down. but if intermediate/previous win rate is taken into consideration, it becomes an insurance, and they know they will get a draw if they were ever ahead once during the game, and everything revert back to the current system.
so no, i don’t think intermediate/previous win rate should be taken into consideration. only the last position should.
1 Like
I also already mentioned that you couldn’t just see if the win-rate was above some threshold previously when deciding outcomes, like in handicap games. So yes there was issues with the suggestion, but I took it more as that maybe taking some previous fluctuations into account might be more useful than just the very last win rate estimate.
It’s just that the difference of one move can cause wild fluctuations for us kyus. So if we timed out, it could very well (effectively) be like flipping a coin as to whether to award a draw or loss to the timeout-er/escaper, that is if you only go by who’s ahead at the very last move of the timeout. It’s just something to consider.
Right but if the person who timed out when they were behind … it’s arguably because they were behind. It doesn’t matter that it flipped back and forth: the problem is when someone bails because they perceive that they are behind and can get away with not losing rank for it.
Therefore, if they bailed when they were definitively behind, even if they were ahead before that, it really does seem fair that they should take the loss - as a disincentive to escape as well as justice when escape happens.
The only argument for why the serial timeout rule exists in the first place is because of the uncertainty about whether the person receiving the rank should get that rank. If a bot thinks that at the time of escape the person remaining was in a winning position, then that has to remove that concern or make it so small as to be negligible. If you get someone into a position where the bot says you are winning, and they then escape, then you probably deserve that rating (arguably, not in opening maybe. But by mid-game - for sure).
2 Likes
You start with ‘wrong’, but after that precisely prove my point: We need to get the wording straight.
Yes, we are discussing how to deal with timeouts with respect to ranking.
For me, when people are saying on this thread that timeouts (sometimes) are not counting as a loss, for some not fully informed readers that might seem to imply that timeouts are counted as draws with respect to tournament standings or personal win/loss stats also, which they are not.
Ranking changes are not the not the only consequence of losing/winning. That’s absolutely all I’m talking about. Other than that I like this discussion a lot, even though I don’t have a strong opinion myself.
The scope of this conversation was clearly stated in the first sentence of the first post.
This is what we are talking about: it is unambiguous.
I have updated the title to also reflect this.
EuG.
2 Likes
Ah, perfect, I didn’t notice the the title change, Thanks!
1 Like
With the current system do people mass timeout to keep their rank? I wonder if it’s a big problem.
I… don’t know how I missed that indeed very clear and unambiguous phrasing.
In that case though, it’s hard to have this discussion without taking into account ones opinion on the current serial timeout mechanics. I still maintain that under no circumstances should the loser by timeout be awarded a win - not even when the game is one in a series of timeout.
Based on this, I still have issues with the phrasing “AI calling the game”. However, if we’re talking about a “bad faith bulk timeout detector” that only has the mandate to turn annulled games into losses for the loser by timeout, then it would be dishonest of me to not admit that I am biased.
Despite the arguments for, I still think the current implementation is worse than always calling timeout losses as full losses. I am not looking to revive that discussion - you have your opinion, I disagree but respect that this is your site - but clearly my opinion on this matter pushes me in the direction of very lenient criterias for “bad faith detection”.
Besides that, there’s a different argument that if we let an AI that we already have some issues understanding make any types of decisions like these, at least the implementation should be easy to understand - number of moves played, AI certainty above X - nothing more advanced than that.
1 Like
Note that I personally don’t agree that serial losses by timeout should be annulled.
However I do understand the argument for this, and that it is unlikely to change.
Which is why I’m all the more enthusiastic about a proposal that mitigates some of the bad effects of a rule that won’t be changed.
I also agree that a person who timed out should never get a win. I wonder if @flovo would consider updating the proposal to this effect, or whether your proposal remains as “decide all games by AI where possible”.
4 Likes
some say there lacks evidence that support the claim, while others insist that they have experienced them first hand. their believes are rooted so deeply in their heart it is almost like a religion.
And… once again… that topic has been done to death. If you want answers to that sort of question, see the links in the original post.
OGS uses the rating to calculate the number of handicap stones, for several pairing modes in tournaments, to find players of equal strength in quick match finder and limit the range in witch we can challenge other players for ranked games.
This gives me the impression that the rating on OGS is supposed to reflect the player’s strength.
If we introduce an asymmetrical rule like the proposed win/annul but never lose, we would possibly disturb the ratings of correspondence players. Therefor I’m favouring either win/lose/annul or annul everyting.
Either way it could be possible to detect players, which are exploiting the rule to keep their rank by tracking the ratio of (timeout a losing game)/(total timeouts) and bring it to moderator attention if the ratio is unusual high.
I don’t think that’s true.
If we adjust a player’s rank based on a timed out which game the AI said they won, then that adjustment and resulting rank is correct. Player X beat Player Y, therefore Player X should be considered stronger than we previously thought - their rank accuracy improves by this action.
If we don’t adjust a player’s rank when a game times out ad the AI thinks they lost, their rank accuracy is no better or worse. We just missed a data point.
At the moment we miss all such data points.
If we chose to accept the timed-out-player-lost datapoints, the remaining player’s rank accuracy improves, AND their experience of playing at OGS improves (compared to having that victory denied due to the serial timeout rule experience).
EuG