[poll] Balancing the new AI score estimator

Yes your insightful post starting with

and ending with

went over my head.

I think you’re taking ranked games on a go server too seriously.

5 Likes

You can say a whole bunch of nothing if you please, just don’t mistake it for logic. If this matter wasn’t serious, why the poll? Never mind, I know you won’t say anything substantial.

2 Likes

I believe it’s because it provides a service that lots of people enjoy and some people pay for or donate to.

I think it wants to keep as many people as happy a possible hence the vote.

Some places could just make unanimous decisions and not include the user base at all. I don’t think it should mean that ranking is of the utmost seriousness and must mimic competitive play over a real board.

I’m not saying we should allow a score estimator in ranked games, not at all, I’m just saying we don’t need to treat it exactly like play over a real board, so I don’t think that’s a reasonable basis for an argument and calling the whole process stupid.

The delete was a misclick on mobile. I apologise.

6 Likes

Sure, I can understand your sentiment. The server should serve its users. And all those functions can happily be employed in unranked. But ranked kind of fundamentally has nothing to do with emotions, or how happy the players are about their experience, it’s a battle on the goban, and as such, such be played in the spirit of the game as it was originally intended. You don’t go for a ranked game when you want to be non-serious.

Don’t take the real board metaphor too far. It just means zero assistance, because we are trying to gauge ability, not contentedness.

4 Likes

Because unranked is covering many different uses, from a tournament which don’t want to use the OGS rating to players who wants to be completely out of any constraints, it seems difficult to satisfy everyone without distinguishing this uses.
One solution could be to define different kind of unranked games with different presets

1 Like

This is not a real problem as all those functions can be disabled in the custom games creation options.

2 Likes

I want an estimator for the exact opposite of this.

I think you should have to show how all the borders get filled in, resolve any unsettled groups, and only then get a valid estimate of the score. The old estimator was okay for this purpose, since you couldn’t trust its resolution of any of the groups or borders.

Basically, it should be equivalent to hiring a 10-rank-lower player to do the actual counting for you, but you have to tell him how you think things are going to play out.

6 Likes

I can partially agree, but I think there’s a difference between playing seriously and competitively.

Some people have expressed elsewhere they don’t really care if their opponent uses albeit the old score estimator.

Let’s assume the new nerfed one is agreeable to a sizeable bunch of people who will either use it or just leave that setting on for auto match: does it actually matter to the people are opposed to it? The proposal seems to suggest you won’t be paired with the people that use it.

Of course there’s the worry it might indirectly affect your rating even if you don’t use it and don’t play against people who use it, or affect the rating system as a whole.

Maybe that’s a valid concern? I don’t know

Maybe I’m late to the party but… why not just make it so the score estimator only tells you a numerical estimate of the score? Why does it need to show anything on the board at all?

The only time I ever use the score estimator is when I want to get a confirmation that yes, I really should just resign. Maybe what I really want is a button that answers yes/no, “Is there any possible hope of me still winning this game?”

I just can’t see any legitimate use cases that involve showing strength on the board in any capacity. I vote for no score estimator at all in terms of displaying information on the board, but very much in favor of a more accurate purely numeric score estimator as an option for games.

10 Likes

I am sure I can win way more games if a well more stronger player informs me that I am leading or losing.

In fact, this is one of the review item in my post game AI analysis. I compare what I thought I was at various points of the game with AI winrate and try to understand why I got it correct or wrong.

It IS difficult.

4 Likes

Since, it wasn’t clear, I added another poll to the above post.

2 Likes

a purely numeric will still give most of the same information, just will take a few more sec to answer the “why a so big difference of points with what I was expecting to be”

1 Like

I think it invites more complexity than necessary. Just disabled all help for ranked, and enabled everything for unranked, or make it customisable for unranked.

Anyway, I think our agreement stems from the fact that we want different things from a ranking system. I would say rank implicitly implies the utmost seriousness, but it only now occurs to me now that some feel otherwise. There is no right or wrong answer to this question, but rather a question of what sort of environment you want to create and what type of players you want to attract.

In my experience, most go players take their current rank quite seriously, especially those that play on a frequent basis. This leads me to believe that it would ultimately be favourable for site traffic and retention to endorse this competitive environment. This would also include addressing other matters more seriously, such as detecting AI abusers and dealing with the volatile behaviour of bots which are allowed to play ranked games.

9 Likes

I guess so. The cases I’m imagining are where the game is effectively decided anyway and the score estimator is just confirming it. But I suppose in a closer game, or in a game where there’s a key point both players are missing, it could end up giving something away. Imagining a case where the score estimator gives +50 for white, then +50 for black after white’s move or something.

I’ve certainly had games where the AI revealed a pattern like that after the game.

Guess I just need to suck it up and count. :laughing:

I’ll register my vote for just disabling it altogether in ranked games.

9 Likes

Another thing I am interested in is how exactly will the SE be nerved? Are the pictures in the OP already produced by a automated system, or just a goal a wish to attain? What will be the mechanism?

Make the stronger estimation optional just for the unranked games and maybe allow it in the review?

3 Likes

I think that’s all completely reasonable: simplicity can be better in lots of situations; definitely there are considerations of what ranks should mean and what users it might attract or deter from the site; hopefully other issues will be dealt with in time like ai usage and/or whether bots should play ranked games pending discussion/vote.

I think there’s more folded into the ranking debate than just a score estimator if one really wants a serious ranking system as a strength estimate that’s equivalent to over the board settings. The different board sizes, rule sets, custom game times all factor into ones overall rating on OGS so even that muddled up the idea of trying to standardise what a ranked game means.

That might be a debate for another time or thread at least.

It was obtained by looking at the fractional output of the first image and showing the ownership as a full sized square instead of a square that’s proportional to the ownership confidence

7 Likes

To my opinion, this AI-based score estimator is good for spectators and for analyzing finished games, but it is too much for the players who are playing a game. Even the most restricted version of the proposal, nerfed and restricted to the current position, will give too much AI help to the players.

Imagine if I see that a group is marked as dead by KataGo, and I didn’t expect it. If it is my opponent’s group, I know there is a tsumego there, I will start looking for a throw in or something, and maybe I will kill it, all thanks to the hint from the AI.

Or imagine I want to invade an area. I check the nerfed estimator, and I see that the area is marked my opponent’s color. Then I know that KataGo thinks it is impossible to invade there, and I will do something else.

This is too much, we don’t want AI-assisted games, right? Or, if someone wants to play an AI-assisted game, that is fine, but should be unrated and marked as a special type of game.

I still think that a score estimator the players can use during the game would be a nice and useful tool, provided it does not contain any AI and applies a simple and stupid algorithm. It should just avoid the busywork of counting.

For example, a score estimator for the players during their game should consider all stones as alive by default, and allow the player to change the life-death status of strings by clicking on them. Empty intersections can be attributed to the color of the closer living stone. A player may have the option to add a red mark to an intersection, and all the intersections closer to a red mark are not counted for anybody (this in order to exclude moyos from the counting, if the territory is not solid enough). Such a score estimator does not contain any AI, and it only help players to count more quickly, without giving any information.

I think it would be very useful to have a simple-stupid score estimator for the players, some other servers have exactly that. Notice that the current score estimator already contain too much AI for my taste. It is considered acceptable because it is bad and unreliable, but I would prefer a score estimator without any AI.

But an AI-based score estimator that gives KataGo’s opinion about the position is really too much.

18 Likes

@LeibnizGW Would you consider just a number of the score, with no area markings, also too much?
For example W+50,5, nothing more?

I don’t disagree with what you said or anything, just expanding.