[poll] Balancing the new AI score estimator

This point is important. For those who love the old estimator, it’s caused confusion because it uses area scoring (Chinese rules) to estimate the score, even if the game rule set differs. I’ve closed many mod reports from players saying, “Your software misled me to my detriment. Annul this game.”

8 Likes

i guess the simplest solution is to let the game creator decide. so a new option should be
“ai estimator for spectators”

No score estimation for players during the game is clearly the best option. I don’t understand the need or desire for it to be there.

4 Likes

Man, so much chat it’s hard to read it all.

Has this been asked and answered: why don’t we have the score estimator JUST give you the estimated score, not show anything on the board?

Why are we offering insight into the position on the board AT ALL when the player asks “what’s the likely score”?

5 Likes

As for the answered part - who knows. But it’s akin to having access to AI eval at any moment so I imagine not much better.

The new estimator looks vastly wrong to me though, even worse than the old estimator.

Yesterday I saw it for the first time, and it’s giving wrong estimation even for finished game +_+

1 Like

Well, ladder games, which are ranked, allow players to analyze during a game. That’s even more powerful than a good estimator imo.

But if you could have it only for the current position: if we gave player only ONE answer, and gave it well: the answer to the question “OK, what is the likely outcome at this point?”

… which lets face it is the actual question we are supposed to be asking …

… then that would seem to be the “right” outcome.

This whole “visualisation of WHY that is the likely outcome” far exceeds the bounds of the actual question, and is in the territory of “we’re showing this because we can and it’s cool”, rather than because its something that’s needed at that point during game play. Save that insight for after the game analysis…

2 Likes

I think being able to access score estimator using analyzed moves is a huge time-saver for everyone in the endgame. We can always count by hand but that process can be quite tedious and unfun. I do not want score estimator to be too strong or predicative in the early/middle game because it can lead to players having an unfair advantage.

1 Like

I think that the biggest timesaver would be to just present a list of AI suggested moves, each turn, and we chose one. We could even save time by letting it chose for us automatically… :wink:

11 Likes

one more reason to me to exclude SE. I like go game as a whole from the very first move to the last dame.

5 Likes

I’ve just changed my vote to, “Do not ever allow score estimation by players at all during ranked games”.

I freely admit that I have taken a different view in the past. I used to believe that score estimation could be a useful tool to newer players and I still believe that it’s “no skin of my nose” if others use it.

However; the low accuracy of the old SE can actually mislead and confuse beginners and I now see that if it is strong enough to be useful then it is too strong to be allowed in ranked games. Save it for unranked and teaching games.

12 Likes

Analyse is an help to visualize moves I still make with my brain, SE will give me assessments which are far beyond what my brain thinks… It’s quite a difference in here.
Not going to discuss analysis tools here besides that, just a reminder that ladder games are correspondence rated games. There are topics already opened on the use of analysis tool, although the new separation between SE and analysis could be a good reason to discuss again about the use of the analysis tool in a new topic.

1 Like

I think any server I’ve played on with a score estimate draws the score it thinks on the board. Because of that I think that’s what people would come to expect from an estimator.

Does that mean we have to conform to the norm, probably not, but I imagine this is what a score estimator “is” for the moment.

It would probably suit people needs in game, as mentioned in the multiple threads, to just have better analysis tools to count by hand - marks stones as dead, mark areas as belonging to one player or another whether that’s smartly decided or just with an area/rectangle tool, even if when one clicked and dragged the number and letter tool if it went onto the next number(letter) rather than dragging a bunch of 1’s everywhere that could be used as an estimator tool.

1 Like

It’s worth pointing out that we also have a hidden score counting tool, the “score breakdown”, which you can activate by clicking the area under the players’ names:

image

It’s extremely accurate but completely dumb, only counting territory that is entirely uncontested - the borders must be sealed and all dead groups killed. So to get a good count you need to clean everything up, being careful to play balanced numbers of stones as required by the ruleset.

And apparently after this new update it doesn’t even display the territory it’s counting anymore, but that’s probably a bug.

2 Likes

While a nice feature, KataGo has superhuman understanding of the game.

I expect that even with a low number of playouts and a discretised representation of player’s areas it still transports more knowledge than a dan player could extract from the board position alone.

A “fair” score estimator would only score territory without enemy stones inside, but would allow the player to select certain groups as dead and assign parts of contested areas to themselves or the opponent. It should not ever think ahead even a single step for the player, but should only do the counting for the player, and not assign any area to any player.

7 Likes

Don’t see how they are quite different. Being able to visualize totally changes the game. It makes reading and evaluating much much easier. I know because I used the analysis tool a few times in my games. Some L&D and endgame prolems which usually take me 30min can be solved in 5min, and you are more confident with your moves.

What I wonder is, why ppl don’t accept SE while in-game analysis exists for so long? With the analysis tool players can be 1 or 2 stones stronger and I don’t really think SE can help as much as the analysis (I might be wrong about this since I haven’t used new SE so far).

I understand the analysis tool has some advantages, such as conditional moves, which make correspondence games faster. On the other hand, I can’t find a scenario where enabling SE is superior than not.

Looks like I have contradicting opinions so to sum up:

  1. SE may not be as bad as many think.
  2. I can’t see why SE is good (for players).

About during game estimator for the players, there is problems with the current proposals:

  • full estimator gives way too much information’s about where katago want to play
  • the proposed “no fractional” is of little interest, as the image is mostly incoherent with the score, and thus only the score is relevant
  • only the score gives a lot of information (is the last move a mistake) and little interest in player evaluation as you don’t know what katago think is alive.

An idea for a score estimator that would be usable, and will fuzy more information than the current proposals. That’s still not perfect.

Take the estimation of katago, and only keep the high confidence points. Draw them, the other will be normal, as with the old estimator. Count according to this: one point for each confident points+komi/handicap correction.

Like that, what’s shown is coherent with the score, and the player can correct according to its own reading and estimation. The drawback is that you still have the insight that katago is sure what is alive, but I suspect it’s not that problematic. You also might see “evident” moves.

Edit: maybe combining katago evaluation with the “next to play” changing, and combining the result (only displaying when the 2 evaluations are confident for the same player) will allow to mask tesuji and L&D.

1 Like

Ways that SE and Analysis are similar:

  • They are tools that let us do what we otherwise would not be able to easily or at all do.

Ways they are different:

  • Analysis does not deduce and tell you things based on superiour analysis of the game. It does relieve you of visualising, but you still have to deduce what it means

  • Score Estimator uses a knowledge of the game vastly superiour to yours to show you things you otherwise would not have seen.

3 Likes

Since there used to be bans on sites like wikipedia and youtube in my country, the default was to use VPN for a long time. I looked up VPN statistics, it is pretty common overall as well.

I agree if someone wants to cheat, they’ll find some way to do it but opening a new window with a VPN is not much of a deterrent in practice, once we assume people might cheat when it is convenient.

That said, I don’t see how to improve on the current situation regarding this. I doubt there is a sure way of identifying users who want to avoid ip detection to cheat in some way.

In the end, there has to be some trust that people are willing to respect others to some extent.

3 Likes