[UPDATE: This feature has been disabled for players of a game while the game is ongoing. While you are playing a game, the old score estimator is still used. After the game is over, or while viewing an uploaded SGF, the new system will be used.]
Starting now when you click ‘Estimate Score’ we will use the AI server cluster to perform the estimation instead of our old random playout estimator. You should see a lot of improvement in terms of quality of estimation and speed in which the result is returned.
We will now also use this system when auto-scoring player vs player games. You will of course still have the opportunity to correct any mistakes the auto-scoring system made, but the initial suggestions should be much better now.
I consider this update somewhat “beta” quality, but almost assuredly better than what we had before.
If and when you run into examples of when the auto-score or score estimation does silly things, please post a link here or send me a PM with the link to the game so I can try and tune things.
Note that this system is not currently being used for player vs bot games, which force an auto-score with the old system still. Once we have things tuned and things ironed out, we’ll switch that system over too.
mm yeah that’s an artifact of me hard counting area that’s not really territory yet - probably better to use the point estimate that KataGo gives back now, I’ll look at updating that.
While I fully welcome the change to improve the situation for scoring at the end of the game, and the eventual roll-out to auto-score human vs bot games, I am concerned about the upgrade to the score estimator for use in the middle of games:
The previous score estimator was terrible, so I don’t think too many people took it seriously and it hopefully didn’t impact too many games. However, with players now able to query a strong AI to evaluate the current position, it seems that the score estimator would provide a reliable (and potentially undesirable) form of strong AI assistance.
For example, sometimes my opponent might make a blunder that I would not realize and fail to capitalize on (essentially one blunder followed by another). However, even with a strong AI that only gives the score estimate (or even just win probability), that might tip me off that my opponent’s last move presents an opportunity that I should search for, and thus even a basic (but reliably strong) score estimator could provide unfair AI assistance.
However, the score estimator provides even more than that, by highlighting confidence about territory, which can guide my next moves. Further, it seems that one can even play out some moves with analysis on and have the score estimate judge those positions, so it provides assistance in judging my next move as well.
I urge that you consider disabling the score estimator feature for all ranked games.
Would a possiblity be to display the score estimate only but without marking up the board for (ranked) games in progress?
It seems to me that knowing the score is of some assistance and being able to reliably estimate score might enable people to resign rather than dragging out games. However being shown exactly how the score is derived from the board, hence identifying unsettled areas seems like rather too much help. Especially if you can try out variations and score those to deduce moves that settle the board in your favour in a given area.
I should say that I would favour disabling this feature for ranked games all together though as @yebellz suggests
Score estimator should exist or not exist. If it exists, it should work correctly. So, good work!.
But maybe it should be turned off by default - so people not starting rioting.
It seems like this kind of thing is not going to be solved by disabling the estimator until the end of the game, since presumably if we had passed at this point, auto-scoring would have revealed the weaknesses.
The score estimator has been a really useful tool for me to learn Go - trying moves out in analysis and getting quick feedback on their impact. I’m almost certainly over-ranked because of this behavior, but for me it’s enjoyable and has taught me a lot. I recently turned off my own analysis so that I can improve in reading “by myself,” but I wouldn’t begrudge an opponent who still uses it in our games.
If your goal is to attract new players to the game, I think this feature makes it a lot more accessible! I can’t speak to the concerns of the people who feel strongly it should be disabled.
I would prefer score estimate where you have huge brush tool like in Paint graphical editor. So it counts geometrical area of what you painted.
So no AI tells you additional Go information, but you can count very easily in geometrical sense.
Yeah! For me it’s just a counting tool, since in correspondence it’s not practical to count the whole board again before each move. I’d be just as happy if it required manually closing all the cutting points and marking dead stones to get a count. If this goes away I might actually have to start using a paint program.
First, big thanks to @anoek for what I think we can all agree is an excellent update
With regards to this:
I think the benefit of this (opponent resigns lost game instead of continuing play) is massively outweighed by the cost (they can reliably know the score and play accordingly without having to count ). I agree with the sentiments of many others - analysis should be disabled by default (especially for live games)
However, if there could still be a way to allow conditional moves just to keep correspondence games moving, that would be good
Caching could be better, I’m feeling it out to see if I should bother. It’s somewhat cached as is as each processing server will cache it for a bit, but the aggregation server does not cache it, so each processing server would have to process it at least once if asked to.
As it is still estimating by area scoring, its giving black extra points for each handicap stone and its also counting stones and eyes in sekis as points. I think there should be some sort of “warning” that the estimator is assuming area scoring, because vast majority of games are played using japanese rules.
Really? It still uses area scoring for Japanese rules games? KataGo supports territory scoring and a very close approximation of the Japanese life and death rules.
). I agree with the sentiments of many others - analysis should be disabled by default (especially for live games)