UPDATE: 2021-05-17T06:00:00Z
Thanks for all the feedback everyone. The new KataGo Score Estimator will not be made available in any form to players while playing a game, nor to spectators coming from the same IP address (so one can’t simply open another browser/private tab to have access to it). For the time being we will still allow players to use the old score estimator when analysis hasn’t been disabled, just like it’s always been.
Looking forward, I think the longer term answer might be to replace the old score estimator with a proper scoring tool for use in analysis and review mode, one that makes it quick for the user to mark out territory and the status of stones, but is “dumb” in the sense that it doesn’t try and figure out life and death too hard. Some balance between convenience and “too smart” will have to be struck there, we’ll work that out when the time comes.
Rational:
At the time of this update the Nerf poll received 962 votes from 762 people. Because the poll allowed people to select more than one response, I did some additional queries against the database to try and tease out what to do by separating out the players that wanted some form of score estimation from those that wanted no score estimator ever, from those that selected a mixture of both or “I don’t care”.
| 45% | Want at least one of the score estimator options but did not click "Do not ever allow" or "I don't care" |
|---|---|
| 36% | Selected "Do not ever allow" but not any other options |
| 19% | Selected "I don't care" and/or "Do not ever allow" and one of the other options |
This would seem to indicate that the majority of people do want some form of score estimator. However the next poll with 554 unique voters and votes conveys a slightly contradictory message:
| 50% | Want the score estimator disabled by default |
|---|---|
| 42% | Want a nerfed score estimator enabled by default |
| 8% | didn't care |
So people want some form of a score estimator, they just want it disabled by default, which somewhat implies that they don’t actually want it in their games perhaps.
The other trend I noticed reading through comments is that several folks, including myself, changed their opinion from “Sure whatever if people want it let them use it” to “KataGo’s estimation is too good for this, even nerfed”, where the reverse was not true that I witnessed.
I think what this tells me is we shouldn’t have any form of katago score estimation for players. However, there is certainly a large fraction of people that do like and use a score estimator sometimes for doing some quick counting that sort of stuff, so for that reason I wanted to keep what we had for them, at least until I can make something better-but-not-too-good for those of us who don’t like counting. There are a lot of problems with the old score estimator, namely it’s a source of confusion for beginners which I don’t like, so I will want to eventually try and figure out something better, but for now I think this is probably the right balance: Old score estimator during play when it’s enabled, just like it’s always been, new AI score estimator for kibitzers and end of game scoring, and eventually make some better tooling to replace the need for the old score estimator.
[end update]
For those catching up, we have a new score estimator that’s powered by KataGo. This is new system is significantly stronger than our old score estimator, so much so that it raises the question about if and when it should be available. Yesterday I threw together a quick poll for some feedback and as a result a lot of good ideas came up and much discussion was had . You can play with the new score estimator by using the score estimator after a game is over, or during a review or uploaded game. Currently if you use the score estimator during the game, the old “bad” score estimator is used.
In summary, what we’re trying to balance is providing a useful tool for players and spectators, but not make it so useful that it’s like getting a hint from KataGo as to where the best place to play would be. Here is my current proposal, subject to further modification based on feedback .
Proposal:
Nerf the score estimator for players of ranked games while the game is in progress
While a ranked game is in progress, the players of the game will have a different experience than spectators or after the game is over. Specifically:
- The score estimator will only be able to estimate the current position. Specifically, players will not be able to estimate analyzed positions (so a player can’t layout a few stones in analysis mode and click estimate to get KataGo’s view on what that would do to the game)
-
The score estimator will not show fractional influence, only a very hard view of predicted territory. You can see the difference illustrated in these two pictures:
This, I feel, reduces the amount of hinting that the player can get from predicted territory, while still remaining somewhat useful as a general counting / estimation tool. - For unranked games, the score estimator will not be nerfed. The score estimator can still be disabled entirely for these games if desired.
Spectators still have access to the “full” score estimator
- Spectators will have access to the “full” score estimator, meaning they can estimate analyzed positions and view the fractional influence estimations to facilitate good kibitzing.
- Spectators coming from the same IP as a player will have the same nerfed capabilities as a player, meaning you won’t be able to simply open up a different browser or a private tab to view the full score estimator. Players that work around this limitation and use multiple devices from different IP addresses will be considered cheating and thus will be subject to a ban.
With these modifications, the open questions I have are: Is the nerf a good idea, does it go far enough, too far, or is it just right? The second question is if we nerf the feature a bit, should it be available during ranked games by default?
Nerf poll
- The above nerf proposal - restrict score estimation to only the current board state (don’t allow estimation of analyzed moves) and do not show fractional influence
- Restrict score estimation to non-analyzed moves but show the fractional influence estimations
- Allow estimating analyzed moves but do not show fractional influence
- Full score estimator capabilities - Allow estimating analyzed moves and allow fractional influence
- Do not ever allow score estimation by players at all during ranked games
- I don’t care
0 voters
Presuming that the responses above indicate that we should allow some version of a nerfed score estimator for ranked games, should that nerfed score estimator be available by default for automatch and custom games?
Note: for automatch games, the default would actually be “no preference” - meaning that by default if a players clicks automatch, they’ll match with an opponent that has indicated a preference one way or another and that preference will be used - if both players have not indicated a preference, the default will be used, which is what this vote is about. So in any case, everyone should have a pretty easy time finding games regardless of their preference, what we’re voting on is the default experience for new players or players who haven’t specified their preference.
- Yes the nerfed version of the score estimator should be available for games by default (similar to how things currently are)
- No, the score estimator should be disabled by default
- I don’t care
0 voters
For unranked games, should players have access to the full score estimator?
- Yes, for unranked games let players use the full score estimator like spectators
- No, unranked games should have the same nerfed score estimator as ranked games.
- I don’t care
0 voters
