Improving the score estimator

Hi,

The score estimator is, to put it mildly, not very good. It’s very inaccurate and therefore quite useless until the end of the end game. With all of the new AI tools available to the devs, including KataGo which has a built-in score estimator, it seems about time it’s reworked.

Example: https://i.imgur.com/FYUrWDJ.png

In the above screenshot the score estimator estimates that black is up by 37 points (without komi). Black is actually up about 12 points. The estimate shouldn’t be that far off in a reasonably settled position like this. Some of the intersections that it considers to be black points are very clearly not.

OGS is, in my opinion, the best Go client in almost every way. This is the only feature I think is severely lacking. Get it up to par and people will have no more excuses not to play here :wink:

So I guess my question is: Are there any plans to improve the score estimator?

3 Likes

I’m all for improving the estimator as far as better functioning methods to mark stones/groups as alive or dead is concerned. Using (a good) AI for it is a no-go in my opinion, as that would be a much too valuable help. Not just help in the sense of ridding you from boring counting, but help in the sense of telling you which groups are alive.

6 Likes

Some more recent discussion around that

2 Likes

Yeah that’s fair, a KataGo score estimator would be too helpful. My point was more that, with such tools available to the devs, it should be easier to improve the score estimator than it was say 5 years ago.

1 Like

There is no reason to not include it in (unranked games) and in (ranked games that ended).

1 Like

In ranked games it’s probably better to have the old SE. It’s more like a duck than score estimator. Cheering you up.

4 Likes

I think the old SE could be improved functionally though. Like if you try to mark one stone as dead it just decides all stones of that colour should be dead. Trying other things like shift clicking to mark individual points (I think it’s shift?) end up just sticking and moving the mouse marks a trail of points etc.

I think you could improve its functionality instead of its accuracy, as people mentioned in the other thread, you more or less have to seal boundaries in analysis mode first before it’s any use. If people are going to do that anyway, why not make it a bit easier for them.

4 Likes

If you improve functionality it’s gonna become more accurate and useful too.

SE should give you a more or less random number.

3 Likes

There’s a balance to be found. It can’t be too helpful, but it should be more helpful than the current SE in my opinion. As is, it’s pretty much useless for anyone stronger than 20kyu. There is already a way to disable it along with analysis mode (which is a thousand times more helpful than any SE would be anyway) so I see no reason why it shouldn’t be improved.

I’m thinking of a score estimator like the one used on Tygem. It gives reasonable estimates even in relatively unsettled positions, but it’s not smart enough to reliably know the status of every group. The ideal SE would be even less smart perhaps, or, as shinuito suggested, it could force the player to mark dead stones themselves. That might be a nice compromise.

3 Likes

There is confusion in this thread about what the score estimator is and isn’t. Games have two separate scoring systems. The score estimator operates on Chinese rules regardless of the game rules. It is an estimator and is generally unreliable. The final score of the game is not calculated by the score estimator. It s calculated by an entirely separate system, which follows the game rules, and is called the “autoscore.” The autoscore is better than the SE, but still makes mistakes, especially about seki. And a recent bug has caused the autoscore to sometimes leave defined territory unscored.

The developer is indeed working on a replacement for the autoscore system. Whether that includes a new SE, I don’t know.

2 Likes

Improving functionality is a really neat idea I hadn’t considered.

Overall though, in unranked games, people are here to learn. Having a quality and accurate score estimator even if it revealed life and death status’s of groups would be ideal.

3 Likes

For me it will be unavoidable that OGS will have some day a SE strong like a today’s go AI.
The question will be how to fix a strong boundary to use it only for after game played reviews, like for AI.

2 Likes

The score estimator was originally introduced to check an item for OGS in server comparison tables. In practice, it serves to misguide beginners and stir drama.

Its faults are that it is too weak to give useful information and so strong that some consider it cheating.

Maybe it would help if all its estimations were automatically posted to the game chat, like a shared variation? Then it could always aid both players equally. My opponent would also be made aware that I am using the feature, which is only fair.

Then it would not matter if it was stronger and more accurate.

4 Likes

As a 16-17 kyu player, I find that many of my opponents will blindly accept obviously incorrect autoscores (usually failing to recognise dead stones). Perhaps they assume that the AI knows better than them.

5 Likes

You are absolutely right. Many TPKs and some high DDKs accept the SE scoring blindly, leading to confusion when the autoscore is different, to disputes when an opponent who knows better tries to correct the score on the board, and to consequent mod reports.

4 Likes

Let me use this tread to remember that one should develop his own capacity to count since it is a basic skill of any serious go player. In a real game you cannot push any button and your counting skill become crucial.

I strongly suggest this book to learn how to do it like a pro.

image

3 Likes

I think we have to improve it or at least the game end one.

What makes you think there is confusion? This thread is about the score estimator only. I see no problems with the autoscoring from a player perspective; If it’s wrong, there is always the opportunity to correct.

It sounded like shinuito was talking about the autoscore, but in rereading it now, I think I was probably mistaken.

Yeah I think the auto score is more or less fine. I think there’ll always be issues with players level and the auto score, whether they mark groups correctly or accept/change the autoscores opinion. I did in other threads raise a potential issue with something like katago scoring the game potentially giving away some information, at least depending on how it’s implemented.

But I did mean the score estimator, improving the functionality can just mean that when you mark a group as dead it doesn’t for some unexplained reason think that all stones of that colour on the board should be dead too. It could mean being able to easily attribute single empty points or regions of the board to one player or the other, possible not giving either player randomly some same as points, possibly even allowing it to count via Japanese rules rather than Chinese as an option.

I don’t think these are ground or game breaking, and I don’t agree that

Discobot can do that

@discobot roll 1d100

1 Like