📝 Make reviews great again! 💪

[Groin]

@tonybe

A given review should (in aggregate) be no shorter than 800-1000 words (about 1.5 to 2 pages of printed text in 12 point font on a blank page - with the understanding that these will be spread out over many comments across various important points throughout the game. Conversely, try to limit your reviews to under 5000 words, as this might overwhelm the reader.

I will be very satisfied if someone points at me some key problem in my game in 10 words only.

@tonybe

for SDK / DDK games, volunteer reviewers can only review games from players that are 8 kyu weaker than them

Strange idea. Maybe simply limit reviewers to be no more 3 kyus lower as the player ( no mistake here, a reviewer 12k could contribute his opinion on a 10k game) if necessary to put some limits but do we need them really? In worst case I 'll be delighted to read a revew from a 25k on a 2d game if this ever happens!

@jlt

I’d say a review should contain between 5 and 20 variations.

I will be very satisfied if someone shows me just 1 killing variation on a group I thought it was alive

Besides, I like short reviews of my game in which a stronger player shows me the exact moment where I made a very bad failure compromising the future of the game. That may involve between 1 to 3 variations only.

[ronin3]

@tonybe

for SDK/DDK […] 8 kyu weaker than them. Dan reviewers need only be 1 dan stronger

I might agree on the spirit of the suggestion not on this clear-cut rule - as it would mean a 1 kyu can only review a 9 kyu, but can get a review from a 1 dan. Maybe a linear interpolation that implements this (1 dan difference at high dan, 10 ranks difference at beginner ranks, and scales proportionally inbetween)?

[Groin]

Honestly I just don’t see the usefulness of these limitations. Players will auto censor themselves as usual. Just don’t forget to show their ranking if you are afraid of “pollution”. And I do respect myself any point of view even if something so strange as a weak player comment a strong one happens.

If you really want to limit by strength then install a system of filter that the user may select himself (which will be at least better as something predefined and hard coded)

[Groin]

About reviews between both players after a game;

You’re surely right that both could be wrong and then the opinion of a stronger player may be very fruitful.

Now I still consider that it’s a big loss to not review together after a game finishes. Why? Because if the game was seriously played you have 2 people who thought, “discuss in silence” for a quite long time and who could finally exchange their ideas and eventually check together their validations helping each other if the good spirit is still here.

I understand ofc that’s not always something obvious on internet. (Already not always obvious IRL but usually not a so bad experience for me at least)

[tonybe]

@Groin

Now I still consider that it’s a big loss to not review together after a game finishes. Why? Because if the game was seriously played you have 2 people who thought, “discuss in silence” for a quite long time and who could finally exchange their ideas and eventually check together their validations helping each other if the good spirit is still here.

I think this would be a great idea for the “Tournament + Review” idea that Adam came up with - a tournament where each player is committed to the idea of participating in a review afterwards. If you’d like to make that idea a reality, I would suggest putting your efforts there, and provide suggestions for how to get that up and running.

However, I was trying to come up with clarifications and concrete ideas for the REQUEST A REVIEW QUEUE - so that we could give the moderators something concrete to work with.

@jlt

The number of words is not a good way to measure the length of a review, the number of variations is more relevant. I’d say a review should contain between 5 and 20 variations.

Great idea - again, I’m not focused on any particular set of suggestions or limits, I just wanted to come up with some examples and see what would work for the community. I like the idea of “at least 5 variations, but no more than X” -

@ronin3

Maybe a linear interpolation that implements this (1 dan difference at high dan, 10 ranks difference at beginner ranks, and scales proportionally inbetween)?

Yes, I like that idea a lot. I feel like at higher ranks, a given player would be able to benefit from a peer review from someone a rank or two above them (i.e. a 1 kyu player would still benefit from a 1 dan review - thoughts?)

So yeah, how can we make a simple linear interpolation that works on clusters rather than a different one for each rank?

so like

• any dan - 1 rank above
• 1 kyu to 3 kyu - 1 dan or above
• 5 kyu to 9 kyu - reviewer should be 4 kyu stronger?
• 10 kyu to 15 kyu - reviewer should be 6 kyu stronger
• 15 kyu to 25 kyu - reviewer should be 8 kyu stronger?

Again - these numbers are just suggestions - all this could be automated under the hood so that at a given rank for a reviewer - only those reviews that fit that criteria will show up in your queue etc. Thoughts? Suggestions?

Again - my goal is to keep this discussion focused and come up with concrete things we could take to the mods and say “Hey, let’s try this and see if it works?”

[tonybe]

Another idea: in order to volunteer to be a reviewer, a user would need to submit a previous review that they’ve done, and that would be read by a volunteer from the category above them (in the clusters I’ve loosely suggested above) and then approved or denied with notes / suggestions.

Thoughts?

[Tschej]

Your idea reminds me of: OGS Teaching Ladder

There is also a nice guideline how a review should be done.

[tonybe]

@Tschej

Your idea reminds me of: OGS Teaching Ladder

There is also a nice guideline how a review should be done.

Wow! I did not know until now that the OGS teaching ladder existed

Yes, that is very handy. They’ve already gone over so many things we’re trying to come up with from scratch

I might reach out to the two people running that and see if they would be willing to modify it to include some of the ideas we’re coming up with here

[Fragile_Pākehā]

@Groin

If you really want to limit by strength then install a system of filter that the user may select himself

This is consistent with the established practice where a player sets limits on opponent strength when they set up a game.

[Groin]

I didn’t know that there is a kind of revival. To me it died some years ago, but it seems that someone is trying is best to make it alive. It started if I am not wrong as the KTL -KGS teaching ladder and extended here with this group. And it had quite some success then. It’s a long story.

[tonybe]

@Groin

I didn’t know that there is a kind of revival.

Well yeah, that’s the whole point of the thread - if you look at the OP - AdamR posted 12 days ago asking for suggestions for how to foster more of a review culture here on OGS / OGF.

I made the suggestion that - rather than trying things we’ve tried before like Tournaments and Ladders - that trying something new and having a DEDICATED QUEUE that was a part of the functionality of the site, where people who wanted their games reviewed could get into that queue be connected to a dedicated pool of volunteers willing to provide those reviews.

We then began drilling down into that concept, and coming up with details to make it more practical, or try to avoid issues that people had raised, etc.

If you’d like to assist in that discussion - we welcome your input. If you’re just here to say that this idea is silly or it probably won’t work - well, thanks for your input, but we’re going to keep plugging on. Cheers.

[Groin]

Did I say the idea is silly? You said yourself you didn’t know about this “OTL” so I add a few I know about it. I added because imho those people who devoted some of their free time before in reviews may have something to exchange here .

Thanks for the resume of the thread but I didn’t really need it, neither did I need to be checked on my intentions here. I hope you’ll make something even better and more lively as the OTL you just discovered.

[AdamR]

Finally had time to read through the thread, so here are is my attempt at summarization, and my thoughts:

As far as I am concerned, the worst possible scenario is that the feature gets introduced, people ask for reviews, and get none. As such, I would be very careful about imposing too many strict limits, I think we need the least ammount of friction for possible reviewers.

Reviewer limits:

• As suggested, I think it only makes sense to have only stronger player review a game. I too would be careful about starting too strict. Honestly I would be happy with 5 ranks for TPK, 3 ranks for DDK, 2 ranks for SDK, 1 rank for dan. Or even more lenient. Hopefully people will auto-censor themselves or we can increase the limit based on how it goes, but again I think the priority should be to make the feature as accesible and as used as possible.
• I would start without any test reviews or anything like that. We want and NEED volunteer reviewers. Don’t make it harder for them.

Review limits:

• Honestly I am cautioning against any significant review limits (words or variations), it will just get in the way. I have seen great reviews with only comments about direction of play with at most “consider A instead” etc. text only, no variations needed. Or in fighty games sometimes variations are all you need, no lengthy explanations needed or useful. You would also need different rules for different board sizes, and it gets very complicated and annoying very quickly.
• We might consider some completely basic limit like at least 2 variations or 20 words just to weed out random clicks, and discovery attempts, but I would not go any higher.

Reviewee (is that a word? ) limits:

• I think it also makes sense and would be useful to limit the number of review requests for a single player. Say 1 per week to start with? Again we want to make sure most games get some attention, and would be bad to have players spaming endless reviews. We can adjust later based on supply/demand.

Feedback options:

• To address some of the fair concernes, it might be a good idea to have the reviewee give thumbs up or down once they have reached the end of the review/are leaving the review. On thumbs up, we can send a thank you to the reviewer to make them feel at least somewhat appreciated. On multiple thumbs down, we can block the user from making more reviews.

Automated behaviour:

• just like we send a comment that chat was added to a finished game, I think we might also note that review request was added. Or if the player is stronger, we might even ask them whether they would be interested in reviewing the game themselves. A bit of a dubious suggestion though, we do not want to be annoying players too much with pop-ups and stuff.