AI Enabled By Default Hurts Teaching

OGS Development
21

Sometimes player is out of sync and he don’t knows, and thinks “teacher” is talking about the position that is not what the player is looking for, or don’t see the sync function, or is lagged and so on. I totally agree with @BenKyo_Baduk and we can learn about what others did great, like in this case, KGS has with teaching games.

Review must have full control for one player and others can control if “admin” gives them control, and ALL must observe. I never play in KGS, and I just requested a Club room for our club only because how teaching/review system works, that is great.

EDITED: Another great feature of KGS is you can rewind in the middle of the match, and/or you can give play controls to 2 others than you players to be able to play on a specific position of the board.

6 Likes
22

^ This

At Lichess.org BTW, your game is not reviewed at all unless you press the review buttons specifically to see AI analysis and moves. I’m constantly turn it in on and off because distracts too much and I want only to check it in specific situations. To have it disabled by default, and a more accesible button/icon like @jlt suggested I think is perfect.

6 Likes
23

Additional options are not always good, because of increased complexity and resulting confusion.

1 Like
24

Oh yes, that reminds me that when I discovered, thanks to another player who had experience with the AI review, that the “territory/score” option was much more useful (in their opinion, for 9x9s, and in my opinion too after testing it), the AI reviews began to be much more helpful and useable to me.

For myself, I found the “win%” option often not so useful (or at least potentially confusing/not helpful, because it’s entirely possible to play moves which lose “win%” well as a human, depending on one’s style) in various situations, especially on small boards.

(and perhaps it would be the case for many others or beginners)

I think modifying one’s play based on what the AI considers to gain/lose “win %”, may not work well at all for one, or help in improvement, if it doesn’t suit one’s understanding or style – and that “win %” risks being a lot more confusing/misleading than the ‘‘change in score’’ option.

It can also make some possibly very good moves (which are more or less even in terms of the AI’s "points lost/gained’’ analysis, look really bad.

For example, I have seen this in reviews in which 1 or 2 moves on 9x9 lose something like -25% for some reason, but are all -0.1, -0.2 -0.3, etc. moves when analysed by “points lost/gained” in the very late yose, with very little apparent difference between them.

And in general, “points lost/gained”, or “win % +/-”, and what is considered the best move/variation by an AI analysis, can also vary depending on the settings of the particular AI, the number of variations it analyses, etc.

So something one version of an AI analysis considers the best, but other AI analyses might consider inferior/bad, could be a bit misleading to beginning/inexperienced players, especially with large +/- changes shown for a particular move.

I wonder whether having the option default to “score” would be more helpful, especially for beginners or players who are still learning and play often on 9x9 ?

I rarely use it on ''win %" mode even when checking it myself now (after some more experience and experimentation).

Actually — apart from the clutter and overload of symbols, numbers and colours on the board, I wonder whether the “win %” default mode was also a large part of the cause of my initial confusion with the system, and the confusion that others express, or at least a barrier to learning to use the AI analysis in a way which makes sense.

(I didn’t know that the “score” option existed for some time either, and it seemed to be the case with other experienced players whom I chatted with who were confused by the AI review features as well.)

(I think “points lost/gained” option potentially makes more sense to a human’s perspective than the often large % swings the ''win %" shows for what are sometimes relatively fine/good moves, especially as a mode to begin using AI analysis for the first time with, and can potentially be more useful in many ways)

3 Likes
25

I’m sorry, are you even a go player? Increased complexity and resulting confusion is what we all live for!

6 Likes
26

You should check the forum’s Go Variants category. :sweat_smile:

2 Likes
27

I’d like to see it off by default.

We definitely need good “discoverability” in order for this to be possible.

I really like the “hover shows you stuff” idea … except that this doesn’t work on a phone.

I’d get fully engaged with “advocating for and implementing” the “turned off by default” if we can nail a solution for discoverability that works on the phone as well.

8 Likes
28

Thanks,

I also discussed this with some of my students, and it seems many of them end up turning AI on at the end of their sessions and always forget to turn it off. That’s where I think it would be great if it were off by default.

I also think that self-review before AI review is very important for training. I think the main thing though again is that people are going to keep forgetting they left it on the last time. Those same people also seem to sometimes forget how to turn it back off (depending on the person).

1 Like
29

Yes, I understand the AI is one of OGS’s money makers, so I know that my suggestion could be in conflict with that a bit. I am a user of OGS AI, but for myself I also do vastly prefer it off by default.

My thinking (this is for all readers) is that if we can agree that off by default is the best, assuming hypothetically that the user is always able to find and know about the AI and its on/off setting, then I think we should assume they will know how to find it and turn it on, and that’s because…

If the user can’t find it, they also presumably can’t find all the other important stuff in the right-side menu, and that’s a huge overall OGS UI problem them if so. If we assume that users know about the right-side menu, we can assume they will turn AI on themselves if they want it on. As a Go teacher, I would hope that many users do self-review before AI review (but I know most don’t, haha…).

For all new or un-savvy OGS users, having it on by default is overwhelming/messy on the board, and confusing, especially in the context of receiving a game review from a teacher.

2 Likes
30

Agree

31

:+1: Personally I’d have no problem with it being on by default in ‘original game’ mode for finished games. It only messes me up when I’m trying to do game reviews for students.

3 Likes
32

Yeah, I do personally think the UI is the problem. I kind of like the hover menu (on desktop, which is all I use) because I know about it, but in my experience, SO many people don’t even know it exists. It would seem that the right-side hover menu is bad UX (?).

I really hope the “on by default” for AI in reviews gets a change, and as you said, if the problem is OGS losing discoverability for this feature, then that should be fixed directly instead of keeping it on by default.

I don’t care if it’s on by default in ‘original game’ mode though, only in review mode do I wish it were off by default.

3 Likes
33

Yeah, as the teacher it’s annoying to have to tell a student every time to turn it off, guide them to the right-side menu etc. Honestly a horrible routine to have to develop, better to change the UI a bit if anything. There are countless ways to make sure the user knows there is an AI feature.

1 Like
34

I think though not once has one of my students known to sync, or even easily seen the sync button. Everyone misses it, and none of us know when sync is required either. I’m sure the decision to implement this made lots of sense at the time and was a suitable solution, but in practice right now I think it’s not ideal.

I teach a lot each week and I don’t think in the last few years there has ever been a time when ‘sync’ was obvious when we were out of sync. In fact, often the lesson would continue on for over a minute while the student finally realizes they’re out of sync, and then I have to tell them where they might see a sync button. Meanwhile, I had no way of knowing they were out of sync and not seeing what I was talking about.

Just reporting a bit on that :slight_smile: I’m not asking for any change to the sync situation right now, I will be overjoyed just to have AI off by default in review mode (on by default in original game mode for finished games is fine by me).

4 Likes
35

I definitely agree with this, that’s exactly one of the several ways AI being on by default has been messing up my teaching–the student thinks the blue move and such are being indicated by me, and I totally lose them lots of teaching/explanation goes down the drain while i don’t even realize it, since I can’t see what they’re seeing. This is also true when they’re “out of sync” and I have no idea that they are.

I definitely agree that we ought to all be seeing the same thing, and that KGS is more natural in this way. Of course, although it’s not ideal to go to another page / click a link to get to a review, I imagine that’s hard for OGS to change at this point, so I’ll just focus on my smaller request haha

4 Likes
36

(Off-topic, but…) FWIW, I think the ability to go out-of-sync is critical for asynchronous reviews, where the reviewer modifies the game tree and leaves move comments and closes the window, and then the (other) player(s) see the review (possibly much later) and reads it. A kind of “correspondence” review. Not to say that the current setup is ideal, just to explain a valid use-case for allowing a review to be out-of-sync.

(Probably there’s a way to handle both live teaching and correspondence reviews well (maybe by having the reviewer choose which it is, and/or switching to correspondence if the reviewer closes the window, or …), but seems like a(n interesting) discussion for a different thread…)

9 Likes
37

Perhaps during the live review, the teacher can choose a teaching mode. While this is enabled, any other user viewing the same review will have the option to enter (or is automatically opted into) student mode which forces sync of the review including settings sync inside the review (while leaving the students personal settings on the rest of OGS untouched). Basically the students get a mirror of the teacher’s Goban. Then once the teacher disables AI, it is disabled for students as well, also for any other review settings the teacher wants to set.

After the live review ends the review is saved as a normal review and anyone can asynchronously view it like now.

I have no idea how feasible this would be technically.

5 Likes
38

I think this would be amazing. Like, “Start Teaching Review”, or “Start Solo/Cooperative Review”

5 Likes
39

I wonder, actually, if there’s ever value in AI review suggestions for readers (such as students) of an authored review.

For the author of the review, the AI review seems potentially useful, and easy enough to disable or ignore. Certainly there’s no ambiguity about where hints are coming from.

But for students/readers, it’s distracting at best. They are looking for insights from the review author, not from the robots. (EDIT: At worst, the robots undermine the author. E.g., the author says “this move simplifies the situation for black” but the robot says “that move loses a point, this other complicated move is better”. It seems like the author should control the narrative.)

IOW, is it reasonable to completely drop the feature for everyone except the review author? (Not just off-by-default, but not even available.)

EDIT: I’m suggesting this because, if so, it seems really easy to implement… no design problems to solve.

3 Likes
40

For the (off-topic) sync issue, maybe there’s a straightforward approach.

  • Author has a toggle that forces sync, on at the start. Readers cannot veer off from author’s POV when this is on.
  • Author can turn it off whenever. That mode is the existing behaviour where readers can do their own thing at their own pace (but can sync back whenever).
  • If author closes review, it automatically turns off. If the author reconnects, it’s stays off, but author can turn it back on.

Could be an action button in the same spot as the “sync” button on the reader side. Depending on current state, either says “Allow async” (to turn off) or “Sync readers” (to turn on).

4 Likes