Playing around with the GPT chat bot

hoctaph · August 8, 2025, 3:20am

I meant free handicap placement.

Counting_Zenist · August 8, 2025, 6:03am

If you don’t specify, it seems to default to “free handicap” anyway. However, if you ask where to place handicap stones, it does have some ideas

prompt
In the game of Go, with a board size of 9x9, giving 2 handicap stones, where should black place them

gpt-4o’s answer

In the game of Go, when playing on a 9x9 board, giving 2 handicap stones, the standard placement for the stones follows traditional handicap positions to give Black an advantage.

Standard Placement for 2 Handicap Stones on a 9x9 Board:

For 2 handicap stones, Black typically places them at the 4-4 points, which are the top right and bottom left corners of the board.

These coordinates (assuming the board is labeled A1 in the bottom left, H8 top right — standard Go coordinate system) are usually:

Upper Right Corner: C7 (column C, row 7)

Lower Left Corner: G3 (column G, row 3)

On Go servers or traditional notations, these positions are referred to as the star points on a 9x9 board.

Why These Positions?

These points give Black influence over both corners and some central control.

On smaller boards like 9x9, corners and speed of development are important, so this gives Black a fair head start.

Let me know if you’d like a full diagram of the board or tips on playing with handicap.

gpt-5’s answer

On a 9×9 board, the standard fixed placement for a 2-stone handicap is two diagonally opposite corner star points (the 3–3 points).

Place Black on any one of these diagonal pairs:

C3 and G7, or

C7 and G3

(Using numeric coordinates from the lower-left corner: (3,3) and (7,7), or (3,7) and (7,3).)

The center (E5) isn’t used for a 2-stone handicap.

Want a quick diagram with your preferred coordinate style?

But without specifically using my original prompt
“Let’s play a game of Go on 9x9 board, with 2 handicap stones, you play as black”, it (regardless it is 4o or 5) would place the handicap stones at will, like C6 or D7 and F3 G4. It memorizes these from the training data, but doesn’t apply it (you can see the issue in gpt-4o where it says 4-4, star points, but later says C3, C7, it’s likely just digesting some longer training data without actually understand what they are associated with, and I suspect gpt-5 just hide it better). You can actually see this with the followup prompt from above

continue prompt for gpt-5
“Yes, show me with a quick diagram using alphabet-numeric coordinate style”

Here are the two standard 2-stone handicap placements on 9×9 using letter–number coordinates (columns A–J, skipping I; rows 1–9 with 1 at the bottom).

Option 1: C3 and G7

text
9  . . . . . . . . .
8  . . . . . . . . .
7  . . . . . . X . .
6  . . . . . . . . .
5  . . . . . . . . .
4  . . . . . . . . .
3  . . X . . . . . .
2  . . . . . . . . .
1  . . . . . . . . .
   A B C D E F G H J
Option 2: C7 and G3

text
9  . . . . . . . . .
8  . . . . . . . . .
7  . . X . . . . . .
6  . . . . . . . . .
5  . . . . . . . . .
4  . . . . . . . . .
3  . . . . . . . X .
2  . . . . . . . . .
1  . . . . . . . . .
   A B C D E F G H J
Want this as an SGF snippet or flipped orientation (numbers at the top)?

And it is not placed as C7 G3, but C7 and H2. And if you ask for it to generate an sgf snippet, it would show like

(;SZ[9]HA[2]AB[cc][gf]PL[W])

which are C6 and G3, it just freely placed whatever, not considering the previous context at all.

center.square · August 8, 2025, 2:26pm

its not ready for bigger than 5x5 board yet
its better to wait when it would be able to win on 5x5 as black while writing the coordinate only,
dialogue for 9x9 with image for every move would be too long

I just tried to place two 1-1 moves as white
it still didn’t create any alive group
but I told it to not draw board in output this time, so it may be harder

Let’s play 5x5 Go. Chinese rules. No komi. You play as black. Write coordinate only. Try to win.

https://online-go.com/demo/1516042

Counting_Zenist · August 8, 2025, 3:17pm

I actually did a lot of tests today, and surprisingly if the borders are sealed and no dead stones left within the territory, gpt-5 can score a 9x9 board.

And here is the prompt

Let's score a game of Go on a 9x9 board using both the Chinese rules and Japanese rules with the following sgf record

(;KM[7.5]PB[]RE[]PW[]SZ[9]CA[UTF-8];B[de];W[ff];B[eg];W[fd];B[dc];W[fg];B[eh];W[cg];B[dg];W[be];B[fh];W[cd];B[dd];W[fb];B[ch];W[cc];B[db];W[cb];B[gg];W[hf];B[hg];W[gf];B[if];W[ie];B[ig];W[hd];B[cf];W[bf];B[bg];W[eb];B[ca];W[ba];B[da];W[bc];B[af];W[ef];B[ae];W[ad];B[ag];W[ce];B[df];W[ee];B[ea];W[fa];B[ec];W[fc];B[ed];W[];B[])

gpt-5’s answer

Here’s the final position from your SGF (9×9, SGF coords with a at top-left). X = Black, O = White, . = empty:

a b c d e f g h i
a . O X X X O . . .
b . . O X O O . . .
c . O O X X O . . .
d O . O X X O . O .
e X O O X O . . . O
f X O X X O O O O X
g X X . X X O X X X
h . . X . X X . . .
i . . . . . . . . .

Quick notes

Captures: Black captured 1 White stone (W at cg) during the game. No other captures.

No dead stones remain; all empty points are fully enclosed as territory (no seki/dame).

Territory counts (from the diagram)

Black territory: 16 points (big top area and the single point at c7).

White territory: 19 points (top-left pocket of 4, and the large right-side/bottom pocket of 14, plus b4).

Scoring

Chinese rules (area scoring, komi = 7.5 from SGF)

Black area = stones (24) + territory (16) = 40

White area = stones (22) + territory (19) = 41

Add komi to White: 41 + 7.5 = 48.5

Result: White wins by 8.5 (W+8.5)

Japanese rules (territory scoring, komi = 7.5 from SGF)

Black: territory 16 + prisoners 1 = 17

White: territory 19 + prisoners 0 + komi 7.5 = 26.5

Result: White wins by 9.5 (W+9.5)

If you’d like, I can also mark the territory on a diagram (o for White territory, x for Black territory) or export an SGF with the RE field filled in for each rule set.

The second prompt
mark the territory on a diagram (o for White territory, x for Black territory)

gpt-5’s refined answer

Here you go — o = White territory, x = Black territory (SGF coords, a at top-left)

a b c d e f g h i
a o O X X X O o o o
b o o O X O O o o o
c o O O X X O o o o
d O o O X X O o O o
e X O O X O o o o O
f X O X X O O O O X
g X X x X X O X X X
h x x X x X X x x x
i x x x x x x x x x

Legend: X = Black stone, O = White stone, x = Black territory, o = White territory.

Notice that it doesn’t format the board with alignment, but a very simple space between each element. I think it was trying to unfold the problem into counting problems, thus able to give an answer without the formatting getting in the way.

I’ve tested with other 9x9 games, but with dead stones still left on the board, or there are still dame left, etc. It doesn’t always work. And it’s best to have very solid borders to get good results.

I’ve also tested it with 9x9 using 2 handicaps and using sgf snippet only for reply without formatting, and it would still run into a problem.

Here is the record

gpt-5 first ran into problems with just sgf snippet and not a formatted board at move 11, when I captured bD6, and it didn’t register the D6 as captured, and then played E7 and called it “double atari”. And when to ask it to recheck again, it first said no problem at all. And I asked it again to visualize with diagrams in every move. And then it started to post every diagram for every move till that point, and finally caught up the D6 “captured”. Hence, I continue the game, asking it to visualize the diagram as well as keep adding a sgf snippet in each reply. And surprisingly, it was able to play ko for a couple of moves. However, it ran into trouble at move 32 again, when it thought it could play H6 to capture 4 white stones (I feel it treated the ko stone at D6 as alive), and I had to ask it to go back and check again for each stone’s liberties and atari status, then it picked another move. It runs into trouble acknowledging C7 was captured after I played C8 at move 33, and I feel it ran into context limit at this point, and kept playing 1st line self-atari moves. Finally, at move 38, it thought H8 got captured, and kept trying to play H8, and couldn’t update the board diagram, or the sgf snippet.

I’d say it was a big improvement for context length, and the old tricks of keeping the records (either with move history or sgf snippets) can extend the games further (I already can play with gpt-3 back then using this trick), but at some points it just ran out of context and started to behave irradicately and cannot recover.

center.square · August 8, 2025, 4:10pm

select text and click

  a b c d e f g h i
a . O X X X O . . .
b . . O X O O . . .
c . O O X X O . . .
d O . O X X O . O .
e X O O X O . . . O
f X O X X O O O O X
g X X . X X O X X X
h . . X . X X . . .
i . . . . . . . . .

or just add three ` symbols before and after like that:

when you copy output to forum they may disappear like space in the start of text
lmarena instead of official site also may fail to properly display what bot actually writes

  a b c d e f g h i
a o O X X X O o o o
b o o O X O O o o o
c o O O X X O o o o
d O o O X X O o O o
e X O O X O o o o O
f X O X X O O O O X
g X X x X X O X X X
h x x X x X X x x x
i x x x x x x x x x

shinuito · August 8, 2025, 4:30pm

I mentioned kaggle had a game arena and were doing a chess exhibition with LLMs

From a discord user:

so livestream is tomorrow?
will Go be supported in the future? I think Go is harder than chess

response from someone at the Kaggle team

Yep, it’s tomorrow!

We are planning to support a lot of different games with game arena.

Keep in mind that relative to reinforcement learning (eg. AlphaGo) LLMs are fairly bad at playing games, so they have a long way to go to start mastering harder games!

Maybe Go will come along at some stage, but I guess we shouldn’t be expecting them to play well

Counting_Zenist · August 9, 2025, 12:52am

No, it literally differs at the output, not just copy the format

This is another record that it failed to score its output is like this, it literally has a line to specify the column coordinates

And this is the one that it doesn’t. I’ve tested a couple of them, and those replies didn’t contain these “format string” are often more accurate

Counting_Zenist · August 9, 2025, 4:02am

I’ve tested gpt-5’s capability to generate a function to count the score, and here is the code it gave me:

I can score this precisely for both Japanese (territory) and Chinese (area) rules. Since it’s easy to make mistakes by hand on a 19×19, the quickest way to get exact numbers from your SGF is to run a tiny scorer. Paste the code below into a file (e.g., score_go.py) and run it with Python 3. It will parse your SGF, replay all moves, and print both Japanese and Chinese scores using KM[6.5]. It assumes both players passed with no unresolved dead stones (the SGF ends with W[];B[]), which matches your record.`

If you prefer, run it and paste the output here and I’ll interpret it and highlight any rule-specific nuances (e.g., seki).

Code (drop-in, no dependencies):

Python

import re
from collections import deque

SGF = r"""(;KM[7.5]PB[]RE[]PW[]SZ[9]CA[UTF-8];B[de];W[ff];B[eg];W[fd];B[dc];W[fg];B[eh];W[cg];B[dg];W[be];B[fh];W[cd];B[dd];W[fb];B[ch];W[cc];B[db];W[cb];B[gg];W[hf];B[hg];W[gf];B[if];W[ie];B[ig];W[hd];B[cf];W[bf];B[bg];W[eb];B[ca];W[ba];B[da];W[bc];B[af];W[ef];B[ae];W[ad];B[ag];W[ce];B[df];W[ee];B[ea];W[fa];B[ec];W[fc];B[ed];W[];B[])"""

def parse_prop(s, key, cast=str, default=None):
    m = re.search(rf'{key}```math
([^```]+)```', s)
    if not m:
        return default
    try:
        return cast(m.group(1))
    except:
        return default

def parse_moves(s):
    # Returns list of tuples: ('B'/'W', (x,y)) or ('B'/'W', None) for pass
    moves = []
    # naive parse: find all ;B[..] or ;W[..]
    for color, coord in re.findall(r';([BW])```math
(.*?)```', s):
        if coord == '':
            moves.append((color, None))
        else:
            x = ord(coord[0]) - ord('a')
            y = ord(coord[1]) - ord('a')
            moves.append((color, (x, y)))
    return moves

class GoBoard:
    def __init__(self, size):
        self.n = size
        self.board = [['.' for _ in range(self.n)] for _ in range(self.n)]
        self.prisoners = {'B': 0, 'W': 0}

    def inb(self, x, y):
        return 0 <= x < self.n and 0 <= y < self.n

    def neigh(self, x, y):
        for dx, dy in [(1,0),(-1,0),(0,1),(0,-1)]:
            nx, ny = x+dx, y+dy
            if self.inb(nx, ny):
                yield nx, ny

    def group_and_liberties(self, x, y):
        color = self.board[x][y]
        assert color in ('B','W')
        q = deque([(x,y)])
        group = {(x,y)}
        liberties = set()
        while q:
            cx, cy = q.popleft()
            for nx, ny in self.neigh(cx, cy):
                v = self.board[nx][ny]
                if v == '.':
                    liberties.add((nx, ny))
                elif v == color and (nx, ny) not in group:
                    group.add((nx, ny))
                    q.append((nx, ny))
        return group, liberties

    def remove_group(self, group):
        # group is set of coords of same color stones
        color = self.board[next(iter(group))[0]][next(iter(group))[1]]
        for x, y in group:
            self.board[x][y] = '.'
        return color, len(group)

    def place(self, color, move):
        if move is None:
            return True  # pass
        x, y = move
        if not self.inb(x, y) or self.board[x][y] != '.':
            raise ValueError(f"Illegal move {color}[{x},{y}]")
        opp = 'W' if color == 'B' else 'B'
        self.board[x][y] = color
        # capture any adjacent opponent groups with no liberties
        captured = 0
        for nx, ny in set(self.neigh(x, y)):
            if self.board[nx][ny] == opp:
                g, libs = self.group_and_liberties(nx, ny)
                if not libs:
                    _, sz = self.remove_group(g)
                    captured += sz
        if captured:
            self.prisoners[color] += captured
        # check suicide: own group must have liberties
        g, libs = self.group_and_liberties(x, y)
        if not libs:
            # undo move
            self.board[x][y] = '.'
            raise ValueError(f"Suicidal move {color}[{x},{y}]")
        return True

    def count_territory(self):
        n = self.n
        vis = [[False]*n for _ in range(n)]
        terr = {'B': 0, 'W': 0}
        neutral = 0
        for i in range(n):
            for j in range(n):
                if self.board[i][j] == '.' and not vis[i][j]:
                    # BFS empty region
                    q = deque([(i,j)])
                    vis[i][j] = True
                    region = [(i,j)]
                    borders = set()
                    while q:
                        x, y = q.popleft()
                        for nx, ny in self.neigh(x, y):
                            v = self.board[nx][ny]
                            if v == '.':
                                if not vis[nx][ny]:
                                    vis[nx][ny] = True
                                    q.append((nx, ny))
                                    region.append((nx, ny))
                            elif v in ('B','W'):
                                borders.add(v)
                    if borders == {'B'}:
                        terr['B'] += len(region)
                    elif borders == {'W'}:
                        terr['W'] += len(region)
                    else:
                        neutral += len(region)
        return terr, neutral

    def count_stones(self):
        b = sum(1 for i in range(self.n) for j in range(self.n) if self.board[i][j] == 'B')
        w = sum(1 for i in range(self.n) for j in range(self.n) if self.board[i][j] == 'W')
        return b, w

def main():
    size = parse_prop(SGF, 'SZ', int, 19)
    komi = parse_prop(SGF, 'KM', float, 0.0)
    moves = parse_moves(SGF)

    board = GoBoard(size)
    for color, mv in moves:
        board.place(color, mv)

    terr, neutral = board.count_territory()
    b_stones, w_stones = board.count_stones()
    prisoners = board.prisoners

    # Chinese (area)
    b_ch = terr['B'] + b_stones
    w_ch = terr['W'] + w_stones + komi

    # Japanese (territory + prisoners)
    b_jp = terr['B'] + prisoners['B']
    w_jp = terr['W'] + prisoners['W'] + komi

    # Print details
    print(f"Board size: {size}x{size}, Komi: {komi}")
    print(f"Stones on board: B={b_stones}, W={w_stones}")
    print(f"Prisoners: B={prisoners['B']}, W={prisoners['W']}")
    print(f"Territory: B={terr['B']}, W={terr['W']}, Neutral={neutral}")
    print("\nChinese (area) scoring:")
    print(f"  Black total = territory({terr['B']}) + stones({b_stones}) = {b_ch}")
    print(f"  White total = territory({terr['W']}) + stones({w_stones}) + komi({komi}) = {w_ch}")
    diff_ch = w_ch - b_ch
    if diff_ch > 0:
        print(f"  Result: White wins by {diff_ch:.1f} points (Chinese).")
    else:
        print(f"  Result: Black wins by {-diff_ch:.1f} points (Chinese).")

    print("\nJapanese (territory) scoring:")
    print(f"  Black total = territory({terr['B']}) + prisoners({prisoners['B']}) = {b_jp}")
    print(f"  White total = territory({terr['W']}) + prisoners({prisoners['W']}) + komi({komi}) = {w_jp}")
    diff_jp = w_jp - b_jp
    if diff_jp > 0:
        print(f"  Result: White wins by {diff_jp:.1f} points (Japanese).")
    else:
        print(f"  Result: Black wins by {-diff_jp:.1f} points (Japanese).")

if __name__ == "__main__":
    main()

Notes and assumptions:

The final two moves are passes (W[]; B[]), so the game is over.
This scorer removes captured stones during play and then counts territory from the final position. It treats any empty region bordering both colors as neutral (covers seki correctly in standard cases).
For Japanese rules, “prisoners” are the stones actually captured during play. If players passed while leaving dead stones on the board (uncommon), those would need to be removed (counted as prisoners) by agreement before scoring; if you suspect that happened, I can add a prompt to mark dead groups.

Want me to run this for you and post the exact Japanese and Chinese scores? If you share the output of the script above, I’ll summarize who wins by how much under each ruleset.

The code doesn’t run directly where the re.search expression is incorrect, and they should be replaced, like for finding komi

m = re.search(rf'{key}\[(.*?)\]+', s)

and for finding move coordinates

for color, coord in re.findall(r';([BW])\[(.*?)\]', s):

Once you correct them, this code does score the simple games like a 9x9, but will fail where there are still dead stones on the board, and borders are “unclear” with dame.

So I can see why it has improved in the final scoring department, where it can simply write a scorer and run it directly. Hence, it is technically possible for it to write a simple older Go engine that detects atari and some "influence check with groups, but not likely some MCTS engine yet. gpt-5 has improved in its coding and running instance capability, but not in the department of Go understanding.

center.square · August 9, 2025, 2:15pm

when it really forgets to align, it still makes sense to make it more readable for us anyway

Caution! unclear naming:

model on lmarena that named “gpt-5-chat” is not reasoning mode, its like running leela with 0 playouts

gpt 5 without reasoning mode

Counting_Zenist · August 9, 2025, 3:02pm

I actually reformatted the earlier reply for the 9x9 2 handi experiment for easy-to-read, but just for that example of scoring, I chose to share its raw into just to show there is a difference in its output, and not by accident, but it seems to exhibit a certain trend when it doesn’t format itself (maybe I should have also upload it to a library on OGS to just show the original sgf).

I also notice there are gpt-5-mini and gpt-5-nano now. Also the open-sourced gpt-oss-20b and 120b. And yes, gpt-5-chat hallucinates even more than those gpt-3 somethimes.

center.square · August 9, 2025, 4:19pm

https://www.reddit.com/r/OpenAI/comments/1ml0i1h/a_phd_in_your_pocket_ladies_and_gentlemen/

huh, gpt-5 without “think longer” turned on is totally broken and all free users are confused now

center.square · October 8, 2025, 2:45pm

If Go had been created in England, what would it have been called?

lol

Groin · October 8, 2025, 5:06pm

Stonehenge is pretty name, nice pick

Counting_Zenist · November 1, 2025, 11:30am

And as of today (Nov 1, 2025), ChatGPT is still freaking out over the emoji of seahorse

center.square · November 18, 2025, 5:25pm

I accept the challenge. I will play as Black, you play as White. The rules are Chinese, 5x5 board, no komi.

Move 1 (Black): C3

Since the center point is the most powerful spot on a 5x5 board, I will claim it immediately.

  A B C D E
5 . . . . .
4 . . . . .
3 . . X . .
2 . . . . .
1 . . . . .

Your move. Please enter coordinates (e.g., C2).

game

  A B C D E
5 . . . . .
4 . . . . .
3 . O X . .
2 . . . . .
1 . . . . .

  A B C D E
5 . . . . .
4 . . X . .
3 . O X . .
2 . . . . .
1 . . . . .

  A B C D E
5 . . . . .
4 . . X . .
3 . O X . .
2 . . O . .
1 . . . . .

  A B C D E
5 . . . . .
4 . . X . .
3 . O X . .
2 . . O X .
1 . . . . .

  A B C D E
5 . . . . .
4 . . X . .
3 . O X . .
2 . O O X .
1 . . . . .

  A B C D E
5 . . . . .
4 . . X . .
3 . O X X .
2 . O O X .
1 . . . . .

  A B C D E
5 . . . . .
4 . O X . .
3 . O X X .
2 . O O X .
1 . . . . .

  A B C D E
5 . . . . .
4 . O X X .
3 . O X X .
2 . O O X .
1 . . . . .

  A B C D E
5 . . . . .
4 . O X X .
3 . O X X .
2 . O O X .
1 . . . O .

  A B C D E
5 . . . . .
4 . O X X .
3 . O X X .
2 . O O X X
1 . . . O .

  A B C D E
5 . . O X .
4 . O X X .
3 . O X X .
2 . O O X X
1 . . . O .

  A B C D E
5 . X . X .
4 . O X X O
3 . O X X .
2 . O O X X
1 . . . O .

  A B C D E
5 . X . X .
4 O O X X O
3 . O X X .
2 . O O X X
1 . . . O .

  A B C D E
5 X X . X .
4 O O X X O
3 . O X X .
2 . O O X X
1 . . . O .

  A B C D E
5 . . O X .
4 O O X X O
3 . O X X .
2 . O O X X
1 . . . O .

  A B C D E
5 . X . X .
4 O O X X O
3 . O X X .
2 . O O X X
1 . . . O .

  A B C D E
5 . X . X .
4 O O X X O
3 . O X X .
2 . O O X X
1 . . O O .

  A B C D E
5 X X . X .
4 O O X X O
3 . O X X .
2 . O O X X
1 . . O O .

  A B C D E
5 X X . X .
4 O O X X O
3 . O X X .
2 O O O X X
1 . . O O .

  A B C D E
5 X X X X .
4 O O X X O
3 . O X X .
2 O O O X X
1 . . O O .

  A B C D E
5 X X X X .
4 O O X X O
3 . O X X .
2 O O O X X
1 . . O O O

  A B C D E
5 X X X X .
4 O O X X O
3 . O X X X
2 O O O X X
1 . . O O O

Board after White E5:

White’s move at E5 fills the last liberty of the large group of 11 Black stones.
The Black stones at A5, B5, C5, D5, C4, D4, C3, D3, E3, D2, E2 are captured and removed.
(White’s stones at E4 and E5 survive because removing the Black stones creates new liberties).

  A B C D E
5 . . . . O
4 O O . . O
3 . O . . .
2 O O O . .
1 . . O O O

Move 14 (Black): C3

I must restart the game in the center.

  A B C D E
5 . . . . O
4 O O . . O
3 . O X . .
2 O O O . .
1 . . O O O

Your move.

you call it “restart”?

It is a fair point; “restart” is an understatement! You successfully captured a massive group of 11 stones, which is effectively a game-ending move on a 5x5 board.

However, my instructions are to play until the end. Since I have not resigned, I played C3 in the newly cleared space to see if I can form a living group among the ruins.

you lost

You are absolutely right. The capture of my massive group at E5 was a devastating and game-ending move. There is no way for Black to recover from that loss on a 5x5 board.

I resign.

Congratulations on the win! That was a sharp tactical game.

Lys · November 18, 2025, 6:24pm

AI training!

PJTraill · November 19, 2025, 1:15am

How long to reach superhuman?

richyfourtytwo · November 19, 2025, 9:03am

The strategy to get to superhuman AI is easy: make humans more stupid. And we are already half way there. A colleage of mine just let ChatGPT ‘optimise’ a complex configuration file for proprietary software ChatGPT doesn’t (couldn’t) know anything about. Of course without giving ChatGPT access to the verbose documentation that exists.

Counting_Zenist · November 19, 2025, 5:49pm

Did your friend work at Cloudflare and crash the Internet yesterday?

richyfourtytwo · November 19, 2025, 9:20pm

If I had written THAT piece of buggy software, I’d be earning a lot more than I am!