OGS API notes

Hi there,

I’ve been toying a bit with making a go bot. My bot’s not doing anything that could be of interest to others at the moment, but along the way I’ve figured out a few things about the OGS APIs that would have been useful to know from the start, so I figure I could write it all out here for others (people writing bots or alternative clients) to benefit.

There are two separate APIs that OGS clients need to interact with:

  • The REST API is used for infrequent client requests such as logging in, adding friends, starting games. REST is all based on standard HTTP requests.

  • The RT API is used for more frequent client requests such as sending chat messages or game moves. It is based on the socket.io library, which is a widely used but not very well documented protocol. However, socket.io itself is built on top of websockets, which is well standardized.

There are various documents describing both APIs (more on this below), though I found them hard to use at first - one first needs to get a general idea of how the APIs work, and then the documents may help fill in some details, but some major pieces seem to be missing still. What I found most helpful is to spy on the requests between the OGS web client and server. Using chrome, open ogs in a tab and in that tab, open the developer tools (under “more tools” in the chrome menu). Go to the Network tab and enable “preserve log” so that logs don’t get cleared whenever you visit a new ogs page. Then as you use the OGS web client, REST requests get logged under “XHR” in the developer tools console, and RT requests get logged (at the websocket level) under “WS”. The RT API calls all share a single websocket connection, so to dig into it one can click on the connection url under “Name” and look at the “Frames” tab to see the individual websocket frames holding each RT API requests.

With this said, I think one should jump right in and describe the minimal API calls for playing moves in an existing OGS game.

  • Client signs in by sending a POST request to https://online-go.com/api/v0/login . The request body is some application/json string: {“username”:“your_username_goes_here”,“password”:“your_password_goes_here”}. The client expects a 200 response. The response sets some cookies which we will want to send along with all future REST requests to authenticate them.

  • Client sends GET request to https://online-go.com/api/v1/ui/config . This goes with an empty body and the HTTP cookies copied from the response to the previous login request. Server response includes a application/json encoded body with a config structure. Some interesting fields in this structure are “chat_auth” and “notification_auth” strings, which will be used for authentication later, and “user” which is another json struct, itself with “username” string and “id” integer fields which the client will also need later.

  • At this point, the client can open a new connection to send RT API calls on. I wrote my client at the websocket level instead of using socket.io. The websocket connection is made to wss://online-go.com/socket.io/?transport=websocket . RT API calls will translate to json encoded text frames sent on that websocket, as we will soon see.

  • The client sends the first RT API call on the websocket. This is a “notification/connect” call with json parameters {“player_id”:“your_id_here”,“username”:“your_username_here”,“auth”:“your_notify_auth”}. The way socket.io encodes this in the websocket is that it sends a text frame with contents 42[“notification/connect”,{“player_id”:“your_id_here”,“username”:“your_username_here”,“auth”:“your_notify_auth”}] (all on one line, no whitespace). The your_* values come from the REST config answer above. The effect of the notification/connect message is that the player now shows up as online to other ogs users. I’m not sure what’s the meaning of 42 here, socket.io just seems to use it in nearly all of its websocket encodings.

  • The client next sends a chat/connect RT API call, so that it will be able to send and receive chat messages. The call is sent as 42[“chat/connect”,{“player_id”:“your_id_here”,“username”:“your_username_here”,“auth”:“your_chat_auth”}]

  • The client also sends an authenticate RT API call. This is necessary for the client to send game moves later on. The call is sent as 42[“authenticate”,{“player_id”:“your_id_here”,“username”:“your_username_here”,“auth”:“your_chat_auth”}]

  • From then on, the client should send RT ping messages every so often (20 seconds works) to keep the RT connection alive. That call is sent out as 42[“net/ping”,{“client”:1528023039000}] where the numerical value is time in milliseconds since the unix epoch.

  • The client should also listen on the websocket for RT messages sent out by OGS. Just like when sending RT API calls, the received messages have the format 42[“message-type”,{message-dependent json details}]

  • To keep things simple, let’s assume the player already has existing games in progress. For each such game, the server will send an active_game message. the active_game json data has some information about the game size and the players strength and whose turn it is, but most importantly the game ID. The received message looks like 42[“active-game”,{“id”:123456, … lots of other fields here }]

  • Given the above game ID, one may join the game by sending 42[“chat/join”,{“channel”:“game-123456”}]" (show up as present in the game chat) and 42[“game/connect”,{“game_id”:123456}] (ask to receive more details about the game).

  • The server sends more information about the game we connected to: 42[“game/123456/gamedata”,{…}] The json gamedata structure has too many parameters to describe here, but it’s fairly straightforward. In particular, the “moves” parameter shows all moves that have been played so for, so that one can reconstruct the game state.

  • The player can submit moves using the game/move RT API: 42[“game/move”,{“game_id”:123456,“player_id”:your_player_id_here,“move”:[2,3]}. This actually moves in [3,4], because of zero-based indexing, or what’s shown as C16 on the UI because things are counted in a different direction there :slight_smile: As noted above, one needs to send the authenticate command at some point before game/move, or the game/move command will be silengly ignored. OGS would also accept the same move as “move”:“cd”. Some protocol details can be weird; the web client receives moves from OGS in numerical format but sends moves with the game/move command in text format, … ???

  • When the opponent makes a move, our player receives notice as an RT message: 42[“game/123456/move”,{“move_number”:123,“move”:[4,2,654321]}. This is a move at [5,3] or E17 in the UI. I’m not sure what the third integer is for; clients can ignore it with no observed ill effects… ???

As we can see from the above, we’ve only used REST APIs for login; after that everything has been using the RT APIs. Let’s try to switch gears and look at something that requires a REST call - for example, accepting a game challenge.

  • The challenge is received from OGS as an RT message. The message looks like the following: 42[“notification”,{“type”:“challenge”,“challenge_id”:1234,…lots of other params here…}].

  • To accept the challenge, one would send an http POST to https://online-go.com/api/v1/me/challenges/1234/accept. This particular REST API does not send any parameters in the request body (note the challenge ID in the URL though). To make this API work (or any other REST APIs that use http POST), one must send it with the cookies that were received from the initial https://online-go.com/api/v0/login call. In addition, if one of the received cookies was named “csrftoken”, then the request should be sent with a Referer: https://online-go.com/ field, as well as X-CSRFToken: field holding the csrftoken cookie value.

  • After the game is accepted, the server sends the “active-game” message on the RT connection, and things proceed just like the above case of continuing play within an existing game.

I didn’t hit any other major hurdles after I figured out the above. And at that point, I had enough background knowledge of the API that I could mostly use the docs to fill in the gaps. There are several docs to browse through:

  • The main doc for the RT API is at https://ogs.readme.io/docs/real-time-api. I found it the most useful of the docs. it’s still missing some important details though - it does not replace trying an API call and seeing what responses you get, and/or spying on web client ogs exchanges to get the answer.

  • The main doc for the REST API is at https://ogs.docs.apiary.io . One can also browse REST API endpoints at https://online-go.com/api/

  • https://github.com/online-go/gtp2ogs is an interface provided by OGS folks for connecting bots to OGS. I didn’t directly use it because 1- I’d have to make a GTP interface for my bot first, and 2- I wasn’t too thrilled about using javascript. But, it can be a good reference to dig into - the protocol used is mostly the same as with the web client (i.e. a combination of REST and RT API calls), with some differences around logging in and authentication.

Authentication mechanisms

Some of the difficulties in using the API have to do with authentication. I found it easiest to copy whatever the web client does, but there are at least two other ways I have heard about:

  • gtp2ogs uses a different mechanism where the bot account gets assigned an API key (the bot owner will see a button on the bot’s account page to generate that key). After opening the RT connection, gtp2ogs sends a bot/id command with the API key as a parameter, and receives back some other authentication tokens, which are then used in place of those the web client would have received from the https://online-go.com/api/v1/ui/config REST call.

  • other OGS clients may obtain some oauth2 keys, which I’m not sure how they would use - I believe this also replaces the https://online-go.com/api/v0/login and https://online-go.com/api/v1/ui/config calls ???

I’m a bit confused about this, and would like clarification from the OGS folks about whether they are OK with non-web clients just using the same APIs as the web client does ?

Avoiding abusive behaviors

I enjoy the friendly atmosphere on OGS and I hope we can all keep it that way. If you are planning to use OGS APIs to submit computer generated moves, please make sure to use a separate user account and ask the mods to get it marked as a bot account. Also, bots are still expected to follow the normal user rules - please don’t make a robo-jerk :slight_smile:

It’s also a good idea to avoid making excessive API calls to OGS. REST calls should be fairly infrequent. The RT API lets you get notifications about most anything, so you shouldn’t have to ever use polling. If your RT connection gets unexpectedly closed, you may want to immediately re-establish it so as to avoid timing out in a game, but please make sure this does not happen at a high rate - for example, you may limit such automated re-connections to once per minute or so. This is all common sense, but we don’t want to take shortcuts and create trouble for the OGS folks.

Hope this helps. I didn’t find many people knowledgeable about this who would have time to help, and it took me longer than expected to figure out the above, so I’m hoping someone else won’t have to redo all the trial and error that I did.

22 Likes

Thanks a lot for this, extremely helpful!

1 Like

Thank you very much for this post. It is incredibly helpful.

I’d like to add that when the opponent makes a move, the third integer in the received RT message is the time in milliseconds it took for the move to be played. So if the clock started at 15 minutes and opponent took 12 seconds to think, then the integer would be 12000. I guess it’s a useful mechanism to correct the game clock on the client side?

2 Likes

You are welcome ! Very happy to hear this helps.

Also, thanks for the hint that the 3rd number is the time it took to play the move - I had an inkling that may be it, but did not spend the time to confirm it.

1 Like

Very helpful writeup on API. I think the oath2 method login may have to do with user tagged as bot.

Here is python code for using OGS API that I found:

This is great documentation. It should be added to the appropriate GitHub page for anyone else

Wait, is the oath2 only for bots?

No. It’s for any third party client.

1 Like

Ok, thanks for letting me know. Are there any good resources for learning how to use this api? This is the first time I have had to use an api with oauth2 and it’s tripping me up here and there.

No. There are no good resources. For the oauth tokens you can search the forum for “oauth”

For how to use the REST-API, there is a very outdated, incomplete doc, but it gives a good first impression on how to use the API

If you have any questions, best ask them in a new forum thread.

Sweet! Thanks for letting me know.

For the Real-Time API you can find some examples here:

Authentication changed since then, but it gives some hints how to use it.

How do you send the cookies set from the POST login request? Is it some sort of id that needs to be taken from the login response?

If you got the token you can add it to the header of your requests as Authorization=Bearer ${auth_token}

Every GET requests on /api/v1 works when you add the token. You don’t need cookies for this.

1 Like