Whitelisting for /api/v1/games

#1

tl;dr: I’m wondering if it is possible gain access to /api/v1/games (for my API key?) if I promise to behave nicely.

More context:

I remember using the /api/v1/games endpoint in the past to get small datasets of certain types of games (I’m tinkering with a go nerual networks to play kyu games, handicap games and do score estimation, etc). But now the API endpoint seems to reject any queries I issue to it:

https://online-go.com/api/v1/games/

Not Found
The requested URL /api/v1/games/ was not found on this server.

I imagine this is an expensive endpoint, so I wonder if it is rate limited (to 0 by default)? Or not available at all any more?

What I’m looking to do is get a few hundred games for sets of different ranks of the players (and possibly handicaps) though should the experiments be successful I might want quite a few more games to properly train the network.

I could approximate my queries by querying /api/v1/players to find matching players and then /api/v1/players/<uid>/games to find games that will probably match the criteria I had in mind. But in case /api/v1/games was rate limited, I don’t just want to go around the intention of the rate limit. But if that sounds OK to you, I could just do that, too.

(bat signal in case you have a moment: @anoek, @matburt)

In case someone finds this message and has a similar problem, so far I’ve gotten games from:


http://www.yss-aya.com/cgos/19x19/archive.html

1 Like

#2

Hello! I used /api/v1/games in the past, but I remember it being very clunky, responses always slow and requests often time out. At one point in time, OGS server almost broke down because of someone’s API requests. Shortly after that /api/v1/games was not available anymore, so I suspect there’s connection here.

If you just want game moves that endpoint is mostly useless anyway, because for moves you need to access each game separately. The best way I managed to find is accessing rating histories of each player https://online-go.com/termination-api/player/109488/rating-history?size=9&speed=live and download games that match my criteria.

Of course, I don’t need that many games. And you need a lot. Within rate limits it’ll probably take months to download. Better to ask for database dump or something indeed, but.13.6 million games from foxy? Why would you even need OGS? OGS game ids reach now 15.5 million, but that’s ids, a lot of them wasted, in all likelihood OGS doesn’t even have that many games.

0 Likes

#3

You are right in that it probably does not matter all that much if the games are from OGS at this point.

The reason I was looking for OGS data was that I was entertaining a notion of tailoring some of those networks for OGS.

0 Likes

#4

I had downloaded games from the API by first searching for users potentially matching my criteria (ranks) and then searching the games per-user. I tried to be courteous (ie: backing off when getting the 429 error code) but eventually it looks like my client was still blocked. So I presume bulk downloading of games (perhaps more than a few hundred) is not appreciated by the devs.

0 Likes

#5

I managed to download ~18 thousands of games, but with 15 or 30s interval between requests.

0 Likes