Error when querying /players - {'error': 'Page size is too large'}

OGS Development
1

Hi all,

I am trying to query the OGS API to get all player ratings and look at statistics such as avg level, median, std, etc.

I’m using Python to iterate through pages using requests.get('http://online-go.com/api/v1/players/?page=1')

Each page of results contains 10 players and their associated ranking.

My method works fine for the first 100 pages, until I reach page 101. Then it returns the following error:
{‘error’: ‘Page size is too large’}

Do you have any idea of what’s happening? I don’t know where to look in the documentation.

1 Like
2

Yeah I think there might just be a weird cap on the number of pages you can go back.

I ran into it before also, but wasn’t sure why it is the way it is.

You can change the page size by using page_size=x e.g.

Player List – Django REST framework

but you run into the same problem at page 101 for whatever reason.

3

Hi @shinuito,

Thanks for your reply. I just tried it and it seems the maximum number of players I can get is 100 players x 100 pages = 10,000 players out of the 1,270,000+ registered.
When I try getting more than 100 players per page, I still get 100 results.
That’s too bad, I won’t be able to compute meaningful stats about them :frowning:

If anyone has a solution I’m happy to hear it. I would also be happy to document this in the API documentation but not sure how to get access.

4

You get the rating history of a player using the query https://online-go.com/termination-api/player/${palyerId}/v5-rating-history?speed=overall&size=0
You have to iterate ids one by one.

5

Hi @flovo,

Thank you for your help!

Firs of all, I apologize if what I say isn’t clear, I’m a beginner in Python & calling APIs.

Do you mean the only way to get current ‘ratings’ for all registered players is to call the API 1.2M times (for each individual player)?

My goal is to put high level players information into a table I can extract stats from. I’m attaching an example of what I have done with 10,000 players I could get with the /players API method.

ogs df
ogs stats

6

If you are interested only in somewhat active players, you can start with any player. This API returns the rating history of the player. You can then download their opponents for the timeframe you are interested in, and so on.

I’m not aware of any API giving you the data on mass.

BTW: Please don’t overwhelm the server with requests.

1 Like
7

Hi @flovo,

Thanks for your reply, I’ll be careful :slight_smile:

8

Very few players are high level so it won’t take that much time if you jump only between strong player. Though I can boast downloading ~500k players which is basically everyone who has meaningful games, I think.

image
image1503×848 82.2 KB

But! But! If you don’t want such a tedious task, I remembered a good workaround for /players url.

You can give additional parameters to that. For example, you can split players by registration date which is what I use, e.g. to get players between 2021-12-15 and 2021-12-31 you can write (some parts can be omitted), if I understand it correctly

Player List – Django REST framework

And come to think of it, you can just restrict players by id, can’t you? To get players with ids between 15000 and 25000 just write

Player List – Django REST framework

Should’ve responded to the topic earlier. Note that api/v1 requests seem slow and heavy on the server and termination-api seems lighter. api/v1 at least at some point had request limit and returned 429 appropriately. Note that sorting by id looks broken.

Also note that we have sgf dump from late 2021 which might be new enough for your purposes.

2 Likes
9

@Allerleirauh awesome, thank you!

How did you DL 500k+ players?

By “high level” I meant general information such as:

  • Number of players on OGS
  • Number of games played per player
  • Their level / rank
  • Their registration date
  • Their countries

In general, how do you find what parameters you are allowed to pass in your API call URL? I don’t know where to look to find ways.

10

Oral tradition in this forum. :slight_smile:
If you can’t find a thread about it, just ask. There’s a small group of API fans that loves to share their knowledge.

Not very reliable information: many users don’t choose a country at all, some like to change it from time to time, some like to pick starfleet or Targaryen or gender pride.

11

This topic was automatically closed 91 days after the last reply. New replies are no longer allowed.