API retrieval of games

[integer]

Hello,

This is related to the humble beginnings of my project at http://racetam.com/lzproblems/ where I generate random whole-board “go problems” taken from games on OGS (in a way inspired by the lichess.org puzzles).

Basically I want to retrieve games/lists of games according to certain parameters. I am not a web developer and so I’m very confused about how this is supposed to work. These questions are related to the /api/v1/games/ address:

1. Where can I find the “logic” of the retrieval query parameters documented properly?

1. For example: are there other non-field query parameters other than ordering that can be used? Is there some support for including logic or simple expressions?
2. … and what are all those __isnull, __gt, etc. Where are they all documented. I know they are on Django’s QuerySet page, but I assume that is documentation for the backend developers (you), not for someone like me who just wants to build a query URL. Because there are discrepancies, for example:
3. I couldn’t get name__contains=... to work, yet name__icontains=... works. Why?
   name__contains=X => “Game has no field named contains” (confusing error message)
   name__icontains=X => { … games with x in title … }
4. How do I do a “not-equals” match, for example outcome__notequals=Timeout.
5. Is there expression support, can I somehow “split” the outcome field so I can match on the points alone (let’s say I want a list of games that were won by >20 points).

Basically I want to know how much of my logic can be offloaded into query parameters without having to do my own filtering/transforms after requesting/downloading dozens and dozens of pages…

2. How do I filter by “sub-object” properties, something like players.white.ratings.overall.rating for example? I tried various things like players.white.ratings.overall.rating__gt=... and players__white__ratings__overall__rating__gt=... but couldn’t make it work. I am failing to understand how this actually works.

players__white__ratings__overall__rating__gt=100 => “Player has no field named ‘overall’” (?? did it do some foreign key look-up here?)

3. [motivation for (2)] Why are the fields X_player_rank (and X_player_rating) sometimes just 0 in the game object, yet for players that are not 30 kyu. Inside the players.X sub-objects they do seem to actually have proper ratings/ranks.

4. Is there a way to get the result in a random order? I want a “random selection” of games given certain parameters. I tried ordering=random and ordering=? but neither seemed to work.

5. Are foreign key filtering supposed to work or not? Playing around with wanting to get a list of games with specific tournament settings I noticed I could quite easily get the server to hang and produce a “there is an unknown connection issue between Cloudflare and the origin web server” page… For example by using a filter such as tournament__board_size=19

6. The one question to trump all the previous ones: is there a simple way to just get the entire list of games without having to step through tens/hundreds/thousands of “pages”? I suppose then I could just download that and do my own N different kinds of filtering/logic… But I’d hate to do that when each of those are going to turn into god knows how many requests because it’s all paginated…

7. [related to above] …and then what happens if new games matching my query is added to the database while I’m on page=7 of some query… Does everything get shifted down/up depending on where the new game appeared in the list?

Thanks.

[integer]

I have another follow-up question, since I couldn’t figure out any of the above and just started mass-scraping, going through hundreds of pages:

8. I am getting a lot of duplicate entries when asking for games, why? (Games with same ID number.) My query is basically

    params = dict(annulled='false',
                  started__isnull='false',
                  ended__isnull='false',
                  tournament__isnull='false',
                  ranked='true',
                  handicap='0',
    #              white_player_rank__gt=24,
    #              black_player_rank__gt=24,
                  width=19,
                  height=19,
    #              ordering='??????????')

With name__icontains='main title' and page=X as well, where I count upward from 1 to infinity, but I am getting a ton of duplicate responses with this, which is unexpected.

Output from my script is like:

1804310 2d 3d Tournament Game: Honinbo Main Title Tournament 2015 (4470) R:1 (Fietsbel vs estrange) ... saved to 1804310.json
1804313 3d 3d Tournament Game: Honinbo Main Title Tournament 2015 (4470) R:1 (gojohn vs GreenLeaf) ... saved to 1804313.json
1804312 3d 3d Tournament Game: Honinbo Main Title Tournament 2015 (4470) R:1 (Ukasuhs vs gojohn) ... already exists
1804315 2d 3d Tournament Game: Honinbo Main Title Tournament 2015 (4470) R:1 (gojohn vs estrange) ... already exists
1804490 4k 3k Tournament Game: Honinbo Main Title Tournament 2015 (4470) R:1 (MatCauthon vs ravasz) ... saved to 1804490.json
5435452 11k 10k Tournament Game: Tianyuan Main Title Tournament 2014 (2503) R:3 (yazphier vs Misawa) ... already exists
5435464 7k 4k Tournament Game: Tianyuan Main Title Tournament 2014 (2503) R:3 (iLq vs Desolation) ... already exists
5435451 10k 10k Tournament Game: Tianyuan Main Title Tournament 2014 (2503) R:3 (orbitaleccentric vs Misawa) ... already exists
5435463 3k 4k Tournament Game: Tianyuan Main Title Tournament 2014 (2503) R:3 (Teldris vs tacololo) ... already exists
5435466 7k 7k Tournament Game: Tianyuan Main Title Tournament 2014 (2503) R:3 (Reemi vs Desolation) ... already exists
5435461 3k 3k Tournament Game: Tianyuan Main Title Tournament 2014 (2503) R:3 (tacololo vs mipli) ... already exists
5435465 7k 5k Tournament Game: Tianyuan Main Title Tournament 2014 (2503) R:3 (Lunnainn vs Desolation) ... already exists
5435453 9k 11k Tournament Game: Tianyuan Main Title Tournament 2014 (2503) R:3 (yazphier vs orbitaleccentric) ... already exists
5435469 4k 8k Tournament Game: Tianyuan Main Title Tournament 2014 (2503) R:3 (Reemi vs iLq) ... already exists
8372594 5k 3k Tournament Game: Honinbo Main Title Tournament 2015 (4470) R:3 (Celebrir vs dudu888) ... saved to 8372594.json

Also, what is etiquette on this forum, should I instead be splitting each question into its own post?

[siimphh]

Did you ever figure any of this out?

[flovo]

[hjkl4444]

Even though this thread is probably dead - have you tried the socket API? With socket.io:

this.socket.emit("gamelist/query", { list: 'live', from: 0, limit: 5, sort_by: 'rank' }, (data) => { // stuff })

I haven’t found documentation for “list” and “sort_by” values yet.

[flovo]

there is no real documentation.

for list you have 'live' and 'corr' according to the reference implementation (website)

since we have no joice how to sort the games on OGS I think the api doesn’t support other values neither.