Is there a way we could do a pattern search through all the game played on OGS and potentially sort it by rank.
It would be very useful for my purposes of trying to make joseki for myself and my students depending on what we face.
If we could also filter it by year it would be cool as well.
You might have to stick with http://ps.waltheri.net/
We have almost 40,000,000 game IDs (though not all of them are games), and aren’t really set up for what you ask, which would be a huge project and I suspect not a trivial server drain to support such a large database performing mirror and rotation searches on millions of games…
What about a filtered list of games we could download and then perhaps use a 3rd party pattern search software on our own pc to not overload the server.
For example, I could filter 5 Kyu 2021 and Download 1,000 games from that.
This is what you are looking for maybe?
Somebody already did a mass
…and Adam beat me to it 
kudos
The question is who is going to build it?
I would also like to explore what’s the common josekis and openings on OGS. I always have the feeling that there are different cultures in different servers, and players just favor some over the others.
It could be something done entirely off of OGS. Waltheri has already built up the technology to do so (is the code behind that site open source?) and the SGF dump is publicly available. It’s only matter of putting the two together. However, maybe scaling up the processing up the size of the OGS game collection is far from trivial.
yeah waltheri have just under 80,000 games so that’s WELL under an order of magnitude less than a potential OGS equivalent, even if we only include the 19x19 games longer than 20 moves.
What if you limited the search to the first 100,000 results to reduce overhead?
I just want to pattern search my own games (few hundred), which is why I bumped this 7 years old feature request to bulk download one’s own games.
this might be something achievable…
I’m not sure why (by implication) the original request has to be deemed “unachievable” just because it is more ambitious than waltheris.
The fact that it would likely call for more server resources or more “whatever” doesn’t rule it out: AI analysis of our games was a big project called for more servers, and this was added because it was a good feature.
The real question then is “Why is this a compelling feature to have? And how compelling is it?”
When the latter question was asked about rengo, we found the answer was “suprisingly, this feature is compelling enough for someone to make it happen”. Similarly, an OGS Joseki Explorer was a big project calling for (a small increment) more compute, and it happened.
So we should have our minds open to this possibility…
If you can store 40M games (with metadata!), you can store a game tree with <40M leaf nodes.
Sure it’s resources, but it shouldn’t be any more intense than the stuff OGS is already doing.
Would take a boatloadhealthy dose of pre-processing though!
anoek said something about the games not just sitting around in storage to be called at any time, but that they are generated on request each time.
I don’t exactly understand what that means or how it works… But I suppose it has something to do with saving space 
might be a speed bump for getting something like this off the ground, especially if you want it to periodically keep up with modern games and not simply stay static at a certain point.
The pattern search index files Kombilo generates are orders of magnitude larger than the input sgfs. In order to provide efficient pattern search your need to create far larger data structures than just the game tree itself. It also takes bloody ages to generate. So the resource demands of this feature seem pretty huge. I’d replace boatload with super tanker. Or maybe super-star-destroyer-ful.
Maybe OGS should invent a crypocurrency and doing this processing can be the proof of work computational task Might as well warm the planet doing something slightly useful.
Who is Kombilo?
Maybe I don’t understand what you’re saying, but I don’t think this is true. An efficient structure to hold this in would just be a tree, with each node holding a coordinate and # of games that hit that branch. It’s big (40M x avg number of moves), but it’s not bigger than the SGFs themselves.
Implementation would require thought, but it’s not a computational absurdity by any stretch.
Surely accounting for mirrors and rotation would add some amount of increased file size? It surely couldn’t just be a move tree…
I didn’t know waltheri had competition, thanks for the link 
Hah, waltheri is the new kid on the block, and sure is convenient for online searching of pro games away from my home PC. But it’s a smaller database than GoGoD and doesn’t have MY games. Kombilo (or crazy man Frank de Groot’s Moyo Go Studio) is how you do offline pattern search, and against your own sgf library. Which is why I want bulk download of my OGS games because then when I am reminded “Oh, this position is like that OGS game I played 12 years ago” I could whack it in kombilo and find the game. Back when I was teaching Go I would often use my OGS games as illustrative examples of concepts for students.