Josekle Development

oh good call, I didn’t realize spoilers could work like that. fixed

3 Likes

Might be because I added this wait for the dictionary, because without I was seeing problems replaying saved guesses

However, I can’t reproduce the original problem now :face_with_monocle:

Meantime I added a license file and moved credits around, not exactly sure the right way to attributed multiple authors, so feel free add/remove yourself as you feel comfy @yebellz @Jon_Ko

1 Like

It should be really easy to create a 9x9 Fusekle fork. The only non-trivial thing is compiling a list of solutions. Then, it’s just a matter of putting those into a puzzles.js file, adding a “size=9” tag to the div with id=“input-board” in index.html, and disabling the board zoom (setting default zoom to full, and removing the buttons).

However, incorporating some sort of AI-based scoring would require a lot more work.

3 Likes

Is there a way to have it just be a switch to a different “mode”, rather than a separate fork? (since Josekle is in active development it seems like it would be easier to keep it all in one place, but I’m not really familiar with the forking workflow so maybe I’m misjudging what’s easy and not)

In any case, like I mentioned in the other thread, I think 9x9 fusekle with standard hint system would be a fun game to play, and I’d happily contribute some solutions. Can I just input lots of variations into one sgf-file, with all leafs being potential solutions?

2 Likes

We can do something like the one color mode, a query parameter flag that is checked on page load and switches to 9x9. Bit of a hack but it works (I should have just started this project with React and some routing thing but that seemed over-engineering)

This also works, I can process it for solutions like we are planning to do soon to expand the solutions for regular Josekle. But it might be a few days until I have time to do that

2 Likes

It could be possible to do some things, like @okonomichiyaki just said, with parameters passed through the URL (to do various things like set board size, select a different solutions file, and disable zoom). Or, it could be a separate fusekle.html file with static modifications.

With a forked repo, it is still possible to pull down new changes from upstream, but the forker has to figure out how to merge in new code and potentially deal with resolving any conflicting changes.

I think the easiest thing to manage at first is just having a separate fusekle.html file within the repo, and this prototype would help us better consider how to integrate more tightly.

Yes, that would be perfect. We can process it from there.

I could take care of this pretty quickly. Maybe I could set up an editor.html page which just displays the BesoGo editor, and I could modify the “file panel” to add a new “export” button that generates the puzzles.js file from the current game tree (which could be loaded from an SGF).

Actually, what’s the status of the joseki dictionary SGF? In the OJE, I guess that not every childless leaf node is a valid joseki (since some leafs are for showing/commenting on mistake variations), and there are some joseki that are entirely prefixes of other joseki. What annotation are we using to mark which nodes as the ends of joseki? Eventually, I could modify the behavior of the above proposed “export” button to behave according to such annotation as well. This would give us an automated way to convert of an SGF-based joseki dictionary into a set of josekis in our desired format.

2 Likes

All sounds good!

Let’s say I give a small sgf to start with, and then a few days later continue adding to that same sgf (because that seems like an easy way for me to track what is already added).

Is it then possible to do a second export while still remembering which variations have already been used as solutions in the first few days? I. e. is it convenient to gradually add new solutions over time?

3 Likes

That’s a great question, and in short, yes, it can and should be done. I have some thoughts on how to do this, but I need some more time to write it out.

1 Like

good idea

The dictionary file we’re using right now (under debug=true mode) is just a raw scrape of the OJE tree, without annotations. My thinking was that a dictionary like this can just be used to validate inputs, and permit any guests that matches any path. Important note:

I’m not scraping any paths marked with a _ or anything marked as a mistake, the dictionary only includes those marked with numbers or letters AND marked as “good” or “ideal”

My plan was to use the OJE tag “Joseki: Done”. This doesn’t really correspond to any SGF annotation, I guess I was imagining that the dictionary in SGF is separate from the list of puzzles, but they are both produced by the scraper. Reason being this way its easily reproducible (just feed JSON into the script and produce a new dictionary and new set of puzzles).

Basically what we are discussing is a new input to this script in the form of SGF. So probably what makes senses to refactor the scraper to produce a dictionary in SGF, with leaves annotated. We could then use the editor.html or another script to produce the list of puzzles. so in other words:

OJE JSON from disk or network => scraper => SGF with annotations based on Joseki: Done
any SGF with annotations => puzzle parser => puzzle list

Right now I need to re-scrape anyway to get the “new” data (since I’m saving raw JSON, this can happen while we work out the details. the biggest issue here is time since I want to tread lightly on OGS API)

my only concern is if we use editor.html as the puzzle parser in this system, it is a little cumbersome (have to run the scraper, then take SGF and open browser, upload, download, etc) and not easy to have a one step process

1 Like

Sorry didn’t realize this thread had moved. What are your criteria for curating solutions?

Length, point loss allowed, engine settings, popularity, etc?

2 Likes

I was thinking just any variation of around 6-10 moves where KataGo steadily evalutes the game as a draw under area scoring, 7.0 komi. I’d begin by inputting common variations from GoQuest, but to get enough variety I think it would be preferable to not have any particular requirement that the opening is popular. We’ll see how far I get with popular openings first though.

2 Likes

I was thinking popular imperfect openings could still make the list with their optimal AI follow-ups. Theyd basically be like Chess gambits that way

2 Likes

We have two things to deal with:

  1. Joseki dictionary for checking valid inputs against.
  2. Managing and curating a list of solutions.

I think that the first should be handled by just scraping the OJE into a local copy of valid joseki sequences in an SGF format. This never needs to be manually edited with the editor tool that I’m proposing, and it would be straightforward to use BesoGo to parse this SGF and check inputs against. Actually, we could refactor that a bit to simplify away the need to create a hidden div, and instead just create a headless editor in the background.

The editor.html tool would be just for helping to curate the list of solutions. For that task, I actually think it would helpful to have the solution set in both SGF and puzzles.js format, since we want to simultaneously address the following concerns:

  1. We need to track and preserve the sequence of joseki solutions, to avoid accidentally repeating things as we perform ongoing curation for new solutions.
  2. We should randomize the order of new joseki being added (to avoid predictable patterns that arise from the procedural extraction from a tree structure).
  3. We should provide a convenient way for many to examine the solution set to add new solutions without overlapping with existing ones.

Before getting into the details of how to do that, let me discuss an aside about SGF formatting.

I think that we should use the SGF comment tag, and include the word “joseki” in the comment. So, just add “C[joseki]” to a node to indicate that it is the end of a joseki, and we can follow the convention that any childless leaf node is automatically considered a joseki (hence, we don’t record any non-ideal or mistake variations).

Similarly, for the solutions SGF file, we can use a “C[joseki]” tag to mark solutions that are prefixes of other solutions. However, for the solutions SGF, we could also add other words like “easy”, “hard”, “ai”, “modern”, “classical”, etc. (not sure about the exact list) to further annotate and facilitate solution subset curation. People could also even record the OJE node number, for further reference. Hence, a comment tag on a node might look like “C[joseki hard classical 42351]”, as simply separating these keywords with spaces is enough to automate later filtering.

To handle this problem, I think the editor.html would need to not only export to the puzzles.js format, but also be able smartly append to only new sequences to an existing puzzles.js file, while preserving the existing sequence of solutions already present. This should be straightforward to do with a new interface button called “append” which brings up a file open prompt to load the existing puzzles.js file and then return the amended puzzles.js file.

Finally, when starting or appending to the puzzles.js file, I think that the sequence of new items should be written in a randomized order, since otherwise, whatever ordering chosen by a particular tree-traversal technique would lead to similar joseki patterns appearing on consecutive days. This would also help the composer of new puzzles still have some degree of surprise as to exactly when they might see the new solutions that they just added.

1 Like

I don’t think we should have any implicit assumptions about childless leaves. Instead I think we should explicitly mark josekis. The reason for this is that I’m incrementally scraping OJE up to a certain depth (builds a tree with height N, then later can come back to scrape more to N+1, etc), which naturally results in partial sequences. We wouldn’t want those to be assumes to be josekis

I’m a little bit hesitant to come up with a new scheme for curated categories for the basic 19x19 version…I think the best way to do this is actually add these categories to OJE, or expand on them.

I think we should maybe separate the goals for 9x9 vs 19x19 versions. Are there actually a large number of 19x19 joseki missing from OJE that folks want to curate? It would have to be a significant number to make a dent, I had 500+ josekis at depth 13-14 just from OJE alone. It sounds like we mostly need manual curation mostly for 9x9?

1 Like

Some users who aren’t in the forums have the following feedback to share:

“Josekle slapsssss”
“i really like josekle too! I didn’t think i would since i’m hoplessly bad at joseki”
“last night I dreamed about doing josekle.”
“JOSEKLE IS A THING? THAT’S SO COOL”
“Josekle is honestly a super exciting project“
:triumph::fire::triumph::triumph::triumph::triumph::triumph::fire::triumph::triumph::fire::fire::fire:“ (I’m pretty sure this was a positive comment)

And that that’s just from my Discord server, and it’s already taken off in the BeginnerGo discord server too :eyes: everybody loves it as much as I do apparently :eyes::eyes::eyes:

9 Likes

:joy: I actually had a Josekle dream too a while back but can’t remember it. I think it was more related to code than joseki :nerd_face:

2 Likes

btw, do we want to use the current format long term? it’s perhaps not ideal but it does work… I think it should maybe just be a JSON blob not a ‘.js’ file but open to suggestions

1 Like

Good point. It does not hurt anything to just explicitly mark all joseki with “C[joseki]”, and seems to make a lot of automated tasks easier.

I am not suggesting that we curate for joseki missing from OJE. The dictionary for valid joseki should all just be automatic.

However, I think we have to figure out how to select the subset of valid solutions. We could define some criteria (like just length, or purely random from all) to automatically select the solution joseki. Or do we want to continue manual curation of the 19x19 solution joseki?

At least for supporting the 9x9 fusekle variant, it would be useful to have such tools.

Yes, I agree as well. I was actually thinking the same thing, but just forgot to mention it. Makes it much easier to just use the built-in JSON parsing and writing functions.

Definitely not. My plan is to do something like this:

  • finally get around to filtering out the avalanche variations from the current list
  • this will give us a lead time of plenty of relatively easier joseki for daily puzzles for a bit
  • in the meantime come up the automatic solution
  • make one push to cut over (after testing) to the new system

My thinking is, after running out of the manual collection we have now, to switch things and start with 2 difficulties:

  • normal which is just the firehose of everything possible in the puzzle set
  • easy which is limited by move length (6 or less?)

(I’m open to swapping these around to easy=>normal, normal=>hard)

the reason to split this way is that I think it’s easier (:upside_down_face:) to manually add some well known longer joseki (eg 3-3 double hane) to the easy collection than to curate a hard collection

and this way as we expand the collection the hard mode will grow in size and interestingness, and maybe easy mode will be slightly static. assuming there’s enough joseki of course, not sure what the breakdown is but will find that out soon

I’m certainly open to ideas. My main concern is, that we want to make it harder and more interesting for some folks without putting off beginners. Basically hopefully everyone can enjoy one of these 2 modes. I think more than 2 modes in the “regular non variant Josekle” is too many to manage

1 Like

I Advertised yesterday on kgs and got a “so awesome!” feedback

6 Likes