ngff repo structural changes

[jo-mueller]

Hi all,

in the past weeks/months, we saw a few issues accumulating here pertaining to the structure and appearance of the repo/webpage:

• The current webpage uses the readthedocs branding Ad-free website #393
• Switching between versions on the ngff page is hard because the specification is only available as a static artifact that's generated by the bikeshed build workflow No easy way to switch between versions of the specification on website #345
• It's unclear how exactly to contribute to the specification text/schemas or examples Provide contribution guide on how to proposed changes to JSON schemas #323 Which copy of the SPEC should changes be proposed against? #276
• Some content is hard to find because the pages don't link it Link to the JSON schemas from the spec #184 Add example data links to ngff specification #51

I think the source/solutions to these issues fall under a few categories:

Bikeshed & static html

While powerful, the inherent "flaw" of using bikeshed in this context is that we are mixing together multiple markdown flavors (sphinx and bikeshed) that don't go so well with each other. As a result, we are building the html-rendered pages of the spec document and serving them as static extra pages with sphinx. On the one hand, this gives access to URLs like ngff.openmicroscopy.org/0.5. On the other hand, these html pages are not part of the website's actual toctree, which makes them hard to access/navigate.

Similarly, there are html pages (like https://ngff.openmicroscopy.org/0.5/schemas/) that aren't linked anywhere on the main page so you kinda have to know that they exist top find them.

Solution The obvious solution here, would be to deprecate bikeshed, convert the index.bs to "normal" markdown and serve them through the sphinx build pipeline.

Readthedocs

Imho, the real power of RTD shines when publishing documents from a versioned repo since RTD provides that nice dropdown in the buttoms with which you can flip between versions easily. Since we are not doing this here on the ngff page, that feature isn't helping us much. RTD does provide a nice pre-rendered version of docs in every PR, which is a valuable thing to have.

Solution: We could think about deprecating RTD and use gh pages instead. I think @lubianat knows of some magic tools to also provide PR preview pages with gh pages, which I think is our primary feature of interest with RTD.

Submodules

Currently, the submodules link out to different branches of this repository, whereas the branches have little in common with the main branch. There's also no real active development head for new spec versions, which led (and still leads) to some confusion.

Solution: Since this makes contributing and managing forks of ome/ngff a bit difficult, we started exploring breaking out the spec content (consisting of schemas, examples and the spec itself) to a different repo - ome/ngff-spec. The latter uses myst/jupyter-powered markdown to build pages (which is really just sphinx with extra steps). Fully endorsing this as the place for the spec to live would require these steps:

• Copy spec versions (0.1-0.5) over there on different branches
• Convert to markdown (cheap convert: Replace headlines with h1, h2, etc and update crosslinks)
• Tag and release these old versions from their own branches
• Some smaller structural changes over at ngff-spec to get clean URLs for the spec on the ngff page Splash repository ngff-spec#36

Once this is done, we could replace the submodules here under the root level and introduce new submodules under ./specifications/0.5/<content of 0.5 branch on ngff-spec>. Since the latter is just a bunch of markdown, we can just add these md files to the toctrees over here and have a fully rendered ngff page. This is how this could (more or less) look like. Any development of the spec would then just include a PR to ngff-spec/main and a tagged release from there would constitute a new spec release. Done.

Embrace myst/jupyter book (optional)

Further down the line, moving more of the doc work in here to myst/jupyter-book would give way to a few more nice styling options. For once, it's easier to create navbar elements, that could move the contents of the primary navigation bar on the left of the ngff page to a new header section. In turn this would make room for less cluttered navigation in the primary sidebar. For instance, the different spec versions could take up that space and make flipping through versions a bit easier. For an example of how that could look like, check out the ro-crate specification page.

Myst supports the toctree directives used throughout the pages here, so we would just have to create a myst.yml file with the basic site layout and convert the index.rst to index.md.

Again, the last item (myst conversion) is entirely optional and not a must to the solution of the above-linked issues.

cc @lubianat @joshmoore @dstansby @govekk @clbarnes @gouttegd ...who else was routinely engaging with documentation stuff here on the page?

Anyway, LMKWYT :)

[lubianat]

On GitHub PR previews:

Native support for that has been a long running discussion:

• https://github.com/orgs/community/discussions/7730

It seems like it has been stalled for ~2 years.

 preview:
    description: 'Is this attempting to deploy a pull request as a GitHub Pages 
preview site?
(NOTE: This feature is only in alpha currently and is not available to the public!)'
    required: false
    default: 'false'

Many people have done workarounds, which usually includes some kind of GitHub Actions workflow building the PR in a folder inside the gh-pages branch

Here is an example of it in practice: German-BioImaging/ome-zarr-acknowledgments#21

and
https://github.com/German-BioImaging/ome-zarr-acknowledgments/tree/gh-pages

The preview cleans up after itself once the PR is merged. It is a very simple case, though, I don't know if the approach scales well (but I think it should)