Standardize package distribution on web (also applies to BMS)

[dtinth]

Criteria:

• Easy to create.
• Easy to distribute.
• Playable on web?

---

Syndication

How to list available songs?

• Bemuse: Custom index.json file.
• CircularRhythm: Custom index.json music.json file.

Another alternative would be to use an Atom feed or RSS feed, like Podcast. Either as an <enclosure> or <link rel="enclosure" or a new XML namespace.

Package Style

Both Bemuse and CircularRhythm are using custom file format to save assets.

An alternative would be to use ZIP files:

1. Single ZIP file containing everything.
   • Have to redownload whole set on network error.
   • Have to redownload whole package when I want to add extra charts.
   • Have to download video when not needed.
2. ZIP archive containing resources, while note files are outside.
   • Have to redownload whole set on network error.
   • Multiple files to send.
3. A sequence of multiple ZIP files. Files in later archive overrides file in previous archive.
   • Multiple files to send.

Some problem with ZIP files:

• Don't know in advance which files are inside which ZIP file. For example, if player doesn’t support BGA it shouldn’t download bga.zip. May need to generate extra metadata file for web distribution.
• File name encoding problem.
• Browser may crash processing large ZIP files.

[zardoru]

Is it possible or practical to use tar files, given the case you don't want the compression? It is already a standard way to package and it does not compress by default.
I do prefer the 3rd option - a sequence that overrides files.

[dtinth]

@zardoru Oh. Tarballs! That’s a great idea. Unfortunately, it doesn’t officially accept non-ASCII file names. :(

---

@wosderge’s notes:
https://docs.google.com/document/d/1ym2Niq3A2pUC9YSWkMAXnYrC3JQN92LSgpSFML4rOgc/edit

---

I also found this thing on W3 recommendation. What do you think?
https://www.w3.org/TR/web-packaging/#streamable-package-format

[zardoru]

Sounds interesting - but given the example files, I doubt its fitness for raw binary data.
I'd still argue in favour of zip files, it is very easy to find a library that is able to handle them (for native players and tools), compression can be disabled, and it is ubiquitous.

[dtinth]

Now we also have this in browser: https://stuk.github.io/jszip/

Two years ago I use another library, https://gildas-lormeau.github.io/zip.js/, to decompress zip files in browser (the prototype version of Bemuse). It turns out to work very slowly — and crashes my iPad.

[dtinth]

If we use ZIP files, maybe some extra tooling is needed: A verifier/fixer which verifies:

• ZIP file’s filenames are in UTF-8 encoding.
• ZIP contents are not compressed.

I think it should be possible to create such tool. Therefore, I also agree about using ZIP files.