ENH: JupyterLite, WebGL

[westurner]

• https://github.com/emscripten-forge/empack
• https://github.com/emscripten-forge/recipes
• https://www.google.com/search?q=jupyterlite+emscripten-forge
• https://jupyterlite.rtfd.io -> Lab
  • https://jupyterlite.readthedocs.io/en/latest/howto/configure/simple_extensions.html#the-case-of-jupyter-widgets-and-custom-renderers
  • https://jupyterlite.readthedocs.io/en/latest/howto/configure/rtc.html#enabling-rtc-in-jupyterlite
• https://www.google.com/search?q=emscripten-forge+webgl

[nir]

I don't think it can be made to work without a SIGNIFICANT amount of work

[westurner]

Hopefully the compiler will do most of that work. Maybe @martinRenou knows of an example emscripten-forge empack package that has OpenGL/EGL code?

ipygany does 3D in notebooks with VTK, which may or may not already be transpiled to WASM IIUC
https://github.com/QuantStack/ipygany

• https://emscripten.org/docs/optimizing/Optimizing-WebGL.html#which-gl-mode-to-target

There may need to be a 'polyfill' like requests-wasm-polyfill?
https://github.com/emscripten-forge/recipes/blob/main/recipes/recipes_emscripten/requests-wasm-polyfill/recipe.yaml

• https://www.google.com/search?q=%22glfw%22+emscripten
• https://www.google.com/search?q=%22moderngl%22+emscripten
  • https://github.com/emscripten-core/emscripten/blob/main/system/include/GL/glfw.h
  • https://gist.github.com/ousttrue/0f3a11d5d28e365b129fe08f18f4e141?permalink_comment_id=4484709#gistcomment-4484709
  • -sUSE_GLFW=3 option

Students could easily develop and share STEM games with a WASM compilation of jupylet that works in JupyterLite or VSCode.dev

[martinRenou]

Maybe @martinRenou knows of an example emscripten-forge empack package that has OpenGL/EGL code?

I don't!

ipygany does 3D in notebooks with VTK, which may or may not already be transpiled to WASM IIUC

ipygany only uses VTK on the back-end for file loading, the rendering is done entirely using WebGL.

From time to time I google to see if Kitware folks provide an easy way to compile VTK for WASM, thankfully today you triggered that search and it seems to be fruitful https://gitlab.kitware.com/vtk/vtk-wasm-docker (initial commit a month ago).

Though I guess all of this VTK discussion is completely out of scope for jupylet, so sorry for the spamming. If you want, @westurner, I'd be happy to have this discussion under an ipygany issue.

[nir]

hi @martinRenou, you were one of the first people to star jupylet when it first came out. thanks! :)

[nir]

@westurner, I don't have the time to jump into such a project at the moment, but I see that jupyterlite supports numpy ipywidgets and ipyevents, and is using async just as classic jupyter, so actually it may be possible to put aside audio, 3d, and sharertoy support and just reimplement the sprites and labels modules and start with them and get an initial version of jupylet for jupyterlite and continue from there (that is how jupylet started anyway).

[westurner]

• Probably worth compiling to WASM and running the tests first
• This is the requests-wasm-polyfill, because actual requests is not built on the ES/JS fetch api:
  https://github.com/emscripten-forge/requests-wasm-polyfill/tree/main/requests

an example emscripten-forge empack package that has OpenGL/EGL code?

It looks like there's already glfw support in emscripten.

Is VTK actually out of scope? There probably could be a VTK backend; IDK what the advantage would be.

actually it may be possible to put aside audio, 3d, and sharertoy support

• W3C Web Audio API

  • https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API
  • https://github.com/topics/web-audio-api

• W3C Gamepad API

  • https://developer.mozilla.org/en-US/docs/Web/API/Gamepad_API
  • [Feature] W3C Gamepad API and controller tester pybricks/support#995

• W3C Sensor API

  • https://developer.mozilla.org/en-US/docs/Web/API/Sensor_APIs

• Like JupyterLite (and the vscode pyodide extensions) pyscript is also built on pyodide:

  • https://dev.jeff.glass/pyscript-audio/index.html
  • https://realpython.com/pyscript-python-in-browser/#sensor-api

• 3d

  • https://news.ycombinator.com/item?id=32657051 :

    • ENH: SensorCraft: replace Pyglet (OpenGL) with an alternate WebGL/WebGPU implementation

• Pybricks for MicroPython on smart LEGO® hubs, Wokwi simulator, MicroPython, CircuitPython quobit/awesome-python-in-education#50