[Request]: Paravirtualized Graphics via virtio-gpu

[MisutaaAsriel]

Feature or enhancement request details

Note: This has also been asked for before, broadly speaking, but in a very vague sense, with multiple possible interpretations in regards to implementation. This feedback seeks to be more narrow and targeted in its request.

My understanding at this time is that Apple Containers & its containerization framework lack support for any form of graphical display or graphics acceleration.

This is understandable to a degree, as Apple's containerization framework & utilities are intended to be a replacement for Docker & kin, which themselves lack any direct graphical output capabilities, and as containers are not intended to replace a full virtual machine.

Ignoring AI, which is the most commonly cited use case, there is nonetheless an argument to be had for using containers, rather than a full desktop VM, for hardware-accelerated and graphically-interactive tasks.

For example, the development and testing of graphical linux utilities in containerized environments.

• Spinning up a full VM requires installation media, virtual drive allotments, hacky host shell integration with serial terminals and spice drivers, excessive amounts of setup, and even then, there may be awkward limitations or unwanted assumptions made by the VM software itself which impact performance, portability, and reliability.
  • Comparatively, containers are extremely simple to spin up, configure, etc.
• A user may wish to test their software between different distributions and environments in rapid succession which a VM may not be conducive for.
• When using a host-side Wayland or X11 compositor with the appropriate sockets exposed to a container, a user may be able to directly integrate any graphical Linux utilities with their host system, allowing for better performance and reliability.

Another example: for shipping and running Linux exclusive software on macOS.

• Container images could be built which take advantage of GPU acceleration through virtio-gpu, allowing for the same "ship the environment" approach to be applied to graphical linux applications on Mac, similar to Flatpaks are themselves on Linux.
• Many graphical utilities & tools may only be available on Linux platforms, and porting may be difficult to infeasible as opposed to running the applications in a virtualized environment.
• Even when command-line utilities exist for a task, graphical utilities may be more comfortable to use, even for the technically minded.

Containers also make it easier to "mix" native ARM & x86 applications, and spin up x86 distributions, over that of a full desktop VM.

Importantly, virtio-gpu in Linux is incredibly mature, receiving support as far back as the Linux 4.4 Kernel, and is the primary means of graphics acceleration for virtualized Linux tasks with tools like QEMU.

• VM frontends such as UTM further build upon this with ANGLE based OpenGL ES hardware translation, for full Metal rendering support.
• MoltenVK may also be leveraged for accelerated Vulkan graphics, as does other Qemu variants, such as the ADV, use.
  • virtio-gpu provides support for the Vulkan API via vulkan-virtio and the virtio-gpu Venus protocol, permitting kernel support has been enabled.

Whilst this would not fulfill everyone's requests when it comes to GPU access, as this would be paravirtualized, with some level of translation, as opposed to "full" GPU "passthrough" some have asked for, this would arguably be simpler to implement, and more readily align with Apple's goals.

Likewise, changes to the frontend set of utilities over at apple/container would need made to display rendered content (when specified), alongside the ability to enable such paravirtualized graphics acceleration in any given container.

• Keeping graphical output and graphical acceleration as separate flags allows for a broader set of use-cases.

I shall include an excerpt of this request in apple/container#62

Code of Conduct

• I agree to follow this project's Code of Conduct

[davidchisnall]

To give a concrete example for this use case:

FPGA toolchains are typically distributed as Linux or Windows binaries. To run Vivado on my Mac, I currently use Docker and XQuartz. Setting this up is quite painful. The install requires some interactive steps and can’t be redistributed and setting up the X forwarding from the container requires some more tweaks.

At some point, I suspect these tools will move to Wayland and then the remote X11 option goes away and this use case will become much harder.