Summary
Enable hardware video encoders to accept VideoData::Gpu directly from the compositor, completing the all-GPU pipeline path: decoder/compositor → encoder with zero CPU pixel touches.
Depends on: #350 (GPU VideoFrame variant)
Parent issue: #167 (web renderer context), but this is an independent pipeline improvement
Per-Encoder Work
| Encoder |
File |
Interop Mechanism |
Effort |
| Vulkan Video |
vulkan_video.rs |
Same wgpu Vulkan device → native VkImage |
M |
| NVENC |
nv_av1.rs |
cuImportExternalMemory (CUDA↔Vulkan interop) |
L |
| VAAPI |
vaapi_h264.rs, vaapi_av1.rs |
Export wgpu texture → DMA-BUF → vaCreateSurfaceFromDmaBuf |
L |
Recommended order
- Vulkan Video (most natural fit — same Vulkan device as wgpu, no cross-API interop)
- VAAPI (DMA-BUF is well-supported on Mesa)
- NVENC (CUDA↔Vulkan interop is more complex)
Architecture
Current path:
Compositor (wgpu) → readback_rgba() → CPU Vec<u8> → encoder upload → encode
Proposed path:
Compositor (wgpu) → VideoData::Gpu(wgpu::Texture)
→ encoder receives texture directly → encode (no CPU touch)
Acceptance Criteria
Complexity
Effort: L (1–2 weeks per encoder)
Each encoder has different interop requirements. Vulkan Video is simplest since it shares the wgpu Vulkan device.
Notes
- This is the final piece of the all-GPU pipeline: web renderer/decoder → compositor → encoder with zero CPU readback.
- Can be implemented incrementally — start with Vulkan Video, add VAAPI and NVENC later.
- The encoder should gracefully fall back to CPU readback if it receives
VideoData::Cpu (backward compatible).
Summary
Enable hardware video encoders to accept
VideoData::Gpudirectly from the compositor, completing the all-GPU pipeline path: decoder/compositor → encoder with zero CPU pixel touches.Depends on: #350 (GPU VideoFrame variant)
Parent issue: #167 (web renderer context), but this is an independent pipeline improvement
Per-Encoder Work
vulkan_video.rsVkImagenv_av1.rscuImportExternalMemory(CUDA↔Vulkan interop)vaapi_h264.rs,vaapi_av1.rsvaCreateSurfaceFromDmaBufRecommended order
Architecture
Acceptance Criteria
VideoData::Gpuinputto_cpu()Complexity
Effort: L (1–2 weeks per encoder)
Each encoder has different interop requirements. Vulkan Video is simplest since it shares the wgpu Vulkan device.
Notes
VideoData::Cpu(backward compatible).