Problem Description
The previous uniform CSS blur filter applied a blur across the entire <video> element, blurring both the speaker and their surroundings. Additionally, WebRTC's native backgroundBlur constraint requires OS- or hardware-level camera driver support (e.g., macOS Apple Silicon / Windows 11 Studio Effects) which is unavailable on Linux and most standard USB webcams. Furthermore, DOM CSS filters do not apply to the raw camera stream captured by MediaRecorder.
Objective
Implement true virtual background blur (bokeh/portrait mode) where the speaker remains crisp and in focus while only the background/surroundings are blurred, with support for both the live preview and the recorded video export.
Architectural Options to Choose From
Option 1: Client-Side AI Segmentation via Google MediaPipe (Recommended for Web)
- Technology: Google MediaPipe Selfie Segmentation (
@mediapipe/selfie_segmentation or @mediapipe/tasks-vision).
- Pipeline:
- Webcam feed frames are processed via WebAssembly / WebGL.
- The model outputs a human silhouette alpha mask at ~30 FPS.
- An off-screen
<canvas> composites the feathered mask: blurring background pixels and layering the sharp person in front.
canvas.captureStream() is fed into MediaRecorder so the exported video take includes the bokeh effect.
- Pros: Runs 100% client-side in the browser, completely private (0 bytes sent to any server), high accuracy and frame rate.
- Trade-offs:
- Requires fetching the lightweight model weights (~3–4 MB).
- Can be loaded via Progressive Enhancement (dynamically loading scripts only when AI Blur is toggled on) to keep the core teleprompter 0-dependency and offline by default.
Option 2: WebCodecs + ONNX Runtime Web / Transformers.js
- Technology: Run an open-source segmentation model (e.g. MobileNet / DeepLabV3) through ONNX Runtime Web with WebGPU acceleration.
- Pros: Flexible model selection, potential WebGPU acceleration.
- Trade-offs: Larger download footprints and higher complexity compared to MediaPipe.
Option 3: External Virtual Camera Documentation (OBS Studio / v4l2loopback)
- Technology: Documenting workflows for users to use OBS Studio (
obs-backgroundremoval plugin) or dedicated webcam software (Camo, Elgato) and select the virtual camera device in the teleprompter.
- Pros: 0 additional lines of JavaScript, 0 client-side bundle size increase, hardware accelerated with zero browser overhead.
- Trade-offs: Requires user to run external software.
Key Technical Considerations
- Repository Philosophy: Maintain the zero-dependency, 100% offline nature of the base prompter by lazy-loading any ML segmentation scripts on-demand.
- Performance: Run segmentation at a lower resolution (e.g. 256x256 or 512x512) and upscale/feather onto the 1080p canvas to maintain smooth 30 FPS without overheating or dropping frames.
- Canvas Recording: Ensure the recorded
MediaStream originates from the composite canvas rather than the raw getUserMedia stream when portrait blur is active.
Problem Description
The previous uniform CSS blur filter applied a blur across the entire
<video>element, blurring both the speaker and their surroundings. Additionally, WebRTC's nativebackgroundBlurconstraint requires OS- or hardware-level camera driver support (e.g., macOS Apple Silicon / Windows 11 Studio Effects) which is unavailable on Linux and most standard USB webcams. Furthermore, DOM CSS filters do not apply to the raw camera stream captured byMediaRecorder.Objective
Implement true virtual background blur (bokeh/portrait mode) where the speaker remains crisp and in focus while only the background/surroundings are blurred, with support for both the live preview and the recorded video export.
Architectural Options to Choose From
Option 1: Client-Side AI Segmentation via Google MediaPipe (Recommended for Web)
@mediapipe/selfie_segmentationor@mediapipe/tasks-vision).<canvas>composites the feathered mask: blurring background pixels and layering the sharp person in front.canvas.captureStream()is fed intoMediaRecorderso the exported video take includes the bokeh effect.Option 2: WebCodecs + ONNX Runtime Web / Transformers.js
Option 3: External Virtual Camera Documentation (OBS Studio / v4l2loopback)
obs-backgroundremovalplugin) or dedicated webcam software (Camo, Elgato) and select the virtual camera device in the teleprompter.Key Technical Considerations
MediaStreamoriginates from the composite canvas rather than the rawgetUserMediastream when portrait blur is active.