Perf: Move resampling/VAD off main thread (AudioEngine onmessage spikes)

[ysdede]

Problem

Audio chunk handling on the main thread is still doing resampling + VAD + buffer updates in workletNode.port.onmessage. In the trace this handler spikes to ~20ms, which can drop frames even before any long-run effects.

Evidence

• Trace: workletNode.port.onmessage (AudioEngine) max ~20ms, avg ~0.7ms across ~184 calls.
• Main-thread Handler includes resampleLinear, ring buffer writes, VAD, segment processing, UI updates.

Proposed change

• Move resampling and/or energy/VAD computation into the AudioWorklet or a dedicated worker.
• Make the main-thread handler minimal: enqueue chunk, update counters, and return.
• Reuse scratch buffers for resampling to avoid per-chunk allocations.

Acceptance criteria

• Max workletNode.port.onmessage handler time <5ms under normal load.
• Main thread no longer does heavy resampling and VAD per chunk.
• CPU usage at startup (fresh session) drops measurably.

Files

• src/lib/audio/AudioEngine.ts
• AudioWorklet processor in AudioEngine (inline processor code)

[ysdede]

Resampling moved into AudioWorklet in release/r1 (worklet now posts resampled 16k chunks + maxAbs). Main thread still runs VAD/state machine, so leaving this open for moving VAD off‑thread or further trimming handler time.

[ysdede]

Status Update (Feb 12, 2026)

Resampling is now fixed — moved into the AudioWorklet processor (AudioEngine.ts lines 221–234). The worklet performs linear interpolation resampling from device rate → 16kHz before posting the chunk, so the heavy resampling path no longer runs on the main thread.

What remains open:

• VAD processing (AudioSegmentProcessor.processAudioData()) still runs synchronously inside handleAudioChunk() on the main thread (line 515)
• Energy calculation, SMA smoothing, visualization buffer updates, and segment validation also still run on the main thread
• These are lighter than resampling but still contribute to onmessage latency

Recommend narrowing this issue's scope to: 'Move VAD + audio processing off main thread.'

[ysdede]

Status check: Resampling is confirmed to be off-thread (moved to Worklet), but the VAD processing (�udioProcessor.processAudioData) still runs synchronously on the main thread within handleAudioChunk. This remains a valid performance concern.