CELT/stereo decode diverges from libopus (18–69 dB SNR) and sample alignment shifts at SILK↔CELT transitions

[ekhodzitsky]

CELT/stereo decode diverges from libopus (18–69 dB SNR) and sample alignment shifts at SILK↔CELT transitions

Summary

Two related conformance findings on the CELT side of opus-rs 0.1.23, from differential testing against C libopus 1.6.1 on real speech:

1. Stereo CELT decode is not bit-exact. On 48 kHz stereo 64 kbps files (pure CELT packets), output is time-aligned with libopus (lag 0) but SNR is only 18.3–68.9 dB depending on content (median windowed ≈ 60–65 dB, 10th percentile ≈ 40 dB). Mono CELT segments in mixed-mode files, in contrast, decode bit-exactly (corr 1.0) — so the divergence looks stereo-specific (M/S coupling, intensity stereo, dual-mono paths).
2. Sample alignment changes at SILK↔CELT mode transitions. In a mixed-mode stream the offset vs libopus jumps between −3 samples (SILK segments) and 0 samples (CELT segments) at 48 kHz; windowed cross-correlation shows the switch happening exactly at the packet where the mode changes. libopus keeps stream-continuous timing across transitions; opus-rs's output gains/loses samples at each switch (a libopus-compatible decoder must reproduce the exact per-transition delay of the reference).

Environment

• opus-rs 0.1.23 (crates.io tarball, sha256 1cc978697f3ada0ad98b323b9d85e85c6b08d478830148636c19b6d5f2ac9b85, verified against the sparse index; used as a path dep with identical content)
• rustc 1.96.0, macOS aarch64 (Apple Silicon), release build
• ffmpeg 8.1.1 with libopus 1.6.1 (reference encoder + decoder)

Reproduction

# stereo CELT file
ffmpeg -y -i speech.wav -c:a libopus -ar 48000 -ac 2 -b:a 64k celt_stereo.ogg
# mixed-mode file (SILK-WB with a CELT island)
ffmpeg -y -i speech.wav -c:a libopus -ar 16000 -ac 1 -b:a 24k mixed.ogg
# references
ffmpeg -y -c:a libopus -i celt_stereo.ogg -f f32le -acodec pcm_f32le -ar 48000 celt_ref.f32
ffmpeg -y -c:a libopus -i mixed.ogg      -f f32le -acodec pcm_f32le -ar 48000 mixed_ref.f32

Decode packets with OpusDecoder::new(48000, ch) (same ~25-line harness as in issue #1) and compare against the reference PCM.

Expected vs actual

A faithful libopus port should decode bit-identically (or SNR > 80 dB) and keep exact stream timing across mode transitions.

Actual (fresh runs, 2026-07-19):

• repro04_celt_48k_stereo_64k.ogg (4 files tested; worst shown): samples=384000 corr=0.9926 snr_db=18.3. Other files: 29.4 / 55.2 / 68.9 dB. Windowed (0.1 s) SNR on the worst file: median 59.3 dB, p10 40.2 dB.

• repro04_mixed_16k_mono_24k.ogg: raw corr=0.9886 snr_db=16.4; windowed best-lag analysis (0.1 s windows, corr at best lag in parentheses):

  window (s)	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9	1.0+
  best lag	−3	−3	−3	−2	0	0	0	0	0	−3 …

  The 0.5–0.9 s region is the CELT island (TOC 0xb8 packets): corr 1.0 at lag 0 (bit-exact, no offset). SILK segments: corr 1.0 at lag −3. The offset changes by +3 samples at SILK→CELT and −3 at CELT→SILK, i.e. the two modes have different internal delays and the transition does not preserve stream timing the way libopus does. (A longer mixed file with more transitions showed drift accumulating to −12 samples.)

Suspected root cause

• Stereo divergence: stereo-specific CELT paths — intensity stereo / dual stereo (dual_stereo handling in src/bands.rs ~line 2340), M/S normalization — differing from libopus's float build. Mono CELT being bit-exact in the same build points away from MDCT/PVQ basics.
• Transition shift: per-mode decoder delay not compensated the way libopus compensates it (libopus aligns SILK and CELT internal frames at transitions; the 3-sample figure at 48 kHz matches one internal 16 kHz sample — see also the constant −3 offset of pure-SILK streams).

Impact

For speech-to-text or media pipelines the ~18–70 dB divergence is usually perceptually tolerable, but it is not conformance-grade, and the sample-level timeline shift at mode transitions breaks any consumer that stitches decoded audio to a reference timeline (A/V sync, sample-accurate editors, echo cancellation reference). Both are blockers for "drop-in libopus replacement" claims.

Reproduction assets

Self-contained: the ffmpeg commands above generate both input files; packets are extracted with any Ogg demuxer (e.g. the ~15-line Python dumper from issue #1's reproduction section). (Pre-generated .ogg / packets.bin / ref.f32 for both files are kept as repro-04.zip; happy to attach or link on request.)