resynthesis-watermark-limits

A Subspace Model of Speech-Watermark Survival Under Analysis–Resynthesis.

A reproducible signal-processing research project (targeting IEEE ICASSP). The surrogate theory and its numerical sanity checks are CPU-only; the real-speech experiments use pretrained neural generators on a GPU. The numerical implementation is sanity-checked against closed-form quantities by the test suite; whether real attackers behave like the model is an empirical question answered by held-out experiments — never assumed.

TL;DR — Speech watermarks are widely observed to fail when audio is regenerated by a vocoder or neural codec. We model a blind launderer as an analysis–resynthesis channel W = S∘A and are precise about what can be said at each level:

1. Any channel (Proposition 1). A data-processing inequality — laundering can only lose detectability — plus an exact-erasure condition: if the embedding leaves A(x) unchanged almost surely, the laundered laws coincide and no detector beats chance. Nothing stronger is claimed for general nonlinear generators.
2. Linear-Gaussian surrogate (Theorem 1). Closed forms: the channel transmits exactly the perturbation's equivalence class modulo ker A; the surviving detection exponent is ⅛‖Pδ‖². Mixed perturbations survive partially; nullspace ones are erased.
3. A rate LOWER BOUND R_LB for blind (host-uninformed) Gaussian embedders — not a watermarking capacity: informed embedding (Costa; Moulin–O'Sullivan) can exceed it, and no rate converse is claimed.
4. A measurable survival predictor. For each real attacker, its own analysis sensitivity s_W(δ;x) = ‖A_W(x+δ)−A_W(x)‖/‖δ‖. Whether s_W predicts post-attack separability is a falsifiable hypothesis — tested on held-out, speaker-disjoint data against competitor predictors with permutation controls.

Not claimed: "fundamental limits" of speech watermarking; a matched converse–achievability pair; that the survivable set is "exactly" the non-nullspace; that any deployed watermark is "information-theoretically erased". Where a deployed detector fails under attack, we say precisely that: an operational failure (score sign/order inversion under the original decision rule), often with substantial attack-conditioned separability remaining.

Invariant/latent-aligned embedding is not our idea — it is established practice (Latent-Mark, Feature-Aligned SW, VoiceMark; in images, Zhao et al., Tree-Ring, Gaussian Shading). What this repo adds: the stated channel model with exactly-scoped guarantees, an explicit account of what is not proven, a channel-relative predictor validated out-of-sample, and a calibrated operational evaluation protocol for deployed watermarks.

---

The model in five lines

Detection is a hypothesis test between the clean law P₀ and the watermarked law P₁; its optimal error exponent is the Chernoff information C(P₀,P₁). A blind launderer applies W = S∘A (analysis to a representation z, resynthesis from z).

• DPI (any W): C(W#P₀, W#P₁) ≤ C(P₀,P₁).
• Exact erasure (any W): if A(E_κ(x)) = A(x) a.s. and S sees only z, then W#P₁ = W#P₀ → C = 0.
• Surrogate (x~N(0,I), W(x)=Px+(I−P)w): W#N(δ,I) = N(Pδ,I), so C = ⅛‖Pδ‖² exactly — the channel keeps only [δ] ∈ ℝⁿ/ker A.
• R_LB: water-filling rate of the invariant sub-channel for blind Gaussian embedders. A lower bound, not capacity.
• Real channels: measure s_W with the attacker's own analysis; test whether it predicts oriented separability. That correspondence is the paper's empirical claim.

Full statements and proofs: docs/THEORY.md.

---

Evaluation protocol (what makes the numbers trustworthy)

• Raw AUC and oriented AUC max(a, 1−a) are both reported. A raw AUC < 0.5 with high oriented AUC is a score inversion = operational failure, never "erasure" (it did not occur anywhere in the final protocol, but the machinery flags it if it does). Theory statements attach to oriented separability only.
• Operating points (TPR and achieved FPR at 1% FPR) use thresholds fixed only on an independent calibration split of 7,157 clean negatives (dev-clean + dev-other + test-other, all speaker-disjoint from the test-clean evaluation split). The test split never touches threshold selection. Counts carry Clopper–Pearson 95% intervals. An attack-aware recalibration row is reported as a labeled diagnostic.
• Cluster bootstrap: AUC CIs resample utterances (paired pos/neg per cluster), never independent pos/neg shuffling.
• Perceptual budget reported (not force-matched): the deployed baselines run at their native full strength — all transparent but not PESQ-equalized (clean-embed PESQ 4.42 / 4.27 / 4.60, SI-SDR 26 / 37 / 34 dB for AudioSeal / WavMark / SilentCipher). We do not force a common PESQ (which would push each scheme off its designed operating point) and we disclose that AudioSeal carries the most energy, so its greater survival is partly a strength effect. Constructed marks (E2) are scaled to a common PESQ-WB 4.2 budget via scale_to_pesq (achieved PESQ/SI-SDR/SNR recorded per instance and used as competitor predictors).
• Speaker-stratified manifests (seeded): ~1000 test clips from all 40 test-clean speakers (voice-active random segments, not file heads), multiple keys, 3 seeds.
• Fail-loudly: formal runs abort if any attacker/baseline/model is missing — results never silently drop a channel.
• Constructed marks are verified, not assumed: the kernel-direction construction (alternating projections onto quadrature ∩ consistent spectrograms) reports its measured analysis-change ratio vs a random control (≈ 0.16× at operating amplitude for the kernel mark vs ≈ 2.3× for the row mark — real numbers from the run).
• The genie-aided matched-filter probe used in the geometry experiment is a diagnostic, clearly labeled; deployed-detector conclusions come only from the real baselines' own detectors.

Attackers and baselines

Attackers (W)	Baselines
STFT-GL (near-lossless control)	AudioSeal (16-bit, prob. detector)
mel-GL — inversion of THE unified 80-mel analysis	WavMark (16-bit payload matcher)
Vocos (neural vocoder), Vocos-EnCodec (codec-token reconstruction)	SilentCipher (44.1k path; the released 16k model is unreachable through the package's byte API — documented)
EnCodec 6/3 kbps, DAC 16k, SNAC 24k	+ constructed kernel/row marks (geometry probes)
kNN-VC self voice-conversion, top-k ∈ {4, 8}

Status — pre-submission fix batch in progress

The E2 protocol was upgraded to (i) a common PESQ-4.2 perceptual budget for every constructed mark (was per-utterance random SNR), with achieved PESQ/SI-SDR/SNR added as competitor predictors; (ii) cluster-aware inference — a common whole-direction permutation across attackers and a two-way (direction×utterance) cluster bootstrap; and (iii) stable seeding (SHA-256 keys, fixed NumPy/Torch/CUDA seeds, rand_init=False Griffin–Lim). The E2 headline numbers below are from the prior random-budget run and are being regenerated under the corrected protocol; they are retained only if they survive the recompute. E1 and E3 are unaffected. See experiments/audio/e2_predictor.py and experiments/audio/repro.py.

Results (headline)

E1 — deployed watermarks (N=1000 clips, each at native transparent strength, PESQ 4.42/4.27/4.60). All three baselines retain strong performance on the near-lossless STFT-GL control (AUC ≈ 1.0, TPR ≈ 0.9–1.0) and lose their calibrated operating point under nearly all nine lossy channels, in four distinct outcomes:

Mode	What happens	Where
Erasure (separability + operating point both lost)	oriented AUC → 0.50, TPR → 0, unrecoverable	SilentCipher (continuous detector, proper CIs — a measured collapse) under all nine lossy channels; AudioSeal under SNAC and self-VC. (WavMark's payload decoder desyncs to a constant 0 → AUC exactly 0.50 with zero-width CI: a deployed-decoder sync failure, consistent with but not itself a measurement of collapse.)
Operating-point loss (weak separability retained)	AUC 0.54–0.63 (CI above 0.5) but TPR ≤ 0.07	AudioSeal under mel-inversion, Vocos, Vocos-EnCodec
Calibration failure (strong separability retained)	AUC 0.93 / 0.81 but achieved FPR explodes 0.01 → 0.73 / 0.80 as the codec pushes clean audio across the boundary	AudioSeal under EnCodec 6k / 3k
Graceful degradation	AUC 0.93 + calibration retained (FPR 0.01), recall reduced (TPR 0.64)	AudioSeal under DAC

No score sign/order inversion occurred in the final protocol.

E2 — predictor validation (held-out). Numbers are regenerating under the corrected protocol (PESQ-matched budget; valid two-way cluster inference; incremental-value ablation of s_W over the static magnitude-fraction geometry). No headline ρ/CI/p is stated here until the frozen three-seed run produces it — the prior random-budget numbers have been removed. What the experiment asks: does a channel-relative geometric quantity add predictive value, over static magnitude geometry, for held-out mel-probe detector-score survival, within each channel?

E3 — multi-bit proof of concept (blind decoder, 8 bits). Per-clip decoded bits retain above-chance dependence after several resynthesizers; block error rate is high (payload reliability is poor). Exact BER/BLER with clip-cluster CIs come from the frozen run.

(Every number is injected into the paper by scripts/make_paper_macros.py from the frozen result JSONs — the manuscript cannot drift from the code, and a formal build fails if any macro is missing.)

Reproducing

# CPU: surrogate theory + numerical sanity checks
uv venv && uv pip install -e ".[dev]" && uv run pytest

# GPU (see requirements-audio.lock for the locked environment + model sources;
# install torch/torchaudio cu128 first; descript-audio-codec needs --no-deps)
python experiments/run_audio_all.py --root /path/to/LibriSpeech_parent --device cuda

requirements-audio.lock records every package version and model checkpoint source (HF repo + revision/commit SHA, GitHub release URLs + checksums) used for the paper runs. Manifests, seeds, and keys are fixed; runs are seeded via experiments/audio/repro.py (set_determinism: PYTHONHASHSEED/NumPy/Torch/CUDA + deterministic cuDNN) and per-utterance score arrays ship with the results.

Repository layout

src/rwl/            # surrogate theory (channel, chernoff, masking, R_LB, marks)
tests/              # sanity checks of the implementation vs closed forms
experiments/audio/  # analysis.py (THE unified mel), marks.py, metrics_audio.py,
                    # attackers.py, baselines.py, e1/e2/e3 drivers
scripts/            # make_manifest.py, make_paper_macros.py, remote.py
docs/THEORY.md      # exact statements: Prop 1 / Thm 1 / R_LB / predictor hypothesis
paper/              # ICASSP spconf source; numbers injected from results JSON
data/  results/     # manifests / result JSONs + score arrays

Scope and honesty

• Blind, non-adaptive attacker; key-aware/adaptive removal (Ding et al. 2026) is out of scope.
• The surrogate's closed forms are not claimed for nonlinear generators.
• R_LB concerns a restricted embedder class; no capacity claim, no rate converse.
• Constructed kernel marks realize exact erasure only to first order; measured second-order leakage at operating amplitude is reported.
• E3 demonstrates bits surviving; it does not measure a rate.

License

MIT.