diff --git a/CLAUDE.md b/CLAUDE.md index 67ee4eaf..201ce9b6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -356,7 +356,7 @@ Gates: `hpx_hf_rungs_survive_fade` (fade), `goodput_gate` (the clean-channel cou **The rate ladder's SNR scales are per-waveform-family by physical necessity — do NOT try to unify them.** `rx_snr_db` dispatches to each plugin's `estimate_snr_db`, and they report different quantities: single-carrier PSK (BPSK, post-#934) reads ~true channel SNR; OFDM/SC-FDMA read a *saturation-bounded plugin-domain* SNR that flattens near ~16 dB (ZF noise-enhancement on faded subcarriers) and **physically cannot** report the 20–30 dB the dense rungs run at. So `hpx_hf`'s SL2–SL6 floors are true channel SNR and SL7–SL14 floors are plugin-domain — two scales, one ladder. This looks like a bug; it is not. Forcing OFDM onto a true-SNR scale (or "unifying the estimators") would put the top rungs' floors above anything the estimate can read → the SNR climb never reaches them → the exact v0.14.0 "AWGN-scale floors never clear" stall. The **evidence-based climb** (#934) is what makes two scales safe: it advances on decode success where SNR saturates. Gate: `snr_scale_boundary` — it fails if OFDM starts tracking true SNR without the floors being re-derived in the same change (the two are one decision). Full single-carrier→true-SNR unification of QPSK/8PSK was scoped and DECLINED: high-risk (floor recalibration across 5 profiles) for churn-reduction only, since the evidence climb already self-corrects the mismatch. -**LLRs already carry `1/σ²` — do not weight them by it again.** `openpulse_dsp::constellation::symbol_llrs` divides every distance by `noise_var`, so a calibrated plugin (SC-FDMA, OFDM) emits true log-likelihood ratios whose magnitude is ∝ 1/σ². For repeated observations of the same bits, LLRs **add**: `combine_llrs_map` is the MAP combine and *is* inverse-noise weighting. The engine used to re-weight that sum by a `1 / mean(|LLR|)` proxy — a second 1/σ² — costing 0.75 dB on graded HARQ attempt sets (fixed in PR #686). `combine_llrs_weighted` is only for LLRs with a noise-blind scale (the ±1.0 trait default). Every shipped plugin is now calibrated (PR #687) **in the weak sense only, and the word hides two different bars**: `llr_calibration.rs` checks that `mean(|LLR|)` GROWS with SNR by a per-plugin floor (BPSK ≥ 8×, QPSK ≥ 1.1×, 8PSK ≥ 1.4×, ideal 15.8×) — that a σ² is divided by at all. The strong bar, that a bit carrying `|L|` is wrong about `1/(1+e^{|L|})` of the time within 4×, is checked only by the five `llr_reliability.rs` files (64QAM, MFSK16, OFDM, PILOT, SC-FDMA). **bpsk, qpsk and psk8 meet the weak bar and are untested against the strong one** — measured 2026-09-13, QPSK250 and 8PSK500 are in fact within ~1.25× at the SNRs they are used at, but nothing holds them there. Before citing "calibrated", say which bar. **Choosing the noise estimator is the hard part**: a demodulator's residual is not all thermal noise — pulse-shaping ISI and equalizer misadjustment vary the symbol *amplitude* with no SNR dependence, so a moment (M2/M4) or distance-to-nearest-point estimator stops tracking SNR entirely. Use the component *orthogonal* to the hard decision (`psk_symbol_noise_var`), or for a differential detector the quadrature companion, where the amplitude cancels exactly (`differential_llr_scale`). +**LLRs already carry `1/σ²` — do not weight them by it again.** `openpulse_dsp::constellation::symbol_llrs` divides every distance by `noise_var`, so a calibrated plugin (SC-FDMA, OFDM) emits true log-likelihood ratios whose magnitude is ∝ 1/σ². For repeated observations of the same bits, LLRs **add**: `combine_llrs_map` is the MAP combine and *is* inverse-noise weighting. The engine used to re-weight that sum by a `1 / mean(|LLR|)` proxy — a second 1/σ² — costing 0.75 dB on graded HARQ attempt sets (fixed in PR #686). `combine_llrs_weighted` is only for LLRs with a noise-blind scale (the ±1.0 trait default). Every shipped plugin is now calibrated (PR #687) **in the weak sense only, and the word hides two different bars**: `llr_calibration.rs` checks that `mean(|LLR|)` GROWS with SNR by a per-plugin floor (BPSK ≥ 8×, QPSK ≥ 1.1×, 8PSK ≥ 1.4×, ideal 15.8×) — that a σ² is divided by at all. The strong bar, that a bit carrying `|L|` is wrong about `1/(1+e^{|L|})` of the time within 4×, is checked only by the five `llr_reliability.rs` files (64QAM, MFSK16, OFDM, PILOT, SC-FDMA). **bpsk, qpsk and psk8 meet the weak bar and are untested against the strong one** — measured 2026-09-13, QPSK250 and 8PSK500 are in fact within ~1.25× at the SNRs they are used at, but nothing holds them there. Before citing "calibrated", say which bar. **Choosing the noise estimator is the hard part**: a demodulator's residual is not all thermal noise — pulse-shaping ISI and equalizer misadjustment vary the symbol *amplitude* with no SNR dependence, so a moment (M2/M4) or distance-to-nearest-point estimator stops tracking SNR entirely. Use the component *orthogonal* to the hard decision (`psk_symbol_noise_var`), or for a differential detector the quadrature companion, where the amplitude cancels exactly (`differential_llr_scale`). **Corrected 2026-09-21 (#1364): `1/σ²` is the right target only where the symbol amplitude is KNOWN.** For a blind differential detector it is the high-SNR *limit* of the true slope `2A²/var(dot)`, and an estimator that holds it all the way down keeps voting at full confidence on an attempt carrying no signal — measured, a noise-only attempt emitted LLRs of std 1.41 at every σ from 0.1 to 2.0, and against the true pairwise DBPSK LLR that is 8× over-confident at −12 dB. That matters only where LLRs are SUMMED across attempts, which is `hpx_hf` SL2–SL5. **Acquire on the normalised correlation, not the unnormalised score.** `IqMatchedFilter::search`'s argmax favours high *energy*, on the reasoning that "a deep-fade low-energy window cannot win". When the **preamble** is the faded part that is exactly backwards — SC-FDMA lost frames to data-region windows 4896 samples later that merely shared the pilot comb (ρ = 0.994 at the true offset with energy 19.4, versus ρ = 0.657 with energy 83.0). Use `search_normalized` with an energy floor (ρ is meaningless on a silent window). This looked like "fade dynamics" for a release and was slated as a channel-estimate fix; **ablating** `smooth_ce` entirely (temporarily, to test the hypothesis) left the numbers bit-identical (PR #689) — the sync was the mechanism. `smooth_ce` itself stays in the SC-FDMA demod (an EMA channel-estimate smoother, live at 6 call sites in `plugins/scfdma/src/demodulate.rs`); it just wasn't the thing that was broken. diff --git a/crates/openpulse-core/src/fec.rs b/crates/openpulse-core/src/fec.rs index 7e7a6455..8e5c5ae6 100644 --- a/crates/openpulse-core/src/fec.rs +++ b/crates/openpulse-core/src/fec.rs @@ -607,7 +607,9 @@ impl Default for SoftCombiner { /// /// # When this is the wrong function /// -/// A true log-likelihood ratio `log P(b=0|y) / P(b=1|y)` already carries `1/σ²` — see +/// A true log-likelihood ratio `log P(b=0|y) / P(b=1|y)` already carries its own noise scaling — +/// `1/σ²` where the amplitude is known, and the vanishing slope `2A²/var(dot)` where it is blind +/// and must be estimated (#1364). See /// `openpulse_dsp::constellation::symbol_llrs`, which divides every distance by `noise_var`. For /// independent observations of the same bit, the MAP combine of true LLRs is their plain **sum**: /// use [`combine_llrs_map`]. Passing `noise_var = σ²` here on top of already-calibrated LLRs applies diff --git a/crates/openpulse-dsp/src/constellation.rs b/crates/openpulse-dsp/src/constellation.rs index 17efee11..3c19bda0 100644 --- a/crates/openpulse-dsp/src/constellation.rs +++ b/crates/openpulse-dsp/src/constellation.rs @@ -342,20 +342,58 @@ pub fn snr_db_from_amp_noise(amp: f32, noise_var_per_dim: f32) -> f32 { /// Scale that turns a differential correlation `dot_k = Re(z_k · conj(z_{k−1}))` into a /// log-likelihood ratio, given the quadrature companion `cross_k = Im(z_k · conj(z_{k−1}))`. /// -/// `dot` is antipodal with mean `±A²`; `cross` has mean 0 and variance `≈ 2A²σ²`. So -/// `2·mean|dot| / var(cross) → 1/σ²` and the signal amplitude cancels — which is what makes this -/// immune to the symbol-amplitude variation that defeats a variance-of-`|dot|` estimate. +/// **The target is `2A²/var(dot)`, NOT `1/σ²` — corrected in #1364, and the difference is the whole +/// defect.** `1/σ²` is the *high-SNR limit* of the true DBPSK LLR slope, not the slope. The true +/// slope vanishes with the signal, and an estimator that holds `1/σ²` all the way down keeps voting +/// at full confidence on an attempt that carries no signal at all. Measured on the shipped formula +/// (`2·mean|dot|/var(cross)`): a noise-only attempt emitted LLRs of std **1.41 at every σ from 0.1 +/// to 2.0** — which is exactly `√2`, i.e. exactly the old contract honoured. Against the true +/// pairwise DBPSK LLR the old formula is **8× over-confident at −12 dB** (slope 0.94 versus 0.11). +/// +/// This matters because `combine_llrs_map` SUMS attempts: `hpx_hf` SL2–SL5 run this path, and the +/// OTA arm retains failed bursts, so a worthless attempt does not merely fail to help — it outvotes +/// the attempts that carry the frame. +/// +/// The estimator is the Gaussian-approximation LLR `2μ_x/s²` with an unbiased fourth-moment `²`: +/// for iid circular noise `E[dot²] − E[cross²] = A⁴` **exactly**, because the `A²v` and `v²/2` terms +/// are common to both. So `² = √(max(0, ⟨dot²⟩ − ⟨cross²⟩))` and the scale is `2²/⟨cross²⟩`, +/// which tends to `1/σ²` at high SNR and to **0** as the signal vanishes. +/// +/// **Premise, pinned by `differential_llr_scale_assumes_iid_noise` rather than trusted:** the noise +/// must be uncorrelated at lag 1. `cancel_crossfade_isi` induces ρ = −1/3 (#1361), under which the +/// identity returns `2ρ²v² = 0.889v²` at A = 0 — a constant floor that would restore the defect. The +/// soft path deliberately does not cancel; if #1361 ever changes that, derive the β-corrected +/// identity instead of reusing this one. +/// +/// No blind estimator does better than `N^-1/4` at A = 0: the score for `A²` equals the score for +/// `v` there, so the Fisher information is singular — the known blind-SNR degeneracy. The residual +/// is sampling noise, not bias, and `SCALE_FLOOR` keeps a zero estimate from emitting `−0.0` LLRs, +/// which `l < 0.0` consumers would read as bit 0. /// /// Multiply the `dot` values by the result. Returns 0 for an empty input. pub fn differential_llr_scale(dots: &[f32], crosses: &[f32]) -> f32 { if dots.is_empty() || crosses.is_empty() { return 0.0; } - let mu = dots.iter().map(|v| v.abs()).sum::() / dots.len() as f32; - let var_cross = crosses.iter().map(|v| v * v).sum::() / crosses.len() as f32; - 2.0 * mu / var_cross.max(1e-12) + // f64 accumulators: these are fourth moments, and the subtraction below is catastrophic + // cancellation by construction — at A = 0 the two means are equal in expectation. + let m_dot2 = dots.iter().map(|v| (*v as f64) * (*v as f64)).sum::() / dots.len() as f64; + let m_cross2 = crosses + .iter() + .map(|v| (*v as f64) * (*v as f64)) + .sum::() + / crosses.len() as f64; + let a2 = (m_dot2 - m_cross2).max(0.0).sqrt(); + ((2.0 * a2 / m_cross2.max(1e-12)) as f32).max(SCALE_FLOOR) } +/// Smallest scale `differential_llr_scale` will return, so a zero estimate cannot emit `−0.0`. +/// +/// `combine_llrs_map` treats 0.0 as "no information" and is indifferent, but a consumer testing +/// `l < 0.0` reads `−0.0` as bit 0 — the convention split `soft_demod_conformance` exists to keep +/// closed. Small enough that an attempt scaled by it contributes nothing to a sum. +const SCALE_FLOOR: f32 = 1e-6; + /// Additive SNR (dB) of a symbol block, with the *multiplicative* channel removed first. /// /// This is the symbol-domain twin of `openpulse_channel::estimate_additive_snr_db`, and it exists for @@ -552,44 +590,166 @@ pub fn normalize_stream_rms(syms: &mut [(f32, f32)]) { #[cfg(test)] mod tests { - /// `differential_llr_scale` must recover 1/σ² independently of the signal amplitude — that - /// cancellation is the whole point (a `var(|dot|)` estimate is defeated by amplitude variation). + /// Deterministic complex Gaussian pair, so these tests need no `rand` dependency. + fn zz_gauss(state: &mut u64) -> (f32, f32) { + let mut u = || { + *state = state.wrapping_mul(6364136223846793005).wrapping_add(1); + (((*state >> 11) as f64) / ((1u64 << 53) as f64)).clamp(1e-12, 1.0) + }; + let (u1, u2) = (u(), u()); + let r = (-2.0 * u1.ln()).sqrt(); + ( + (r * (std::f64::consts::TAU * u2).cos()) as f32, + (r * (std::f64::consts::TAU * u2).sin()) as f32, + ) + } + + /// `dot`/`cross` from ACTUAL complex symbols `z_k = A·a_k + n_k`, not from the asymptotic model. + /// + /// This is the whole reason #1364 went unnoticed: the previous fixture synthesised `dots` and + /// `crosses` directly as `±A² + √(2A²σ²)·g` and `√(2A²σ²)·g`, which **omits the `n·conj(n)` + /// term** — the term that produces the low-SNR floor. A fixture built from the asymptotic limit + /// cannot exhibit a defect that lives below that limit, and it swept only 10 and 20 dB. + /// `v` is total complex noise power `E|n|²`. + fn zz_dots_crosses(amp: f32, v: f32, n: usize, seed: u64) -> (Vec, Vec) { + let mut st = seed | 1; + let sigma_c = (v / 2.0).sqrt(); // per component + let mut bit = 1.0f32; + let z: Vec<(f32, f32)> = (0..n) + .map(|i| { + if i % 3 == 0 { + bit = -bit; // a differentially-encoded antipodal stream, not a constant one + } + let (gr, gi) = zz_gauss(&mut st); + (amp * bit + sigma_c * gr, sigma_c * gi) + }) + .collect(); + let (mut d, mut c) = (Vec::new(), Vec::new()); + for k in 1..z.len() { + let ((xr, xi), (yr, yi)) = (z[k], z[k - 1]); + d.push(xr * yr + xi * yi); + c.push(xi * yr - xr * yi); + } + (d, c) + } + + /// The formula shipped before #1364, kept as the CONTROL: it must fail the properties below. + /// A test whose old implementation also passes is not measuring the change. + fn zz_old_scale(dots: &[f32], crosses: &[f32]) -> f32 { + let mu = dots.iter().map(|v| v.abs()).sum::() / dots.len() as f32; + let vc = crosses.iter().map(|v| v * v).sum::() / crosses.len() as f32; + 2.0 * mu / vc.max(1e-12) + } + + /// The scale must track the Gaussian-approximation LLR slope `2A²/var(dot)` across the whole + /// range — **including A = 0, where it must vanish** (#1364). /// - /// Only above the ~3 dB where `mean|dot| ≈ A²` holds; below it the estimator is decision-directed - /// and saturates, which is the safe direction (under-confident LLRs). + /// `1/σ²` is the high-SNR LIMIT of the true DBPSK slope, not the slope; holding it all the way + /// down is what let a signal-free attempt vote at full strength into `combine_llrs_map`. #[test] - fn differential_llr_scale_recovers_inverse_noise_var_at_any_amplitude() { + fn differential_llr_scale_tracks_the_true_slope_not_the_high_snr_limit() { + let n = 20_000; + // Swept only where the target is RESOLVABLE TO THIS TOLERANCE, and the bound is derived + // rather than chosen. The estimate rests on `⟨dot²⟩ − ⟨cross²⟩ = A⁴`, whose sampling spread + // is `O(v²/√N)`. Two different limits follow, and conflating them cost two iterations here: + // * RESOLVABLE at all: `A⁴ ≳ v²/√N`, i.e. `Es/N0 ≳ N^(−1/4)` = −10.8 dB at N = 20 000. + // Below this no blind estimator can separate `A²` from `v` — the Fisher information is + // singular at A = 0 — so a tracking assertion there would be asserting against noise. + // * Tracking to 20 %: needs `spread ≲ 0.44·A⁴`, about 2× more margin in `Es/N0`, i.e. + // ≳ −6 dB at this N. Measured: −12 dB reads 2.01× target, −9 dB reads 1.36×, both the + // sampling floor rather than a defect. + // The A = 0 end is covered by `a_signal_free_attempt_votes_at_essentially_nothing`, which + // asserts the floor itself instead of a ratio to it. for amp in [0.2f32, 1.0, 5.0] { - for snr_db in [10.0f32, 20.0] { - let sigma2 = amp * amp / 10f32.powf(snr_db / 10.0); - // dot ~ ±A² with var 2A²σ²; cross ~ 0 with the same variance. - let n = 20_000; - let mut state = 0x1234u64; - let mut g = || { - state = state.wrapping_mul(6364136223846793005).wrapping_add(1); - let u1 = (((state >> 11) as f64) / ((1u64 << 53) as f64)).clamp(1e-12, 1.0); - state = state.wrapping_mul(6364136223846793005).wrapping_add(1); - let u2 = ((state >> 11) as f64) / ((1u64 << 53) as f64); - ((-2.0 * u1.ln()).sqrt() * (std::f64::consts::TAU * u2).cos()) as f32 - }; - let sd = (2.0 * amp * amp * sigma2).sqrt(); - let dots: Vec = (0..n) - .map(|i| { - let b = if i % 3 == 0 { -1.0 } else { 1.0 }; - b * amp * amp + sd * g() - }) - .collect(); - let crosses: Vec = (0..n).map(|_| sd * g()).collect(); - let k = differential_llr_scale(&dots, &crosses); - let expected = 1.0 / sigma2; + for es_n0_db in [-6.0f32, 0.0, 6.0, 20.0] { + let v = amp * amp / 10f32.powf(es_n0_db / 10.0); + let (d, c) = zz_dots_crosses(amp, v, n, 0x5EED); + let got = differential_llr_scale(&d, &c); + // Gaussian-approximation LLR slope: 2·mean(dot) / var(dot), + // with var(dot) = A²v + v²/2 for iid circular noise. + let want = 2.0 * amp * amp / (amp * amp * v + v * v / 2.0); assert!( - (k / expected - 1.0).abs() < 0.15, - "amp={amp} snr={snr_db} dB (σ²={sigma2:.5}): scale {k:.2} vs expected {expected:.2}" + (got / want - 1.0).abs() < 0.20, + "amp={amp} Es/N0={es_n0_db} dB: scale {got:.4} vs derived target {want:.4}" ); } } } + /// A signal-free attempt must not vote. The bound is DERIVED, not fitted: with no signal the + /// only estimate left is sampling noise in `⟨dot²⟩ − ⟨cross²⟩`, whose conditional mean gives + /// `std(LLR) ≈ 2.76·N^(−1/4)`; the assertion allows 2×. The old formula returns √2 ≈ 1.41 here + /// at EVERY noise level, which is the defect — and the control below pins that it did. + #[test] + fn a_signal_free_attempt_votes_at_essentially_nothing() { + let n = 20_000; + let bound = 2.0 * 2.76 / (n as f32).powf(0.25); + for v in [0.02f32, 0.125, 0.5, 2.0, 8.0] { + let (d, c) = zz_dots_crosses(0.0, v, n, 0x5EED); + let sd = |k: f32| { + let l: Vec = d.iter().map(|x| x * k).collect(); + let m = l.iter().sum::() / l.len() as f32; + (l.iter().map(|x| (x - m) * (x - m)).sum::() / l.len() as f32).sqrt() + }; + let now = sd(differential_llr_scale(&d, &c)); + let before = sd(zz_old_scale(&d, &c)); + assert!( + now <= bound, + "v={v}: signal-free vote {now:.3} exceeds derived bound {bound:.3}" + ); + // The control: the pre-#1364 formula votes at √2 regardless of noise power, which is + // the old contract honoured exactly. If this stops holding, the control has rotted and + // the assertion above is no longer measuring the change. + assert!( + (before - std::f32::consts::SQRT_2).abs() < 0.25, + "v={v}: control expected the old formula to vote ~1.41, got {before:.3}" + ); + } + } + + /// PREMISE PIN (#1364 × #1361). The identity `E[dot²] − E[cross²] = A⁴` holds only for noise + /// uncorrelated at lag 1. `cancel_crossfade_isi`'s backward substitution induces ρ = −1/3, under + /// which a signal-free attempt yields `2ρ²v²` instead of 0 — restoring the defect silently. + /// + /// The soft path deliberately does not cancel, so the premise holds today. This test exists so + /// that if #1361 ever changes that, the coupling is a failing test rather than a silent + /// regression in HARQ weighting. + #[test] + fn differential_llr_scale_assumes_iid_noise() { + let n = 20_000; + let v = 0.5f32; + let (d_iid, c_iid) = zz_dots_crosses(0.0, v, n, 0x5EED); + let iid_vote = differential_llr_scale(&d_iid, &c_iid) * d_iid[0].abs().max(1e-9); + + // Same stream with the cancellation's lag-1 correlation imposed on the NOISE. + let mut st = 0x5EEDu64 | 1; + let sc = (v / 2.0).sqrt(); + let raw: Vec<(f32, f32)> = (0..n + 1).map(|_| zz_gauss(&mut st)).collect(); + let beta = 1.0f32 / 3.0; + let z: Vec<(f32, f32)> = (1..raw.len()) + .map(|k| { + ( + sc * (raw[k].0 - beta * raw[k - 1].0), + sc * (raw[k].1 - beta * raw[k - 1].1), + ) + }) + .collect(); + let (mut d, mut c) = (Vec::new(), Vec::new()); + for k in 1..z.len() { + let ((xr, xi), (yr, yi)) = (z[k], z[k - 1]); + d.push(xr * yr + xi * yi); + c.push(xi * yr - xr * yi); + } + let m_d2 = d.iter().map(|x| (*x as f64) * (*x as f64)).sum::() / d.len() as f64; + let m_c2 = c.iter().map(|x| (*x as f64) * (*x as f64)).sum::() / c.len() as f64; + assert!( + m_d2 - m_c2 > 0.2 * (v as f64) * (v as f64), + "lag-1 correlated noise should break the identity (got {:.4}, iid vote {iid_vote:.6}); \ + if it no longer does, re-derive before letting the soft path cancel (#1361)", + m_d2 - m_c2 + ); + } + /// The orthogonal residual must track σ² even when the symbol *amplitude* wanders — the failure /// mode that makes a moment or distance-to-nearest-point estimator stop responding to SNR. #[test] diff --git a/crates/openpulse-modem/tests/llr_calibration.rs b/crates/openpulse-modem/tests/llr_calibration.rs index 8bf44d1b..62d58120 100644 --- a/crates/openpulse-modem/tests/llr_calibration.rs +++ b/crates/openpulse-modem/tests/llr_calibration.rs @@ -1,4 +1,6 @@ -//! Every soft demodulator must emit *calibrated* LLRs: magnitude ∝ 1/σ². +//! Every soft demodulator must emit *calibrated* LLRs: magnitude ∝ 1/σ² **where the symbol +//! amplitude is known**. For a blind differential detector that limit is not the target — the true +//! slope is `2A²/var(dot)` and vanishes with the signal, which is the #1364 correction. //! //! `openpulse_dsp::constellation::symbol_llrs` divides distances by `noise_var`, so a true //! log-likelihood ratio grows as the noise falls. Nothing that decodes a single frame notices — soft diff --git a/docs/dev/project/traceability.md b/docs/dev/project/traceability.md index dd4ad9ff..f35d873f 100644 --- a/docs/dev/project/traceability.md +++ b/docs/dev/project/traceability.md @@ -15,6 +15,69 @@ and the actually-observed results per change. --- +## 2026-09-21 — a signal-free HARQ attempt voted at full strength; #1364 + +**Change.** `differential_llr_scale` returned `2·mean|dot|/var(cross)`. On an attempt carrying **no +signal** that emits LLRs of std **1.41 at every σ from 0.1 to 2.0** — measured through the shipped +function. Because `combine_llrs_map` SUMS attempts and the OTA arm retains failed bursts, a worthless +attempt did not merely fail to help: it outvoted the attempts carrying the frame. `hpx_hf` SL2–SL5 +run this path. + +**The diagnosis I brought to review was wrong twice, and the review corrected both** +(`docs/dev/reviews/review-1364-differential-llr-scale.md`). First, at A = 0 the old estimator returns +**exactly the `1/σ²` its contract promises** — √2 is that contract honoured. The defect is the +**contract**: `1/σ²` is the high-SNR *limit* of the true DBPSK LLR slope, which vanishes with the +signal; against the exact pairwise LLR the old formula is 8× over-confident at −12 dB. Second, my +"target 0.04" was a category error — `σ₁²/σ₂²` is a *scale* ratio while the gate's 0.21 is a +*magnitude* ratio, and under exact `1/σ²` the magnitude ratio really is ≈0.20. + +**Implementation.** `² = √(max(0, ⟨dot²⟩ − ⟨cross²⟩))`, scale `2²/⟨cross²⟩` — the +Gaussian-approximation LLR with an unbiased fourth-moment amplitude, tending to `1/σ²` at high SNR +and to 0 as the signal vanishes. f64 accumulators (the subtraction is catastrophic cancellation by +construction), and a `SCALE_FLOOR` so a zero estimate cannot emit `−0.0`, which an `l < 0.0` consumer +reads as bit 0. + +**Premise pinned, not assumed.** The identity holds only for noise uncorrelated at lag 1. +`cancel_crossfade_isi` induces ρ = −1/3, under which A = 0 yields `2ρ²v²` and the defect returns — +so `differential_llr_scale_assumes_iid_noise` makes #1361's open question a failing test rather than +a silent regression. + +**Why nothing caught it.** The old unit test claimed this exact property and passed, because it +**synthesised `dots`/`crosses` from the high-SNR asymptotic model**, omitting the `n·conj(n)` term +that causes the floor, and swept only 10/20 dB — and its `expected = 1/σ²` was the wrong target +anyway. Rebuilt from actual complex symbols, swept to A = 0, with the old formula retained as a +control asserting it votes ≈ √2. + +**A limit that is fundamental, not a shortfall.** At A = 0 the score for `A²` equals the score for +`v`, so the Fisher information is singular and any blind estimate is sampling-noise limited at +`N^(−1/4)`. The tracking sweep is therefore asserted only where `²` is resolvable **to the stated +tolerance** — derived as ≳ −6 dB at N = 20 000, after −12 dB read 2.01× and −9 dB read 1.36×. + +**Tests → results (actually run).** + +- Sabotage, both directions: the rebuilt tests fail on the old estimator + (`v=0.02: signal-free vote 1.439 exceeds derived bound 0.464`) and pass on the new one. +- Seed-triple sweep, 16 triples, both arms measured here rather than cited: + **old 7/16 failing, mean delta 0.656 dB → new 1/16, mean delta 0.156 dB.** (#1364 recorded 3/16 on + its own smaller triple set; this is a different set, not a reproduction of that number.) +- `a_deeply_faded_extra_attempt_does_not_hurt` still passes — #832's gate is improved, not changed. +- **No fade regression**, which was the review's live risk (Jensen: `√E[A⁴] ≥ E[A²]`, ×1.41 on + Rayleigh). `moderate_f1`, 32 seeds, 1 clean + 1 faded: 19/23/27 new against 20/22/27 old at + 6/8/10 dB; the 2-clean rows are identical. **No fade benefit either** — the gain is AWGN-specific. +- Full gate: see the `GATE:` line on PR #1426. + +**The wrong contract was written in five places** and all were swept: `constellation.rs`'s doc, the +two `bpsk_demodulate_soft` comments, `llr_calibration.rs`'s header, `fec.rs`'s `combine_llrs_map` +doc, and CLAUDE.md's "LLRs already carry 1/σ²" edge — which now records that this is the right target +only where the symbol amplitude is KNOWN. + +**Filed, not folded in: #1425.** The property is a class — every blind scale-invariant calibrator +votes at a σ-independent magnitude on noise, and 8PSK500 does so at **0.44** of a good attempt +against BPSK250's 0.009. The fix here is specific to a differential detector's dot/cross pair and +does not generalise to the coherent plugins. + +--- + ## 2026-09-20 — #1234 CLOSED by deciding not to wire the keystore; nine baseline pairs are permanent **Decision (maintainer, 2026-09-20).** #1234's two halves are both resolved, and the wiring half is diff --git a/docs/dev/reviews/review-1364-differential-llr-scale.md b/docs/dev/reviews/review-1364-differential-llr-scale.md new file mode 100644 index 00000000..0ab3c836 --- /dev/null +++ b/docs/dev/reviews/review-1364-differential-llr-scale.md @@ -0,0 +1,84 @@ +--- +project: openpulsehf +doc: docs/dev/reviews/review-1364-differential-llr-scale.md +status: resolved +last_updated: 2026-09-21 +--- + +# Design review — #1364, `differential_llr_scale`'s low-SNR floor + +## Prompt + +Fable was asked to **falsify** a diagnosis and a candidate fix before implementation, not confirm +them. Seven numbered attack points went out with the apparatus: the measured floor (a noise-only +attempt emitting LLRs of std 1.459 at every σ from 0.1 to 2.0, taken through the shipped function), +my candidate `2²/⟨cross²⟩` with `² = √(⟨dot²⟩ − ⟨cross²⟩)`, and its measured shortfall. The hardest +question was put first and flagged as unverified: *"is the identity `E[dot²] − E[cross²] = A⁴` correct +for this model, given dot and cross are formed from adjacent symbols and share `z_{k−1}`? If it is +wrong the candidate is unsound regardless of the numbers."* + +## Verdict + +**Build with changes.** The candidate is sound; my *framing* of the defect was wrong in two ways that +would have gone into the commit message and the docs. + +1. **The headline was backwards.** At A = 0 the shipped estimator returns **exactly the `1/σ²` its + contract promises** — the measured std of √2 ≈ 1.414 is that contract honoured, not violated. The + defect is the **contract**: `1/σ²` is the high-SNR *limit* of the true DBPSK LLR slope, and the + true slope vanishes with the signal. Regressed against the exact pairwise LLR, the old formula is + **8× over-confident at −12 dB** (slope 0.94 against 0.11). My candidate turns out to be the + Gaussian-approximation LLR `2μ_x/s²` with an unbiased fourth-moment `²` — a principled + derivation rather than the patch I presented it as. +2. **My 0.04 target was a category error.** `0.0398 = σ₁²/σ₂²` is a **scale** ratio; #1364's 0.21 and + the gate are **magnitude** ratios, and under exact `1/σ²` with the true σ the magnitude ratio in + that regime is **0.20**. So the shipped 0.21 was its contract, not a miss. The candidate reaches + the correct Gaussian target: 0.040 measured against shipped 0.149. +3. **Q1, the identity:** correct for iid circular noise (agreement < 1 % at five (A,σ) points); the + shared `z_{k−1}` widens the sample spread, not the expectation. **But it breaks under lag-1 noise + correlation**, which `cancel_crossfade_isi` induces at ρ = −1/3 — returning `2ρ²v² = 0.889v²` at + A = 0 and restoring the defect. That couples this fix to #1361's open question, and is now pinned + by `differential_llr_scale_assumes_iid_noise` rather than left implicit. +4. **Q2, the residual floor is sampling noise:** confirmed, `∝ N^(−1/4)`, with `P(scale = 0) ≈ 0.5`. + My "still flat across σ" observation was a tautology — any scale-invariant estimator is + σ-independent at A = 0, and a fixed seed makes the five values identical by construction. +5. **No blind estimator does better.** At A = 0 the score for `A²` equals the score for `v`, so the + Fisher information is singular — the known blind-SNR degeneracy. Side information buys a constant, + not a rate. So the floor is fundamental, not a shortcoming of the candidate. +6. **Q5, why it was never caught:** confirmed, plus a reason I had not seen. The old fixture + synthesised `dots`/`crosses` from the asymptotic model (omitting `n·conj(n)`) and swept only + 10/20 dB — but its `expected = 1/sigma2` was *also* the wrong target, so even a correct fixture + would have measured against the wrong reference. The "safe direction (under-confident)" doc claim + is backwards relative to the true LLR. +7. **Q6, regression risk:** none on AWGN (good attempts' scales move by a common factor 1.09, so + thresholds do not shift). Two caveats raised: a Jensen effect (`√E[A⁴] ≥ E[A²]`, ×1.41 on a + Rayleigh envelope) meaning a within-frame-faded attempt votes louder, **measured and not + observed** — `moderate_f1` at 6/8/10 dB is 19/23/27 new against 20/22/27 old; and `−0.0` LLRs from + a zero scale, closed with `SCALE_FLOOR`. +8. **Q7, blast radius wider than I stated**, and the property is a **class**: every blind + scale-invariant calibrator votes at a σ-independent magnitude on noise, with 8PSK500 at 0.44 of a + good attempt against BPSK250's 0.009. Filed as #1425. + +## Consumer + +`plugins/bpsk/src/demodulate.rs:427`, the BPSK soft path — live in production, since the OTA arm +admits `FecMode::Rs` (`engine.rs:3040`) and `hpx_hf` SL2–SL5 are BPSK. Also +`apps/openpulse-testbench/src/signal_path.rs:91` (display only) and the BPSK `-RRC` modes through the +same call, whose LMS-equalised symbols carry coloured noise — no profile uses BPSK-RRC, but that is +the ρ-floor case from point 3 and is why the premise pin is a test rather than a comment. + +## Prior art + +`grep -rn "differential_llr_scale"` → one production caller, one unit test, one testbench display. +The existing test `differential_llr_scale_recovers_inverse_noise_var_at_any_amplitude` claimed exactly +this property and passed; reading it is what found the synthesised fixture. #687 is the original +calibration work whose contract this corrects; #832 is the decision whose gate +(`a_deeply_faded_extra_attempt_does_not_hurt`) this improves without changing. + +## Twins + +`psk_symbol_noise_var` is the coherent sibling and is **not** changed here: it estimates noise from +the component orthogonal to the hard decision, so its amplitude reference is a different quantity and +needs its own derivation — that is #1425, filed rather than folded in. Within this change the two +directions are twins and both are sabotage-verified: the rebuilt tests fail on the old estimator +(`signal-free vote 1.439 exceeds derived bound 0.464`) and pass on the new one, with the old formula +retained inside the test as a control asserting it votes ≈ √2. diff --git a/plugins/bpsk/src/demodulate.rs b/plugins/bpsk/src/demodulate.rs index 02e5e6a9..e66cccdd 100644 --- a/plugins/bpsk/src/demodulate.rs +++ b/plugins/bpsk/src/demodulate.rs @@ -378,7 +378,7 @@ pub fn bpsk_demodulate_soft( let offset = find_timing_offset(samples, n, fc, fs); // NOTE: crossfade-ISI cancellation is deliberately NOT applied on the soft path. BPSK is // *differential*, so the backward-substitution recursion inflates the noise LLRs of a deeply - // faded attempt instead of suppressing them — that breaks the 1/σ² LLR calibration HARQ MAP + // faded attempt instead of suppressing them — that breaks the LLR calibration HARQ MAP // combining relies on (regressed `llr_calibration::a_deeply_faded_extra_attempt_does_not_hurt`). // The cancellation stays on the hard differential path (`bpsk_demodulate`), where it restores the // decision margin without disturbing any soft-combining scale. @@ -420,7 +420,9 @@ pub fn bpsk_demodulate_soft( }) .collect(); - // Calibrate the soft values into *true* log-likelihood ratios (magnitude ∝ 1/σ²). Nothing that + // Calibrate the soft values into *true* log-likelihood ratios. The target is the DBPSK LLR + // slope `2A²/var(dot)`, NOT `1/σ²` — that is only its high-SNR limit, and holding it all the + // way down is what let a signal-free attempt vote at full strength (#1364). Nothing that // decodes a single frame notices — soft Viterbi, min-sum LDPC and max-log turbo are all // scale-invariant — but HARQ soft combining across receive attempts does: uncalibrated, an attempt // from a deep fade votes as loudly as a clean one. See `openpulse_core::fec::combine_llrs_map`.