fix: EP projection feeds linear sample weights as log weights (boundary attractor, sigma-collapse)

[Jammy2211]

Overview

Every sampler-driven EP factor update projects its posterior through Result.projected_model, which passes linear importance weights (samples.weight_list) into AbstractMessage.project — whose contract (and math, w = exp(log_w − max)) requires log weights. Linear weights through exp are near-uniform over the whole nested-sampling run, so the moment matching in the message's canonical (logit) space is dominated by boundary-adjacent prior-phase samples: the projected message lands at a prior bound with spuriously tiny std, poisons the EP cavity, and any hierarchical graph then sigma-collapses (F10) — the diagnostics correctly flag a collapse whose root cause is upstream of them.

Found by the slope_hierarchy science project deliberately exercising the 2026-07 EP wave on a realistic lensing case (Jammy2211/slope_hierarchy#1 has the full forensics).

Smoking-gun repro (slope_hierarchy, RAL job 330591 — 1 imaging lens, 1 EP step, Nautilus factor optimiser):

• factor's internal Nautilus fit: slope = 2.0448 (2.0150, 2.0748) ✅ (matches the standalone fit of the same dataset)
• projected mean-field message, same variable: 2.9875 ± 0.011 ❌ (UniformPrior(1.5, 3.0) upper bound; step flagged BAD_PROJECTION; per-factor log_evidence 19–148 vs standalone logZ ≈ 6600, exploding to 3×10¹² over steps)
• damping does not help (δ=0.5 collapsed identically — RAL 330532): the poison enters at step 0.

Plan

• Fix the weight-semantics mismatch at the projected_model seam (convert to log weights, or change AbstractMessage.project to take linear weights — one convention, unambiguous kwarg name).
• Add a regression test that projects a known weighted posterior sample set through a UniformPrior message and asserts the projected moments match the sample moments (fails loudly today).
• Audit every other .project(...) / log_weight_list call site for the same confusion.
• Archaeology: establish when weight_list semantics and project diverged (older EP runs, e.g. the concr H0 project, may have carried the same bias silently).
• End-to-end validation: rerun the slope_hierarchy single-lens EP probe — projected slope must match the factor fit's slope.

Detailed implementation plan

Affected Repositories

• PyAutoFit (primary)

Branch Survey

Repository	Current Branch	Dirty?
./PyAutoFit	main	clean

Suggested branch: feature/ep-projection-weights

Implementation Steps

1. autofit/non_linear/result.py::projected_model (~line 341) — pass log weights: np.log(...) with a floor for zero weights (e.g. clip at the smallest positive float, or mask zero-weight samples before projecting). Alternatively (preferred if reviewers agree): rename AbstractMessage.project's kwarg to make the expected scale explicit and accept linear weights, logging internally — pick ONE convention and make the other impossible.
2. autofit/mapper/prior/abstract.py::Prior.project (~line 201) — align the docstring and the forwarded kwarg with the chosen convention.
3. autofit/messages/abstract.py::AbstractMessage.project (~line 266) — no math change needed if callers are fixed; assert/validate weight scale if cheap (e.g. reject all-positive "log" weights whose exp underflows nowhere — heuristic, discuss in PR).
4. Grep audit: every caller passing weights into project / log_weight_list (graphical laplace path, tests, docs snippets).
5. Regression test (numpy-only, no JAX per unit-test convention): weighted samples from a known Gaussian truncated inside UniformPrior(1.5, 3.0) with realistic nested-sampling-like weights (many near-zero prior-phase samples + concentrated posterior mass); assert projected mean/std ≈ weighted sample moments; assert the old behaviour (linear-as-log) fails it.
6. EP integration check: the existing graphical integration tests + a small factor-fit projection test if one doesn't exist.

Key Files

• autofit/non_linear/result.py — the buggy seam (projected_model)
• autofit/mapper/prior/abstract.py — Prior.project docstring/forwarding
• autofit/messages/abstract.py — AbstractMessage.project contract
• test_autofit/messages/ + test_autofit/graphical/ — regression coverage

Validation

• pytest test_autofit (numpy-only unit suite).
• External end-to-end: slope_hierarchy 1-lens EP probe on RAL (projected slope ≈ 2.04, not 2.99); then the project's blocked goal-2 parity run (EP vs NUTS parent posterior).

Notes

• The Bug Agent sized this too-large on keywords; the fix itself is one focused PR (seam fix + tests + audit). If the call-site audit uncovers wider semantic drift, split then.
• Related but separate: draft/feature/autofit/ep_optimise_expose_updater_delta.md (EP damping API gap, filed from the same investigation).

Original Prompt

Click to expand starting prompt

PyAutoMind prompt: bug/autofit/ep_projection_linear_weights_as_log.md — see that file for the full write-up (symptom, cause seam, fix sketch, context).

🤖 Generated with Claude Code

[Jammy2211]

Fix is up: PR #1383 — #1383

Investigation upgrade while implementing: the linear-vs-log weight confusion is real but turned out to be the secondary defect. The primary one is that TransformedMessage.project never transformed its samples into the base message's space — even equal-weighted samples clustered at 2.05 inside UniformPrior(1.5, 3.0) projected to 2.97 (reproduced in a 0.2s unit test, no sampler needed). Both are fixed in the PR; the new regression test fails on either defect alone. Full test_autofit: 1494 passed / 1 skipped.

Post-merge: slope_hierarchy reruns the 5-lens EP fit for the EP-vs-NUTS parity table (Jammy2211/slope_hierarchy#1).