bug: AnalysisPoint eager log_likelihood drifted significantly since 2026-04-24 (FitPositionsImagePairAll + PointSolver chain)

[Jammy2211]

Summary

Eager (xp=np, no JIT) log-likelihoods from AnalysisPoint for a fixed simulator-truth model on a fixed seeded PointDataset have drifted significantly between 2026-04-24 and 2026-05-16, in a way that suggests a real behaviour change in the PointSolver → FitPositionsImagePairAll → χ² chain rather than floating-point noise.

This surfaced during the autolens_profiling Phase 1 follow-up triage (PyAutoLabs/autolens_workspace_developer#67). The triage repo runs both eager and JIT log-likelihoods for these scripts and asserts them against a hardcoded EXPECTED_LOG_LIKELIHOOD_* constant set 22 days ago against a stable seeded simulator.

Observed drift

Tested on PyAutoLens 2026.5.14.2 (commit 694285c3f) against clean origin/main of autolens_workspace_developer:

Script	Expected (2026-04-24)	Actual (2026-05-16)	Drift
jax_profiling/jit/point_source/image_plane.py	0.0748	-362.21	sign change, ≈ 4843× absolute
jax_profiling/jit/point_source/source_plane.py	-294.14	-3598.97	≈ 12× more negative

The constants were last set in autolens_workspace_developer commit cfa5378 (2026-04-24) — "jax_profiling: evaluate all scripts at simulator truth via GaussianPrior pattern" — which pinned each script's model to the simulator-truth vector, so the expected log-likelihoods are evaluations at the truth.

Reproduction

cd autolens_workspace_developer    # on clean origin/main
source ../activate.sh
python jax_profiling/jit/point_source/image_plane.py
python jax_profiling/jit/point_source/source_plane.py

Both scripts raise AssertionError from the EXPECTED_LOG_LIKELIHOOD_* regression assertion (image_plane.py:444, source_plane.py:492).

Suspect commits (since 2026-04-24)

Light bisect of PyAutoLens / PyAutoArray / PyAutoGalaxy commits doesn't surface an obvious smoking gun. Most-likely candidates worth a closer look:

1. c002f18 (autolens_workspace_developer, 2026-05-06) — "align jax_profiling point-source simulators with realistic noise scales". Changed positions_noise_map from grid.pixel_scale (0.1″ for the simple dataset) to a fixed 0.005 (5 mas HST PSF-centroiding precision) in dataset_setup/point_source.py. This commit only touches the simulator scripts, not the committed on-disk dataset — but if anyone re-ran the dataset_setup between then and now, the on-disk positions_noise_map value changes. Current on-disk dataset/point_source/simple/point_dataset_positions_only.json has positions_noise_map = 0.05 (50 mas), which is neither the old 0.1 nor the new 0.005 — possible third regeneration.
2. d333fb17e (PyAutoLens, 2026-04-28) — "synthetic positions fallback in test mode and PYAUTO_SMALL_DATASETS". Adds a fallback path in PointSolver.solve and Result.positions_likelihood_from. Both are gated on env vars not set in the smoke runs, but worth confirming no behaviour change leaked into the default path.
3. 6fc18e32e (PyAutoLens, 2026-05-14) — "added maximum threshold (added maximum threshold #505)". Touches autolens/analysis/result.py — could affect positions threshold derivation downstream of Result.positions_likelihood_from.
4. a04a50102 (PyAutoLens, 2026-05-14) — "AnalysisPoint kwargs passthrough". Small, but touches AnalysisPoint.__init__ directly.

None of these jumps out as an obvious culprit; the drift magnitude (sign change for image_plane, 12× for source_plane) is too large to come from any of them alone.

What's likely needed

• Confirm whether the on-disk point_dataset_positions_only.json (positions_noise_map = 0.05) was meant to be the source-of-truth or whether it should be regenerated.
• Verify FitPositionsImagePairAll chi² formula hasn't silently changed sign or noise-map handling (e.g. treating noise_map as variance vs sigma).
• Check whether PointSolver is now returning different model positions for the seeded truth model — easy to spot by printing model positions vs the smoke output's eager evaluation.

Context

This issue triages only the eager (numpy) log-likelihood drift. The same scripts also assert JIT and vmap log-likelihoods against the same constants; those drift identically (consistent with xp being plumbed correctly — the drift is in the underlying chain, not the JIT compilation).

Related

• autolens_workspace_developer#67 — parent triage (autolens_profiling F1 follow-up).
• autolens_workspace_developer#68 — jit/imaging/pixelization.py JIT vs eager mapping-matrix shape mismatch (separate bug surfaced by the same smoke).
• autolens_workspace_developer#69 — jit/imaging/delaunay.py log_evidence rebuild returns -inf (separate bug surfaced by the same smoke).
• PyAutoLabs/autolens_profiling#2 — Phase 1 mirror PR (carries the same constants verbatim, so its smoke fails the same way; the mirror is intentionally pinned to the assertion-as-load-bearing pattern while this issue is open).