feat: opt-in gradient-safe log-det via Settings (default unchanged)

[Jammy2211]

Overview

Add an opt-in, non-default gradient-safe log-determinant method to inversions, selected via Settings. The default evidence path is unchanged and byte-identical; the new path exists so gradient-based searches (which the current Cholesky log-det blocks with a NaN — autolens_workspace_developer#104) have a finite alternative we can measure against the Cholesky truth.

This is the only change endorsed by the independent review of the reg-logdet non-finite investigation (PyAutoMind/draft/bug/autoarray/PROBE_reg_logdet_justification.md). That review confirmed the current evidence is scientifically correct — d log det(H) / d log λ = 1798 to machine precision, so the λ-dependence that drives inference is right, and the 1e-8 term is only a constant offset. Changing the default was therefore ruled out (a relative lift is a regression; a pseudo-determinant breaks archived-evidence comparability). Hence: opt-in only, default preserved.

Plan

• Add Settings.log_det_method following the existing Optional[…] = None → config-fallback pattern.
• Branch both evidence log-det terms on it: "cholesky" (current, default) vs "slogdet" (finite where Cholesky NaNs, identical where positive-definite).
• Default config value "cholesky" ⇒ default behaviour byte-identical.
• Validate that the default moves nothing (the release-safety gate), that slogdet ≡ cholesky where PD, and that slogdet is finite where cholesky NaNs.

Detailed implementation plan

Work Classification

Library (PyAutoArray). Single repo. The JAX gradient-finiteness assertion is downstream coverage routed to autogalaxy_workspace_test at ship time (option i), not claimed here.

Worktree root

~/Code/PyAutoLabs-wt/gradient-safe-logdet/

Branch Survey

Repository	Current Branch	Dirty?
./PyAutoArray	main	clean

Suggested branch: feature/gradient-safe-logdet

(autolens_workspace_test is claimed by dpie-lenstool-default; deliberately not used — the downstream JAX assertion goes to autogalaxy_workspace_test instead.)

Implementation Steps

1. autoarray/settings.py — add log_det_method: Optional[str] = None to Settings.__init__, store as self._log_det_method, and add a log_det_method property falling back to conf.instance["general"]["inversion"]["log_det_method"] — identical to the use_positive_only_solver pattern directly above it. Docstring: default "cholesky" = current behaviour; "slogdet" = a finite, differentiable log-abs-det, identical where H is positive-definite, non-NaN where it is not; intended for gradient-based searches and for comparison, not as the default evidence path.
2. autoarray/inversion/inversion/abstract.py — in BOTH log_det_regularization_matrix_term (:734-764) and log_det_curvature_reg_matrix_term (:706-731), branch on self.settings.log_det_method:
   • "cholesky" → the existing 2 * xp.sum(xp.log(xp.diag(xp.linalg.cholesky(M)))), unchanged, including the existing test-mode LinAlgError guard.
   • "slogdet" → xp.linalg.slogdet(M) → return the logabsdet. (Equals log det when the sign is +1, i.e. wherever the matrix is positive-definite; finite otherwise.)
3. autoarray/config/general.yaml — add log_det_method: cholesky under the inversion section (the fallback target that keeps the default byte-identical).
4. test_autoarray/ — numpy-only unit tests: (a) a well-conditioned fixture where slogdet and cholesky log-dets agree to ~1e-10; (b) a fabricated non-PD matrix where cholesky raises but slogdet returns finite; (c) that the default Settings() selects cholesky and reproduces the current value.

Key Files

• autoarray/settings.py — Settings (the Optional→conf pattern to mirror).
• autoarray/inversion/inversion/abstract.py:706-764 — the two log-det terms.
• autoarray/config/general.yaml — the inversion config section.
• autoarray/inversion/inversion/abstract.py:83 — self.settings = settings or Settings() (already threaded).

Validation

• Byte-identical default (release-safety gate): figure_of_merit from current main == this branch with the setting at default/absent, on the MGE and pixelization fiducials.
• slogdet ≡ cholesky where PD: ~1e-10 on a well-conditioned fixture.
• slogdet finite where cholesky NaNs: the Feature/mapping plot #104 seed-0 reject (draw 12/35), reproduced with the 30-second local spectrum script (the 10.9 GiB figure was value_and_grad; the forward matrix is cheap — no A100 needed).
• JAX gradient-finiteness assertion → autogalaxy_workspace_test at ship time (mirror autofit_workspace_test/scripts/jax_assertions/fitness_nan_gradient_contract.py).

Out of scope

• Never change the default. Confirmed byte-identical is the gate.
• Analytic precomputed-eigenvalue log-det (exact, rectangular-Constant-only; cannot serve Adapt's per-pixel weights) — a possible second method the same toggle could select later.
• The two independent bugs this investigation surfaced (constant_zeroth_broken_dead_code.md, PROBE_adapt_double_square_coefficient.md) — filed separately.

Original Prompt

Click to expand starting prompt

See PyAutoMind/draft/feature/autoarray/gradient_safe_logdet_settings_option.md (carries the full human verdict from the adversarial review: C4 protects the current evidence, C10 refuted, default must not move).

[Jammy2211]

Shipped — PR #392 merged.

Settings.log_det_method is live: default "cholesky" (byte-identical to prior behaviour — 926 tests pass, every existing log_evidence test unchanged), opt-in "slogdet" for gradient-based searches and comparison. Both evidence log-det terms route through a shared helper.

Two things worth recording:

• No workspace config mirroring was needed — packaged-only keys layer through the production config (verified against the existing packaged-only nnls_target_kappa).
• slogdet is slightly more accurate than Cholesky on ill-conditioned PD matrices (matches numpy log det exactly; Cholesky ~2.5e-7 off), so it is not bit-identical even in the PD regime — one more reason it is non-default and for comparison.

Completion record: PyAutoMind/complete/2026/07/gradient-safe-logdet.md.

Follow-up (new coverage, not yet done): a JAX gradient-finiteness assertion in autogalaxy_workspace_test. Closing.