Skip to content

feat: opt-in gradient-safe log-det via Settings (default unchanged)#392

Merged
Jammy2211 merged 1 commit into
mainfrom
feature/gradient-safe-logdet
Jul 17, 2026
Merged

feat: opt-in gradient-safe log-det via Settings (default unchanged)#392
Jammy2211 merged 1 commit into
mainfrom
feature/gradient-safe-logdet

Conversation

@Jammy2211

Copy link
Copy Markdown
Collaborator

Closes #391.

Adds an opt-in, non-default gradient-safe log-determinant method to inversions, selected via Settings. The default evidence path is unchanged and byte-identical.

Why

The two Bayesian-evidence log-det terms compute 2*sum(log(diag(cholesky(M)))). On the JAX backend the Cholesky returns NaN (numpy raises) when M is not numerically positive-definite — which stalls gradient-based searches on ill-conditioned regularization matrices (autolens_workspace_developer#104: the pixelized likelihood's NaN wall localised to exactly this term).

Why opt-in and not a default change

An independent adversarial review of the reg-logdet investigation established that the current evidence is scientifically correctd log det(H) / d log λ = 1798 to machine precision, so the coefficient-dependence that drives inference is right, and the 1e-8 conditioning term is only a constant offset. A relative lift would be a regression; a pseudo-determinant would break archived-evidence comparability. So the default is deliberately preserved and this adds an alternative only, for gradient work and for comparison.

Change

Settings.log_det_method, following the existing Optional[…] = None → conf general.yaml pattern (mirrors use_positive_only_solver):

  • "cholesky" (default) — the historical computation, byte-for-byte unchanged including the test-mode LinAlgError guard. Both terms now route through a shared _log_det_symmetric_from helper; the default branch is the identical expression.
  • "slogdet"logabsdet of xp.linalg.slogdet(M): identical where M is positive-definite, finite and differentiable where the Cholesky would NaN. General (serves the adaptive schemes, unlike an analytic precomputed-eigenvalue route). Opt-in via al.SettingsInversion(log_det_method="slogdet").

Applies to both evidence log-det terms (log_det_curvature_reg_matrix_term and log_det_regularization_matrix_term) — a gradient search needs both finite.

API Changes

Additive only. New optional Settings(log_det_method=...) field, default None → config "cholesky" → current behaviour. No removed/renamed/changed symbols; no downstream migration.

Detail
  • Added: Settings.log_det_method (optional kwarg + property); AbstractInversion._log_det_symmetric_from (private helper); inversion.log_det_method: cholesky in packaged general.yaml.
  • Removed / Renamed / Changed Signature: none.
  • Changed Behaviour: none at default — the two log-det properties produce the identical value; only an explicit log_det_method="slogdet" opt-in differs.
  • Migration: n/a. No workspace config mirroring required — packaged-only keys layer through production config (verified against the existing packaged-only nnls_target_kappa).

Testing

  • Full test_autoarray/ suite: 926 passed. Every existing log_evidence test (fit_imaging / fit_interferometer / fit_dataset) unchanged — this is the byte-identical-default gate.
  • New unit tests: default log_det_method == "cholesky" and reproduces numpy log det; slogdet == cholesky on a positive-definite matrix (~1e-8); slogdet finite (== np.linalg.slogdet) on a non-PD matrix where Cholesky fails.
  • Note: on ill-conditioned PD matrices slogdet matches the true log det more closely than Cholesky (~2.5e-7 on a 25-pixel Constant reg matrix, slogdet being the accurate one) — a further reason it is non-default and for comparison rather than a silent swap.

Follow-ups (not in this PR)

  • JAX gradient-finiteness assertion → autogalaxy_workspace_test (mirrors autofit_workspace_test/scripts/jax_assertions/fitness_nan_gradient_contract.py).
  • Two independent bugs this investigation surfaced, filed as separate PyAutoMind prompts: ConstantZeroth is dead code (shape + missing-arg), and Adapt's double-square 4th-power coefficient.

Gate

  • Tests — 926 passed in the task worktree.
  • Smoke — n/a at library level; downstream JAX assertion follows to autogalaxy_workspace_test.
  • HeartRED, acknowledged contemporaneously by the human on 5 pre-existing unrelated reasons; RED reason verbatim: PyAutoLens: 1 uncommitted source change(s) (a stray paper_jax/paper.md edit in the main checkout, untouched by this branch).

…lt unchanged)

The two Bayesian-evidence log-det terms (log_det_curvature_reg_matrix_term and
log_det_regularization_matrix_term) use 2*sum(log(diag(cholesky(M)))), which
returns NaN on the JAX backend (raises on numpy) when M is not numerically
positive-definite -- stalling gradient-based searches on ill-conditioned
regularization matrices (autolens_workspace_developer#104).

Add Settings.log_det_method (Optional -> conf general.yaml fallback, mirroring
the use_positive_only_solver pattern):

- "cholesky" (default) -- the historical computation, byte-for-byte unchanged
  including the test-mode LinAlgError guard. Both terms route through a shared
  _log_det_symmetric_from helper; the default branch is the identical expression.
- "slogdet" -- logabsdet of xp.linalg.slogdet(M): identical where M is
  positive-definite, finite and differentiable where the Cholesky would NaN.
  Opt-in, non-default, general (serves the adaptive schemes too); intended for
  gradient-based work and for comparison against the Cholesky evidence.

Scope decided by an independent adversarial review of the reg-logdet
investigation: the current evidence's coefficient-dependence is correct to
machine precision, so the DEFAULT is deliberately not changed. This adds an
opt-in alternative only.

Full test_autoarray suite 926 passed (every existing log_evidence test
unchanged = byte-identical default). New tests: default==cholesky==numpy log det;
slogdet==cholesky on a PD matrix; slogdet finite where cholesky fails on a
non-PD matrix. No workspace config mirroring needed -- packaged-only keys layer
through production config (verified, like nnls_target_kappa).

Closes #391.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Jammy2211 Jammy2211 added the pending-release PR queued for the next release build label Jul 17, 2026
@Jammy2211
Jammy2211 merged commit a240782 into main Jul 17, 2026
4 checks passed
@Jammy2211
Jammy2211 deleted the feature/gradient-safe-logdet branch July 17, 2026 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pending-release PR queued for the next release build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: opt-in gradient-safe log-det via Settings (default unchanged)

1 participant