Skip to content

fix: test-mode bypass tolerates a single-eval FitException (resample-equivalent)#1408

Merged
Jammy2211 merged 1 commit into
mainfrom
feature/interferometer-delaunay-flaky-fitexception
Jul 21, 2026
Merged

fix: test-mode bypass tolerates a single-eval FitException (resample-equivalent)#1408
Jammy2211 merged 1 commit into
mainfrom
feature/interferometer-delaunay-flaky-fitexception

Conversation

@Jammy2211

Copy link
Copy Markdown
Collaborator

Summary

TEST_MODE=2's _fit_bypass_test_mode (autofit/non_linear/search/abstract_search.py) calls the likelihood once to verify the analysis runs. If that single prior-median instance is pathological — a non-positive-definite inversion, or a degenerate Delaunay mesh that yields NaN vertices — the analysis raises af.exc.FitException. That is exactly the signal a real non-linear search absorbs by resampling. In test mode there is no sampler, so the exception hard-failed the whole run.

This made flaky pixelization scripts intermittently break CI smoke. Concretely, autolens_workspace/scripts/interferometer/features/pixelization/delaunay.py fails ~50% on CI py3.12 with ValueError: Points cannot contain NaN from the Delaunay/qhull triangulation, via source_pix_2_fit_bypass_test_modelog_likelihood_function → inversion → scipy_delaunay (autolens_workspace#307, PyAutoLens#640). The same workspace commit passed on a parallel run — a true nondeterministic (CI-thread/BLAS-dependent) flake.

Change

Wrap the single verification eval: on FitException, log the cause and keep the existing -1.0e99 sentinel (the same effect as a resample-to-reject). Only FitException is caught — genuine code errors (e.g. ValueError, AttributeError) still propagate and fail test mode, so a truly broken likelihood stays loud. Central fix: benefits every analysis (imaging, interferometer, galaxy), not just this one script. Complements PyAutoLens#639 (raise_fit_exception preserves the cause behind the FitException).

Tests

  • TestBypassToleratesFitException::test__test_mode_2__fitexception_tolerated__fit_completes_with_sentinel — fit completes, sentinel log-likelihood retained.
  • ...::test__test_mode_2__non_fitexception_still_propagates — a ValueError still raises.
  • 108 non_linear/{search,result,samples} tests pass; no regressions.

API Changes

None. Behaviour change is confined to the TEST_MODE=2 bypass path; public API, signatures, and normal-search behaviour are unchanged. No downstream workspace migration required.

Follow-ups

  • autolens_workspace: un-park the interferometer Delaunay no_run.yaml entry + add to smoke (library-first: merge this PR first).
  • Phase 2 (PyAutoArray): fix the actual NaN/non-PD producer (candidate: fnnls.py:134 divide-by-zero; degenerate Hilbert-mesh vertices) so the fit is not merely tolerated but correct.

🤖 Generated with Claude Code

…equivalent)

TEST_MODE=2's _fit_bypass_test_mode calls the likelihood once to verify it runs.
If that single instance is pathological (non-PD inversion, or a degenerate
Delaunay mesh yielding NaN vertices), the analysis raises FitException — the
signal a real sampler absorbs by resampling. With no sampler in test mode, this
hard-failed the run, making flaky pixelization scripts intermittently break CI
smoke (autolens_workspace#307; interferometer Delaunay qhull "Points cannot
contain NaN" via source_pix_2 -> _fit_bypass_test_mode -> log_likelihood_function).

Catch FitException at the single verification eval, log the cause, and keep the
existing -1.0e99 sentinel (the same effect as resample-to-reject). Only
FitException is caught — genuine code errors still propagate. Central fix:
benefits every analysis (imaging, interferometer, galaxy), not just this script.

Tests: FitException tolerated -> fit completes with sentinel; ValueError still
propagates. 108 search/result/samples tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Jammy2211

Copy link
Copy Markdown
Collaborator Author

Design validation (parity with existing autofit conventions). This isn't a bespoke workaround — FitException → resample sentinel is how autofit already handles pathological instances in three places, and this change makes the test-mode bypass consistent with them:

  1. Real numpy fitness wrapperfitness.py:255-256:

    except exc.FitException:
        return self.resample_figure_of_merit

    The live sampler already treats a FitException as a resample-to-reject. The bypass is the only likelihood call that didn't.

  2. Sentinel value matches the searches these scripts use — Nautilus and Dynesty set resample_figure_of_merit=-1.0e99 (nautilus/search.py:194,215, dynesty/.../abstract.py:184). That is exactly the -1.0e99 the bypass already defaults to (abstract_search.py:872) and that this fix retains — so a resample-rejected instance produces the identical sentinel in test mode and in a real Nautilus fit.

  3. Initializerinitializer.py:35,39 treats both FitException and figure_of_merit < -1e98 as "invalid, resample" (return None), and its own test-mode bypass uses -1.0e99 (initializer.py:233).

Scope is deliberately narrow (only FitException, matching fitness.py) so genuine code errors still fail test mode loudly. Complements PyAutoLens#639 (raise_fit_exception preserves the cause), whose body explicitly flagged the "single evaluation not absorbed by a sampler under PYAUTO_TEST_MODE" problem but scoped the tolerance out — this PR completes it.

CI: unittest 3.12 + 3.13 green, docs green.

@Jammy2211
Jammy2211 merged commit 4d0b3f7 into main Jul 21, 2026
5 checks passed
@Jammy2211
Jammy2211 deleted the feature/interferometer-delaunay-flaky-fitexception branch July 21, 2026 18:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pending-release PR queued for the next release build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant