Skip to content

Add post-freeze abstention stress for prospective designs - #16

Draft
FlorianPfaff wants to merge 39 commits into
agent/profiled-design-evidence-20260825from
agent/design-open-set-stress-20260825
Draft

Add post-freeze abstention stress for prospective designs#16
FlorianPfaff wants to merge 39 commits into
agent/profiled-design-evidence-20260825from
agent/design-open-set-stress-20260825

Conversation

@FlorianPfaff

@FlorianPfaff FlorianPfaff commented Aug 25, 2026

Copy link
Copy Markdown
Member

Scope

This is a separate, post-freeze sensitivity benchmark stacked on the corrected profiled-Gaussian design evidence. It does not alter or replace the immutable five-seed matched-generator baseline.

It adds:

  • matched pure-generator recovery at the fixed N=60 benchmark allocation, with Wilson intervals;
  • disjoint deterministic threshold-calibration, calibration-audit, and evaluation streams, all hash-bound in the artifact;
  • a familywise null threshold over the best of six pure labels, a separate winner/runner ambiguity threshold, and an all-six ridge adequacy gate whose penalty is selected inside training data only;
  • null evaluation, all 15 equal 50/50 in-span mixtures, and one explicitly bounded full-grid-orthogonalized tanh(surprise) residual probe;
  • explicit abstention and false-confident-pure-call rates rather than forced closed-set classification;
  • checksum-verified ingestion of the chronologically earlier heuristic N=60 allocation used by the accepted five-seed benchmark;
  • clean-Git enforcement, a canonical configuration digest, payload hashes, CLI, tests, and scientific-scope documentation.

The primary allocation is locked by SHA-256 a823be49faf6c6cbebf60b11d4b5ca895cf7734d6e9c577ee98f97a5907b69b2 and source commit 1b2028929ac6ebc1cce0882f0c22af9918044342. It was fixed before either the exact certificate or stress results. Certified N=45/N=60 allocations are not substituted after observing their downstream recovery.

The mixture and nonlinear probes are diagnostics, not an open-set robustness certificate. The simulation remains conditional on effectively independent Gaussian observations, the fixed finite grid, globally standardized candidates, and independently instantiated cells. It is not an executable ordered-history or animal protocol. N=60 is a benchmark count, while the analytic N_eff index remains a population observation-equivalent quantity.

Verification state

At head 0a0a5fba03ca47c04963a636b9041fd8c5f1df08, the sole prior CI issue was corrected import ordering; the new CI and an independent server gate are running. No claim-bearing stress artifact has yet been committed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant