Sub-project A: strict EF-CQS + determinism CI gate

[sangicook]

Summary

• New ef_cqs_strict observation field on CompanyCQS + CQSResult. Strict denominator keeps explained_variance_count as failures instead of laundering them into free passes. Runs parallel to lenient ef_cqs; lenient stays the decision gate during a ≥5-cohort-run observation window.
• Determinism CI gate at tests/xbrl/standardization/test_determinism.py — runs compute_cqs twice on a fixed 10-company cohort and asserts max per-company EF-CQS delta < DETERMINISM_THRESHOLD (5e-05, measured 0.0 on 2026-04-06 = bit-identical). Joins the nightly regression suite automatically via @pytest.mark.regression.
• Chokepoint decision threshold helper — get_decision_threshold() returns 0.005 normally, 0.01 when EDGAR_DETERMINISM_DEGRADED=1. Unwired in this PR; Sub-project B's chokepoint will consume it. The determinism test failure message points operators at this escape hatch.

Why now

The deep-consensus session on 2026-04-06 produced a Phase A Safety-Net Minimum. The deepthinker synthesis was explicit: "Without bit-identical measurement (determinism) and honest measurement (strict CQS), the chokepoint and regression gate in Sub-project B would be making decisions off measurement noise and laundered free-pass credit." This PR is that foundation.

Run 025 rebaseline (first parallel measurement)

All 123 onboarded companies, snapshot_mode=True, 1642s runtime:

Metric	Value
EF-CQS (lenient, current gate)	0.8537 (vs Run 022 0.8492)
EF-CQS (strict, observed)	0.8151
Delta	+0.0386 (3.86 pp of laundering)
explained_variance_count total	200 across 84 of 123 companies (68%)

Top laundering contributors (utilities + conglomerates dominate):

Ticker	Delta	Lenient	Strict	EV
DUK	+0.2400	0.8400	0.6000	10
GE	+0.1911	0.7857	0.5946	9
SO	+0.1905	0.8571	0.6667	8
NEE	+0.1862	0.8148	0.6286	8
BRK-B	+0.1754	0.8519	0.6765	7

39 of 123 companies have zero laundered divergences.

Raw output: edgar/xbrl/standardization/escalation-reports/run_025_strict_rebaseline_2026-04-06.json

Determinism measurement

All 10 DETERMINISM_TEST_COHORT tickers (AAPL, MSFT, JPM, BAC, XOM, WMT, JNJ, CAT, V, NEE) showed a per-company EF-CQS delta of 0.0000000000 between back-to-back runs. The pipeline is currently fully deterministic — DETERMINISM_THRESHOLD set to 5 × max(observed_noise, 0.00001) = 5e-05 per the Sub-project A spec.

Cut-over criterion

The gate flips from lenient → strict in a separate PR (likely bundled with Sub-project B's chokepoint) after both conditions hold:

1. At least 5 cohort runs producing parallel (lenient, strict) pairs at the all-onboarded scope.
2. Zero determinism CI regressions during the same window.

This PR is Run 1 of 5.

Out of scope (Sub-project B / C)

• Chokepoint (propose_global_change) and baseline regression gate — Sub-project B
• Loader-level hash registry + ADD_DEFINITION_OVERRIDE typed action — Sub-project B
• Grouped escalations by (metric, root_cause) + dual scoring — Sub-project C
• Flipping the gate from lenient to strict — separate PR after observation window

Test plan

• 20/20 scoring integrity tests pass (15 existing + 6 new TestEfCqsStrict / TestEfCqsStrictAggregation, minus 1 redundant test removed during simplify review)
• 279/279 broader standardization fast suite passes
• Determinism CI gate passes end-to-end in ~12 min (max delta 0.0 across all 10 cohort members, well below 5e-05 threshold)
• EDGAR_DETERMINISM_DEGRADED=1 env var correctly widens get_decision_threshold() from 0.005 to 0.01
• determinism pytest marker registered and accepted under --strict-markers
• CQSResult.to_dict() / from_dict() roundtrip preserves ef_cqs_strict (nested and top-level)
• Run 025 measurement script is idempotent and re-runnable (used get_config() helper, not raw yaml)
• Next overnight regression run confirms determinism gate is green on CI infra (not local)

Files

Modified (5): auto_eval.py (+91/-8), test_scoring_integrity.py (+115), architecture.md (+6/-4), roadmap.md (+13), pyproject.toml (+1)

New (4): test_determinism.py, scripts/run_025_rebaseline.py, docs/autonomous-system/strict-cqs-rebaseline.md, escalation-reports/run_025_strict_rebaseline_2026-04-06.json

🤖 Generated with Claude Code

[sangicook]

Review fixes — 5 commits pushed on top of 8486d70f

Addresses the code review on this PR. All commits are test/refactor only; no production code behavior changed. Pushed as a fast-forward.

Commit	Review item	Priority	Scope
7f536630	#3 freeze DETERMINISM_TEST_COHORT	Minor	auto_eval.py — list → tuple so the CI gate's cohort is immutable by construction (spec called for "frozen"). Both consumers only iterate, so drop-in compatible.
2f206cc9	#1 pin EDGAR_DETERMINISM_DEGRADED parsing contract	Important	New tests/xbrl/standardization/test_decision_threshold.py (12 cases). Sub-project B's chokepoint will consume get_decision_threshold() — spec required "the helper + its tests." Pins strict == "1" parsing against 9 edge-case values ("0", "true", "TRUE", "yes", "", " 1", "1 ", "01", "2"), plus the unset, exact-match, and degraded > normal invariant.
2501f1ea	#4 legacy from_dict backward-compat	Minor	New case in TestEfCqsStrictAggregation — passes a dict without ef_cqs_strict (both top-level and nested CompanyCQS) and asserts restored.ef_cqs_strict == 0.0. Protects re-reading of pre-Sub-A ledger/graveyard JSON.
e977da40	#7 strengthen test_strict_zero_division_safe	Minor	Expanded to two cases. Case A (unchanged intent): all-unverified → both ef_cqs and ef_cqs_strict return 0.0 via guard. Case B (new): 1 passing + 1 known_divergences + 1 unverified → lenient 1/1 = 1.0 vs strict 1/2 = 0.5. Case B fails loudly if the strict denominator ever forgets to subtract unverified_count or accidentally subtracts explained_variance_count.
bcc3edd2	#2 determinism gate also covers ef_cqs_strict	Important	Tracks both lenient and strict deltas in parallel, asserts max(max_lenient, max_strict) < DETERMINISM_THRESHOLD. CI logs both columns separately for faster root-causing. Catches FP-reduction or iteration-order bugs that affect only the strict path's wider denominator before Sub-project B flips the gate.

Local verification (fast suite)

$ python -m pytest tests/xbrl/standardization/test_scoring_integrity.py \
                   tests/xbrl/standardization/test_decision_threshold.py -q
33 passed in 8.74s

Determinism test collects cleanly (1 test, still @pytest.mark.regression @pytest.mark.slow). It will run on the next nightly regression pass — infra verification, not a code change.

Deliberately deferred

• Minor Unhashable type Series for filings xbrl  #5 (Run 025 JSON snapshot sprawl across the 5-run observation window) — out of scope for this PR. Filing a follow-up to consolidate to a rolling run_log.jsonl before Run 026.
• Commit message "6 new" tests — actually 5. Trivial; not worth amending a squashed commit.

Diff summary

 edgar/xbrl/standardization/tools/auto_eval.py      |  7 +-
 tests/xbrl/standardization/test_decision_threshold.py | 84 +++++++++++++++++++
 tests/xbrl/standardization/test_determinism.py     | 46 ++++++++---
 tests/xbrl/standardization/test_scoring_integrity.py | 96 ++++++++++++++++++++--
 4 files changed, 214 insertions(+), 19 deletions(-)

🤖 Generated with Claude Code