Skip to content

Commit e977da4

Browse files
ychanclaude
andcommitted
test(auto-eval): strengthen test_strict_zero_division_safe with mixed case
Address review comment #7 (Minor) on PR #764 Sub-project A. The original zero-division test only asserted ``ef_cqs_strict == 0.0`` in an all- unverified cohort, which would pass even if the strict denominator math were broken (numerator is also 0 in that state). That's necessary but insufficient coverage. Strengthened to two cases within the same test: Case A (unchanged intent) — all-unverified degenerate state: Guards ef_cqs AND ef_cqs_strict both return 0.0 without raising. Case B (new) — 1 passing + 1 explained_variance + 1 unverified: effective_total = 3 - 0 - 1 - 1 = 1 → lenient ef_cqs = 1/1 = 1.0 strict_total = 3 - 0 - 1 = 2 → strict ef_cqs_strict = 1/2 = 0.5 Case B would fail loudly if the strict denominator forgot to subtract unverified_count, or accidentally subtracted explained_variance_count (turning it into the lenient formula). The two cases together pin the full contract: guard correctness + arithmetic correctness. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 2501f1e commit e977da4

File tree

1 file changed

+39
-5
lines changed

1 file changed

+39
-5
lines changed

tests/xbrl/standardization/test_scoring_integrity.py

Lines changed: 39 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -356,14 +356,48 @@ def test_strict_lower_than_lenient_with_divergences(self):
356356
assert result.ef_cqs_strict < result.ef_cqs
357357

358358
def test_strict_zero_division_safe(self):
359-
"""Strict EF-CQS is 0.0 when strict denominator is 0 (degenerate case)."""
360-
# All metrics are unverified — strict_total = 0
361-
metrics = {
359+
"""Strict denominator math is correct across degenerate AND mixed cases.
360+
361+
The zero-division guard alone is necessary but not sufficient — a weak
362+
test that only asserts ``ef_cqs_strict == 0.0`` would still pass if the
363+
strict arithmetic were broken (since 0/anything and anything/0 both hit
364+
the 0.0 fallback). Two cases give the test real teeth:
365+
366+
Case A (degenerate) — all metrics unverified, strict_total = 0:
367+
Guard must return 0.0 without raising ZeroDivisionError. Lenient
368+
ef_cqs should also be 0.0 — asserting both proves the two paths
369+
share the same fallback semantics.
370+
371+
Case B (mixed) — 1 passing + 1 explained_variance + 1 unverified:
372+
total = 3, unverified_count = 1, explained_variance_count = 1
373+
effective_total (lenient) = 3 - 0 - 1 - 1 = 1 → ef_cqs = 1/1 = 1.0
374+
strict_total = 3 - 0 - 1 = 2 → ef_cqs_strict = 1/2 = 0.5
375+
This would fail loudly if the strict denominator were misspelled
376+
(e.g. forgetting to subtract unverified_count, or accidentally
377+
subtracting explained_variance_count).
378+
"""
379+
# Case A: degenerate — only the zero-division guard runs.
380+
degenerate = {
362381
"Revenue": _make_mapping_result("Revenue", validation_status="unverified"),
363382
}
364-
result = self._compute(metrics)
383+
result_a = self._compute(degenerate)
384+
assert result_a.ef_cqs_strict == 0.0, "zero-division guard failed on strict"
385+
assert result_a.ef_cqs == 0.0, "zero-division guard failed on lenient"
386+
assert result_a.unverified_count == 1
365387

366-
assert result.ef_cqs_strict == 0.0
388+
# Case B: mixed — both lenient and strict non-zero but different.
389+
mixed = {
390+
"Revenue": _make_mapping_result("Revenue"),
391+
"Capex": _make_mapping_result("Capex"), # marked as known_divergence below
392+
"R_and_D": _make_mapping_result("R_and_D", validation_status="unverified"),
393+
}
394+
result_b = self._compute(mixed, known_divergences={"Capex"})
395+
assert result_b.explained_variance_count == 1
396+
assert result_b.unverified_count == 1
397+
# Lenient laundering: 1 pass / 1 effective = 1.0
398+
assert result_b.ef_cqs == pytest.approx(1.0, abs=1e-9)
399+
# Strict honesty: 1 pass / 2 strict_total = 0.5
400+
assert result_b.ef_cqs_strict == pytest.approx(0.5, abs=1e-9)
367401

368402

369403
class TestEfCqsStrictAggregation:

0 commit comments

Comments
 (0)