Skip to content

test(se-audit): fiddly-bits SE parity locks (C2 DOF, C3 df, C4 intercept SE, C5 Yatchew p/sigma2, G2 cluster band)#628

Merged
igerber merged 1 commit into
mainfrom
feature/se-audit-fiddly-locks
Jul 6, 2026
Merged

test(se-audit): fiddly-bits SE parity locks (C2 DOF, C3 df, C4 intercept SE, C5 Yatchew p/sigma2, G2 cluster band)#628
igerber merged 1 commit into
mainfrom
feature/se-audit-fiddly-locks

Conversation

@igerber

@igerber igerber commented Jul 6, 2026

Copy link
Copy Markdown
Owner

Summary

  • Second "fiddly-bits" coverage batch of the standard-error parity audit (follow-up to test(se-audit): CI-lock flagship + previously-unasserted SE parity (G2 + C1-C3/C5) #624). Pins previously-unasserted golden SE/DOF/intercept fields that already matched R but had no CI assertion — the latent-risk pattern that once hid the CallawaySantAnna reg-method gap:
    • C4 — estimatr HC1/CR1 2SLS intercept SE, via an opt-in return_intercept_se hook on the private _fit_mass_point_2sls (surfaces sqrt(V[0,0]); @overload keeps every caller's 3-tuple return byte-unchanged). Matches estimatr::iv_robust se_intercept to ~1e-15.
    • C2TwoWayFixedEffects(hc2_bm) dof_hc2_bm and MultiPeriodDiD(hc2_bm) per-period dof_per_coef, via CI-inversion (no new result fields): reconstruct conf_int from the golden Satterthwaite DOF — with ATT+SE already pinned, the CI matches iff the DOF matches.
    • C3 — PlaceboTests leave-one-out df, recovered from the public leave_one_out_effects finite count.
    • C5 — DIDHAD Yatchew linearity test p_value + sigma2_lin + sigma2_diff (through the documented N/(N-1) sample-vs-population convention shift; sigma2_diff holds despite the literal-2G divisor — verified by measurement).
    • G2fixest DiD/TWFE cluster-robust SE band pin within the documented ~0.25% CR1 small-sample DOF-convention band.
  • C1 (StackedDiD intercept SE) dropped after measurement: it diverges ~0.3% from R — a nuisance-parameter reference-cell/parameterization gap, not machine-precision lockable (the event-study interaction SEs already match ~2e-13). The candidate source field was reverted, so this PR makes zero result-class API changes. Tracked in TODO.md.

Methodology references (required if estimator / math changes)

  • Method name(s): HeterogeneousAdoptionDiD mass-point Wald-IV / 2SLS SE (intercept surfacing); TwoWayFixedEffects / MultiPeriodDiD CR2 Bell-McCaffrey Satterthwaite DOF; PlaceboTests leave-one-out; DIDHAD Yatchew linearity test; fixest cluster-robust SE.
  • Paper / source link(s): de Chaisemartin & D'Haultfœuille (HAD); Bell & McCaffrey (2002); estimatr / fixest R packages. See docs/methodology/REGISTRY.md.
  • Any intentional deviations from the source (and why): estimatr classical intercept SE excluded (same documented O(1/n) projection/DOF deviation as the slope); fixest cluster SE band-pinned only (documented ~0.25% CR1 DOF-convention difference); StackedDiD intercept SE not locked (~0.3% nuisance-parameter parameterization gap). All tracked in TODO.md / REGISTRY.md.

Validation

  • Tests added/updated: tests/test_estimatr_iv_robust_parity.py, tests/test_estimators_vcov_type.py, tests/test_did_had_parity.py, tests/test_methodology_placebo.py, tests/test_fixest_did_twfe_parity.py. New locks pass at ~1e-10 to ~1e-16; the 3-tuple return contract of _fit_mass_point_2sls is byte-unchanged across all callers (verified via tests/test_had.py).
  • Backtest / simulation / notebook evidence (if applicable): N/A.

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

🤖 Generated with Claude Code

https://claude.ai/code/session_0178r8ZK6VRiXbaBiknrWjQB

…p/sigma2, fixest cluster band

Second "fiddly-bits" coverage batch of the SE parity audit. Pins previously-
unasserted golden SE/DOF/intercept fields that already matched R but had no CI
assertion (the latent-risk pattern that once hid the CallawaySantAnna reg gap):

- C4: estimatr HC1/CR1 2SLS intercept SE via an opt-in return_intercept_se hook on
  the private _fit_mass_point_2sls (surfaces sqrt(V[0,0]); @overload keeps every
  caller's 3-tuple return byte-unchanged). classical intercept excluded (O(1/n)).
- C2: TWFE dof_hc2_bm and MPD per-period dof_per_coef via CI-inversion (no new
  result fields; reconstruct conf_int from golden DOF, att+se already pinned).
- C3: PlaceboTests leave-one-out df via the public leave_one_out_effects count.
- C5: Yatchew p_value + sigma2_lin + sigma2_diff (through the N/(N-1) shift).
- G2: fixest DiD/TWFE cluster-SE band pin within the documented ~0.25% CR1 band.

C1 (StackedDiD intercept SE) dropped after measurement: it diverges ~0.3% from R
(nuisance-parameter reference-cell gap, not machine-precision lockable; the event-
study interaction SEs already match ~2e-13) - documented in TODO. Remaining
deferrals (G2 hetero golden regen, boundary_gap, C6-C8) tracked in TODO.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0178r8ZK6VRiXbaBiknrWjQB
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown

Overall Assessment

✅ Looks good. No unmitigated P0/P1 findings.

Executive Summary

  • The only implementation change is a private, opt-in _fit_mass_point_2sls(..., return_intercept_se=True) hook; default production callers still receive the existing 3-tuple.
  • The intercept SE uses the already-computed 2SLS sandwich diagonal sqrt(V[0,0]), consistent with the HAD mass-point 2SLS registry entry and in-code docstring.
  • The CR2/Bell-McCaffrey DOF, Yatchew convention shift, placebo LOO df, and fixest cluster-SE band additions are parity/test locks, not estimator behavior changes.
  • Documented deviations/deferred items are tracked in REGISTRY.md / TODO.md, so they are not blockers.
  • I could not run tests in this environment because pytest and numpy are unavailable.

Methodology

Finding: P3 informational
Impact: The estimatr classical intercept SE is intentionally not parity-locked because the weighted classical path has the same documented O(1/n) projection/DOF deviation as the slope. This is explicitly documented in docs/methodology/REGISTRY.md:L3164-L3170 and tracked in TODO.md:L69.
Concrete fix: No action required.

Finding: P3 informational
Impact: The fixest cluster-robust SE assertion is a band lock, not machine-precision parity. The remaining exact hetero/cluster golden regeneration is tracked in TODO.md:L69; related CR1/fixest small-sample convention context is documented in docs/methodology/REGISTRY.md:L342-L357.
Concrete fix: No action required.

No undocumented methodology mismatch found in the changed implementation. The new intercept SE is derived from V = ZtWX_inv @ Omega @ ZtWX_inv.T and returns sqrt(V[0,0]) only when requested: diff_diff/had.py:L2619-L2634.

Code Quality

No findings. The optional return arity is centralized through _pack, covering early exits and success returns consistently: diff_diff/had.py:L2487-L2502.

Performance

No findings. The new intercept SE calculation reuses the existing 2x2 sandwich and adds only one scalar square root on the opt-in path.

Maintainability

No findings. Existing production call sites continue to unpack three values with return_intercept_se omitted: diff_diff/had.py:L3817-L3825 and diff_diff/had.py:L4732-L4745.

Tech Debt

Finding: P3 informational
Impact: Remaining SE-audit work is explicitly deferred and scoped in TODO.md:L69, including fixest machine-precision hetero/cluster SE, PlaceboTests boundary_gap, StackedDiD intercept SEs, and estimatr classical intercept SE.
Concrete fix: No action required for this PR.

Security

No findings. The diff does not introduce secrets, external I/O, credential handling, or unsafe execution paths.

Documentation/Tests

Finding: P3 informational
Impact: The new tests improve parity coverage for HAD 2SLS intercept SE, CR2/BM DOF CI inversion, Yatchew p/sigma components, placebo LOO df, and fixest cluster-SE bands. I could not execute them locally because pytest is not installed and importing the package also fails due missing numpy.
Concrete fix: Ensure CI runs the touched test files: tests/test_estimatr_iv_robust_parity.py, tests/test_estimators_vcov_type.py, tests/test_did_had_parity.py, tests/test_fixest_did_twfe_parity.py, and tests/test_methodology_placebo.py.

@igerber igerber added the ready-for-ci Triggers CI test workflows label Jul 6, 2026
@igerber igerber merged commit 1cf035e into main Jul 6, 2026
35 of 36 checks passed
@igerber igerber deleted the feature/se-audit-fiddly-locks branch July 6, 2026 22:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-ci Triggers CI test workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant