Skip to content

fix(callaway): IF-based per-cell SE for no-covariate doubly-robust#627

Merged
igerber merged 1 commit into
mainfrom
fix/cs-dr-nocov-per-cell-se
Jul 6, 2026
Merged

fix(callaway): IF-based per-cell SE for no-covariate doubly-robust#627
igerber merged 1 commit into
mainfrom
fix/cs-dr-nocov-per-cell-se

Conversation

@igerber

@igerber igerber commented Jul 6, 2026

Copy link
Copy Markdown
Owner

Summary

  • CallawaySantAnna no-covariate estimation_method="dr" (the default method) per-cell (g,t) SE now uses the influence-function form sqrt(sum(phi^2)) instead of the ddof=1 plug-in sqrt(var_t/n_t + var_c/n_c) — the last per-cell SE not on the IF convention already used by the reg/ipw branches and R's DRDID::drdid_panel (an O(1/n) deviation, and method-dependent on the default path).
  • Without covariates DR reduces to difference in means, so the per-cell SE is now bit-identical to the vectorized no-covariate regression path. The returned influence function is left byte-identical, so point estimates, aggregated SEs, and event-study SEs are unchanged (aggregation/bootstrap already consumed this same IF).
  • Updates the methodology registry (DR no-cov now IF-based), removes the resolved TODO row, adds a CHANGELOG entry.

Methodology references (required if estimator / math changes)

  • Method name(s): CallawaySantAnna (Callaway & Sant'Anna 2021); DRDID panel doubly-robust per-cell SE (Sant'Anna & Zhao 2020 / DRDID::drdid_panel).
  • Paper / source link(s): docs/methodology/REGISTRY.md "CallawaySantAnna" section (DRDID IF-parity note).
  • Any intentional deviations from the source (and why): None new. This removes a previously-documented deviation (the DR no-covariate ddof=1 plug-in), bringing that branch into line with R's analytical SE. Two pre-existing residual deviations (propensity clipping; no-covariate ipw treated as unconditional) are unchanged.

Validation

  • Tests added/updated: tests/test_methodology_callaway.py — new TestDRNoCovariateSEUniformity: reg-vs-dr per-cell effect+SE parity (atol=1e-12), a hand-calculated IF SE with a plug-in-direction sentinel, and aggregate + event-study SE invariance.
  • Verified: per-cell dr no-covariate SE now equals reg exactly (ratio 1.0125–1.0138 → 1.000000); point estimates, overall_se, and event-study SEs unchanged. reg / ipw / DR-covariate / RCS paths byte-stable (160 tests across test_methodology_callaway, test_csdid_ported, test_staggered_rc).
  • Backtest / simulation / notebook evidence (if applicable): N/A

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

🤖 Generated with Claude Code

https://claude.ai/code/session_01LHDijzf8zHXk5T8ahS2mKi

CallawaySantAnna's no-covariate doubly-robust (dr, the default
estimation_method) per-cell (g,t) SE used the ddof=1 plug-in
sqrt(var_t/n_t + var_c/n_c) — the last per-cell SE not on the
influence-function form sqrt(sum(phi^2)) that the reg/ipw branches and R's
DRDID::drdid_panel use (O(1/n) deviation, method-dependent on the default
path). Without covariates DR reduces to difference in means, so the per-cell
SE is now bit-identical to the vectorized reg path.

Point estimates, aggregated SEs, and event-study SEs are unchanged: the
returned influence function is left byte-identical and aggregation/bootstrap
already consumed it. Closes the CS-scaling Actionable TODO row.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LHDijzf8zHXk5T8ahS2mKi
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown

Overall Assessment

✅ Looks good. No unmitigated P0/P1 findings.

Executive Summary

  • Affected method: CallawaySantAnna, panel no-covariate estimation_method="dr" per-cell SE in diff_diff/staggered.py::_doubly_robust.
  • The new SE uses the same centered influence-function scaling as the no-covariate reg path and the aggregation/bootstrap paths.
  • The change matches the updated Methodology Registry note and the DRDID analytical IF convention for drdid_panel.
  • No new inline inference, partial NaN guard, public parameter propagation issue, or control-group logic issue found.
  • Tests were added for reg/dr parity, hand-calculated IF SE, and aggregate/event-study invariance, but I could not execute them locally because pytest and numpy are unavailable in this runner.

Methodology

  • Severity: N/A
  • Impact: No methodology defect found. The affected branch now computes inf_treated, inf_control, and se = sqrt(sum(phi^2)), matching the sibling no-covariate regression path and the registry’s documented convention: diff_diff/staggered.py:L3460-L3480, diff_diff/staggered.py:L1225-L1239, docs/methodology/REGISTRY.md:L520-L549.
  • Concrete fix: None.

Code Quality

  • Severity: N/A
  • Impact: No code-quality blocker found. The changed branch reuses computed means and does not add inline t_stat = effect / se inference. Downstream inference still goes through safe_inference: diff_diff/staggered.py:L2307-L2319, diff_diff/utils.py:L327-L354.
  • Concrete fix: None.

Performance

  • Severity: N/A
  • Impact: No meaningful performance regression found. The change replaces two sample-variance calls with existing IF sum-of-squares work on the same arrays.
  • Concrete fix: None.

Maintainability

  • Severity: N/A
  • Impact: No maintainability issue found. The comments explain why this branch now aligns with reg/DRDID IF scaling and why aggregation is unchanged: diff_diff/staggered.py:L3464-L3479.
  • Concrete fix: None.

Tech Debt

  • Severity: N/A
  • Impact: Removing the resolved TODO row is appropriate because the registry now documents the updated behavior: TODO.md:L31-L36, docs/methodology/REGISTRY.md:L544-L549.
  • Concrete fix: None.

Security

  • Severity: N/A
  • Impact: No changed-code security issue or accidental secret found in the diff scope.
  • Concrete fix: None.

Documentation/Tests

  • Severity: P3-informational
  • Impact: The new tests cover the key correctness claims: per-cell reg/dr parity, hand-calculated IF SE versus old plug-in direction, and aggregate/event-study SE invariance: tests/test_methodology_callaway.py:L334-L423. Local execution was not possible because this environment lacks pytest and numpy.
  • Concrete fix: No PR code fix required; ensure CI runs tests/test_methodology_callaway.py::TestDRNoCovariateSEUniformity plus the existing Callaway/SantAnna methodology suite.

@igerber igerber added the ready-for-ci Triggers CI test workflows label Jul 6, 2026
@igerber igerber merged commit acb89d0 into main Jul 6, 2026
35 of 36 checks passed
@igerber igerber deleted the fix/cs-dr-nocov-per-cell-se branch July 6, 2026 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-ci Triggers CI test workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant