perf(efficient-did, had): chunk multiplier-bootstrap weight generation to bound peak memory

[igerber]

Summary

• Wire the EfficientDiD and HeterogeneousAdoptionDiD multiplier bootstraps through the diff_diff/bootstrap_chunking.py helper (PR-B2 of the memory-scaling work), so the dense (n_bootstrap × n_units) multiplier-weight matrix — the dominant allocation at large n_units (~40 GB at 5M units × 999 reps) — is generated and consumed one draw-block at a time instead of materialized in full. These were the two remaining consumers of the same matrix that CallawaySantAnna chunking (perf(callaway-santanna): chunk multiplier-bootstrap weight generation to bound peak memory #561) already eliminated.
• EfficientDiD: tile the per-(g,t) EIF perturbation loop over draw-blocks across the survey-PSU / cluster / unit weight paths; the overall/event-study/group aggregations and all stats read the small bootstrap_atts matrix unchanged. eif scaling stays keyed on unit_level_weights (a separate axis from the weight path) so a weights-only SurveyDesign keeps the survey-score perturbation. The per-(g,t) scaled EIF is a bounded O(n_units) temporary, not an O(n_gt × n_units) cache.
• HAD: accumulate weights @ influence into the small (B, n_horizons) perturbations one block at a time on both the survey (psu_block @ Psi_psu) and iid paths; the Psi_psu aggregation, stratum centering, and sup-t reduction (max-over-horizons then quantile) are unchanged.
• Weight stream bit-identical on both backends; end-to-end bootstrap SEs and sup-t critical values match the un-chunked path to within BLAS floating-point reassociation (~1 ULP, far below bootstrap Monte-Carlo error). Stratified survey designs are unchanged (full generation + sliced blocks; the deferred per-stratum tiling is tracked in TODO.md).

Methodology references

• Method name(s): EfficientDiD multiplier bootstrap on EIF values (fixed cohort-size aggregation); Heterogeneous Adoption DiD event-study sup-t simultaneous band (δ = ξ @ Ψ with analytical-SE normalization).
• Paper / source link(s): docs/methodology/REGISTRY.md EfficientDiD multiplier-bootstrap and HAD sup-t bootstrap sections. This is a numerical implementation change (draw-block tiling of weight generation/consumption), not a methodology change.
• Any intentional deviations from the source (and why): None. The perturbation formulas, NaN/degenerate-SE guards, and aggregation conventions are preserved; the weight stream is bit-identical and end-to-end results differ only by BLAS reassociation (~1 ULP).

Validation

• Tests added/updated: tests/test_bootstrap_chunking.py — EfficientDiD chunk-invariance across 4 paths (unit / cluster / survey-PSU / weights-only SurveyDesign decoupling) and HAD (survey cband + a direct iid-branch test), each comparing tiny-chunk vs single-chunk to assert_allclose.
• Backtest / simulation / notebook evidence: N/A. Verified on both backends — chunk-invariance plus the full EfficientDiD and HAD regression/methodology/parity suites green.

Security / privacy

• Confirm no secrets/PII in this PR: Yes

---

🤖 Generated with Claude Code

[github-actions]

Overall Assessment: ✅ Looks Good

Executive Summary

• No unmitigated P0/P1 findings.
• Affected methods: EfficientDiD EIF multiplier bootstrap and HeterogeneousAdoptionDiD event-study sup-t multiplier bootstrap.
• Methodology is preserved: this PR tiles draw generation/consumption only; perturbation formulas, survey PSU/FPC handling, fixed aggregation, and NaN gates remain intact.
• Deferred stratified survey tiling is properly tracked in TODO.md.
• I could not run tests locally because pytest is not installed in this environment.

Methodology

• Severity: None.
  Impact: EfficientDiD still perturbs ATT(g,t) with unit/cluster/PSU multipliers and preserves survey-score scaling via unit_level_weights * eif / sum(w); fixed cohort-size bootstrap aggregation remains consistent with the registry note. See diff_diff/efficient_did_bootstrap.py:L190-L240 and docs/methodology/REGISTRY.md:L1040-L1075.
  Concrete fix: None.

• Severity: None.
  Impact: HAD sup-t bootstrap still computes PSU/unit perturbations as weights @ influence, normalizes by analytical SE, reduces by max-over-horizons, and NaN-gates degenerate bands. See diff_diff/had.py:L2163-L2259.
  Concrete fix: None.

Code Quality

• Severity: None.
  Impact: No new inline inference anti-patterns were introduced; changed code does not add t_stat = effect / se or else 0.0 inference guards. Existing inference paths continue through safe_inference() / compute_effect_bootstrap_stats().
  Concrete fix: None.

Performance

• Severity: None.
  Impact: The main allocation is bounded by block-sized multiplier matrices. EfficientDiD correctly bases survey block size on expanded unit width, and HAD avoids materializing the full (n_bootstrap × n_units) weights-only matrix. See diff_diff/bootstrap_chunking.py:L44-L100, diff_diff/efficient_did_bootstrap.py:L118-L216, and diff_diff/had.py:L2169-L2218.
  Concrete fix: None.

Maintainability

• Severity: None.
  Impact: Both estimators now share diff_diff/bootstrap_chunking.py, reducing duplicated dense-weight generation logic while preserving old generator contracts.
  Concrete fix: None.

Tech Debt

• Severity: P3-informational, tracked in TODO.md.
  Impact: Stratified survey multiplier generation still materializes the full (n_bootstrap × n_psu) matrix before slicing. This is explicitly documented as deferred work and is not a blocker under the review rules. See TODO.md:L68 and diff_diff/bootstrap_chunking.py:L143-L153.
  Concrete fix: No action required for approval; future work can tile per-stratum generation.

Security

• Severity: None.
  Impact: No secrets, unsafe file/network access, or security-sensitive behavior found in the changed files.
  Concrete fix: None.

Documentation/Tests

• Severity: None for PR content.
  Impact: New tests cover EfficientDiD unit, cluster, survey-PSU, and weights-only survey paths, plus HAD survey and direct iid branches. See tests/test_bootstrap_chunking.py:L327-L506. CHANGELOG.md and TODO.md document the behavior and remaining deferred stratified gap.
  Concrete fix: None.

• Severity: P3-informational, review environment.
  Impact: I could not execute tests/test_bootstrap_chunking.py because pytest is not installed here.
  Concrete fix: Ensure CI runs the added test file.