Skip to content

Commit ad053ff

Browse files
igerberclaude
andcommitted
test(lpdid): R-parity validate complex-survey path vs survey::svyglm (Phase D2)
Pin the merged D1 LPDiD survey path end-to-end against survey::svyglm (Lumley) goldens on a dedicated staggered-absorbing survey panel. Closes Phase D of the LP-DiD salvage initiative. No diff_diff/ change (verify-and-document). Three structurally-distinct survey variance paths are R-validated per-horizon (point/SE/df) + pooled-post/pre: * VW full design (strata + PSU + FPC) vs svyglm(ids=~psu, strata, fpc, weights); df = n_PSU - n_strata = 10. * weights-only, unit injected as PSU vs svyglm(ids=~unit, weights); df = n_PSU - 1. * direct covariate inclusion vs svyglm(Dy ~ tr + x + factor(time)). Each LP-DiD horizon uses a FRESH svydesign over that horizon's clean long-difference sample (matching the library's per-sample resolution, not subset() of a full design). svyglm is the reference implementation of the Binder TSL sandwich, so it anchors the variance directly; the clean-sample construction is independently cross-checked against alexCardazzi/lpdid (unweighted VW event study matches to <1e-8), and every svyglm WLS point is gated == weighted feols on the same sample (<1e-7). Tolerances: point abs=1e-6, SE rtol=1e-5/abs=1e-7, df exact. New TestLPDiDSurveyParityR + generate_lpdid_survey_golden.R + lpdid_survey_panel.csv / lpdid_survey_golden.json (own seed -> absorbing/non-absorbing goldens byte-identical). REGISTRY Deviation #8 + checklist upgraded to "validated vs svyglm"; CHANGELOG; TODO row closed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 193f0ea commit ad053ff

7 files changed

Lines changed: 2285 additions & 4 deletions

File tree

CHANGELOG.md

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
2121
each a deferred follow-up. `LPDiDResults` gains `survey_metadata` / `n_strata` / `n_psu`, a
2222
`"survey_tsl"` `vcov_type`, and a Survey Design block in `summary()`. The non-survey path is
2323
byte-for-byte unchanged. Validated against `survey::svyglm` on the stacked long difference
24-
(numeric golden parity is the D2 follow-up).
24+
(full numeric golden parity in Phase D2, below).
25+
- **`LPDiD` complex-survey R-parity validation** (Phase D2). Pins the D1 survey path end-to-end
26+
against `survey::svyglm` (Lumley) goldens on a dedicated staggered-absorbing survey panel:
27+
per-horizon point/SE/df + pooled-post/pre for three variance paths: the variance-weighted full
28+
design (strata+PSU+FPC), the weights-only unit-injected-PSU design, and the direct-covariate
29+
variant (point ~1e-6, SE ~1e-5, df exact via the per-design `n_PSU - n_strata` / `n_PSU - 1`
30+
formula). Each horizon uses a fresh `svydesign` over that horizon's clean sample, matching the
31+
library's per-sample resolution; `svyglm` is the reference implementation of the Binder TSL
32+
sandwich so it anchors the variance directly, and the clean-sample construction is independently
33+
cross-checked against `alexCardazzi/lpdid` (the unweighted variance-weighted event study matches
34+
to <1e-8). New `tests/test_methodology_lpdid.py::TestLPDiDSurveyParityR` +
35+
`benchmarks/R/generate_lpdid_survey_golden.R` + `lpdid_survey_panel.csv` /
36+
`lpdid_survey_golden.json` (own seed, so the absorbing / non-absorbing goldens stay
37+
byte-identical). No estimator change.
2538
- **`TROP` non-absorbing (on/off) treatment support** (Athey, Imbens, Qu & Viviano 2025,
2639
§2.1 / Eq. 12 / Algorithm 2). New `non_absorbing` parameter (default `False`). The paper
2740
supports general assignment patterns ("units moving into and out of treatment"), not only

TODO.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,6 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m
6969
| `ImputationDiD` covariate-path variance lacks a dedicated parity anchor — only the no-covariate staggered panel is R-parity'd, though the covariate path shares the same validated projection code. Add a small dense-design **hand-calc** for the covariate projection (no external tooling), or a covariate (time-varying X) R `didimputation` golden asserting overall/ES SE parity (the golden variant needs local R). | `tests/test_methodology_imputation.py`, `benchmarks/R/generate_didimputation_golden.R` | imputation-validation | Mid | Low |
7070
| Add true half-sample BRR replicate-weight regressions per estimator family (current tests use Fay-like 0.5/1.5 perturbations; `test_survey_phase6.py` covers true BRR at the helper level). | `tests/test_replicate_weight_expansion.py` | #253 | Mid | Low |
7171
| Port the CI `<notebook-prose>` extraction into the reviewer-eval harness so `docs/tutorials/*.ipynb` cases (currently guarded out of `verify-corpus`/`run`) can be reviewed with CI-equivalent context. | `tools/reviewer-eval/adapters/ci_prompt.py` | local-review | Mid | Low |
72-
| **`LPDiD` survey-design R-parity (PR-D2)** — pin the stratified-PSU Taylor-linearization standard errors against `survey::svyglm` on the stacked long difference (per-horizon + pooled-post point/SE/df), mirroring `benchmark_survey_estimators.R`. The D1 build ships the survey path validated by pure-Python invariants (reduction/FPC/stratification/lonely-PSU/NaN-consistency); D2 adds the numeric `svyglm` golden + parity test. | `benchmarks/R/`, `tests/test_methodology_lpdid.py` | PR-D1 | Mid | Low |
7372

7473
---
7574

0 commit comments

Comments
 (0)