Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,6 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m

| Issue | Location | Origin | Effort | Priority |
|-------|----------|--------|--------|----------|
| SE-audit CI-lock — remaining "fiddly bits" after the second coverage batch landed (that batch pinned C2 `dof_hc2_bm`/`dof_per_coef` via CI-inversion, C3 LOO `df`, C4 estimatr HC1/CR1 intercept SE, C5 Yatchew `p`/`sigma2_lin`/`sigma2_diff`, and the G2 fixest cluster-SE band). Still deferred, each needing a golden regeneration, new computation, or a documented-deviation call — **(a) G2 hetero/cluster is DONE (2026-07-07)**: the unbalanced/heteroskedastic-DGP regen landed, the DiD-path hetero AND cluster CR1 SEs are machine-precision-locked (the plain-OLS CR1 matches fixest exactly — the DOF-convention deviation is absorbed-FE-only), and the TWFE cluster band-pin is retained for the documented non-nested-FE ssc deviation (tracked under "Needs external reference"); TWFE has no public unclustered-hetero surface (auto-cluster convention). Remaining: **(b) PlaceboTests `boundary_gap`** — a permutation randomization-inference margin NOT computed anywhere in code (a new feature + result field, not a coverage lock); **(c) StackedDiD intercept SEs** (`se_cr1/cr2_intercept`, C1) — MEASURED to diverge ~0.3% from R: a nuisance-parameter reference-cell/parameterization gap, NOT machine-precision lockable (the event-study interaction SEs already match ~2e-13; surfacing it would add an unasserted, R-divergent public field); **(d) estimatr `classical` intercept SE** — same documented `O(1/n)` projection/DOF deviation as the slope (reference-only, excluded from parity). C6 CLOSED 2026-07-08 (dr remainder measured ~6e-11 ATT / ~2e-11 rel SE on the no-cov fixture and tightened to 1e-8 with the previously-missing SE assertion added; reg/ipw ATT bands 0.02/0.05 also tightened to 1e-8; the covariate-DR 2e-3 band stays as documented small-sample numerics). C7/C8 re-derivation IN PROGRESS from a fresh loose-tolerance inventory (2026-07-09): the TWFE live-R fixest parity bands were measured and tightened (ATT 1e-3→1e-12 machine, SE/CI 0.01→0.005 documented-CR1-band pins, p-value 0.01→1e-12 both-zero pin, root-caused to the clustered-CR1 inference-df convention: Python t(residual df) vs fixest t(G−1) — REGISTRY Note added + its own Actionable row below); remaining inventory candidates are statistical-by-design (DGP recovery, RNG moments) or fixture-not-committed (triplediff CSVs) — re-measure any new loose golden-backed sites before tightening. | `benchmarks/R/generate_fixest_did_twfe_golden.R`, `tests/test_fixest_did_twfe_parity.py`, `tests/test_methodology_stacked_did.py`, `tests/test_methodology_placebo.py` | SE-audit | Mid | Low |
| Clustered-CR1 inference **df convention**: Python uses the residual df (`n − K_full`, e.g. t(148) on the TWFE live-R panel) for clustered t-stats/p-values/CIs while `fixest`/Stata use the cluster df `G − 1` (t(49) same panel) — the common applied recommendation for few-cluster inference. Both are t-distributions; at large \|t\| tails diverge by orders of magnitude (documented as a REGISTRY Note (deviation from R), 2026-07-09). Decide: adopt cluster-df library-wide (moves EVERY clustered p-value/CI — needs a full-suite impact sweep + R parity regen) or keep residual-df as the documented convention. | `diff_diff/linalg.py::LinearRegression.get_inference` + `safe_inference` df-passing callers | SE-audit C7/C8 | Heavy | Medium |

---
Expand All @@ -64,6 +63,7 @@ Not currently actionable. Retained for provenance + AI-review deviation-document

| Issue | Location | PR | Priority |
|-------|----------|----|----------|
| `PlaceboTests` `boundary_gap` — a permutation randomization-inference margin (SE-audit item (b)); NOT computed anywhere in code today, so this is a new feature + result field, not a coverage lock. **User-locked 2026-07-09: defer until a derivation/paper source exists** — do not design or implement from scratch. | `tests/test_methodology_placebo.py`, `diff_diff/diagnostics.py` | SE-audit | Low |
| `SyntheticControl` fit-snapshot residency (`_SyntheticControlFitSnapshot`) — **investigated 2026-07-07, parked**: the snapshot ALIASES the fit's own working pivots (zero extra construction cost); the retained residency implements the documented freeze contract (post-fit mutation of estimator inputs must not change `in_space_placebo()` / `leave_one_out()` / conformal output on an already-returned results object, and `__getstate__` already excludes it from pickles). A compact array representation saves only pandas overhead (the float panel dominates); releasing residency needs new API surface (`release`/opt-out flag) or a freeze-contract change. Revisit on user demand for very large donor panels. | `synthetic_control.py`, `synthetic_control_results.py` | follow-up | Low |
| Stratified survey-PSU multiplier-weight draw-tiling — **investigated 2026-07-07, parked**: the stratified generator (`generate_survey_multiplier_weights_batch`) consumes ONE sequential rng stream stratum-major (`rng.choice(size=(n_bootstrap, n_h))` per stratum, then lonely-PSU pooling), so draw-chunked assembly CANNOT reproduce the stream bit-identically (contra the old row's parenthetical) — it would need per-stratum generator state skipping (PCG64.advance + per-weight-type variate accounting; fragile) or a stream-layout change (MC-level SE changes → baseline/golden recapture + REGISTRY note). Stratified designs have few PSUs, so the full `(n_bootstrap × n_psu)` matrix rarely matters; unstratified (the large-`n_units` case) is already tiled. Revisit only if a large-PSU stratified design hits memory, as a documented stream change. | `diff_diff/bootstrap_chunking.py::iter_survey_multiplier_weight_blocks` | follow-up | Low |
| CBWSDID covariate balancing (`StackedDiD(balance="entropy")`) v1 supports only balanced event windows + `weighting="aggregate"`; unbalanced/ragged panels fail closed (unit-count vs observation-count corrector convention unresolved off balanced panels). Matching-based balancing and the repeated `0→1`/`1→0` episode extension are also deferred. Documented in REGISTRY StackedDiD "Covariate balancing (CBWSDID)" Notes. | `stacked_did.py`, `balancing.py`, REGISTRY | follow-up | Low |
Expand Down Expand Up @@ -140,6 +140,8 @@ Doable in principle, but no current caller and/or explicitly out of paper scope.

| Decision | Location | Verified |
|----------|----------|----------|
| **StackedDiD intercept SEs not parity-lockable (SE-audit C1/(c)).** Measured ~0.3% divergence from R: a nuisance-parameter reference-cell/parameterization gap, not machine-precision lockable (the event-study interaction SEs already match ~2e-13). Surfacing `se_cr1/cr2_intercept` would add an unasserted, R-divergent public field. Waived with the SE-audit row closure. | `tests/test_methodology_stacked_did.py` | SE-audit / 2026-07-09 |
| **estimatr `classical` intercept SE excluded from parity (SE-audit (d)).** Same documented `O(1/n)` projection/DOF deviation as the slope; reference-only. Waived with the SE-audit row closure. | `tests/test_estimatr_iv_robust_parity.py` | SE-audit / 2026-07-09 |
| **`bread_inv` reuse not bit-identically achievable.** "Factor `(X'WX)` once, reuse across HC2/HC2-BM" can't be done bit-identically (the bar for a pure perf refactor of the inference path). Internal bread ops solve against *different* RHS (`X.T`, `eye`, `meat`+`temp.T`, `contrasts`); only same-RHS results are bit-reusable. Measured: `lu_solve(lu_factor(A),B)` differs from `solve(A,B)` up to 6.4e-15; the `inv(A)@meat@inv(A)` sandwich differs up to 1.24e-14 — both nonzero and *below* the affected goldens' tolerances (1e-12/1e-10), so a broad reuse would silently shift SEs without tripping the suite. The one genuine bit-identical redundancy (a duplicated `solve(bread, X.T)` in the unweighted one-way `hc2_bm`+`return_dof` path) is dwarfed by that path's dense `M=I−H` build, so the saving is negligible. | `linalg.py::compute_robust_vcov` | 2026-06-01 |
| **R-script-per-test consolidation has no CI impact.** No CI workflow installs R, so every R-parity test skips in CI behind a per-file availability gate — consolidating `Rscript` spawns yields zero CI speedup. `test_methodology_twfe.py` already session-caches its R fits. The only residual is a LOCAL-dev micro-opt for `test_methodology_continuous_did.py` / `test_methodology_callaway.py` (re-spawn `library(...)` per call). Low value; retained as a local-dev note. | `tests/test_methodology_continuous_did.py`, `tests/test_methodology_callaway.py` | #139 / 2026-06-07 |
| **`HeterogeneousAdoptionDiD` mass-point IV bread is non-symmetric — `_rank_guarded_inv` inapplicable.** The structural rank-guard sweep (continuous_did / two_stage / spillover / conley) excluded `had.py`'s `ZtWX = Zd'WX` ([1, instrument]' × [1, endogenous]): it is a non-symmetric 2×2 Wald-IV bread (`V = ZtWX_inv @ Omega @ ZtWX_inv.T`), and `_rank_guarded_inv` assumes a **symmetric PSD** Gram (symmetric `D=diag(A)` equilibration + eigendecomposition), so applying it would be methodologically wrong. The existing fallback already returns NaN SE on a singular bread; an IV-appropriate near-singular guard would need a different mechanism. | `had.py` | structural-rank-guard / 2026-06-28 |
Expand Down
Loading