diff --git a/TODO.md b/TODO.md index 3de64cce..9f1d4182 100644 --- a/TODO.md +++ b/TODO.md @@ -30,9 +30,7 @@ The `Origin` column (Actionable tables) and the `PR` column (Deferred tables) bo |-------|----------|--------|--------|----------| | `SyntheticControl` conformal (CWZ 2021) extensions: (a) one-sided / signed-`t` variants (§7); (b) covariates in the conformal proxy (`X_jt`, eqs 4/6 — current proxy is outcomes-only); (c) AR / innovation-permutation path (Lemmas 5-7) for time-series proxies. The joint test, pointwise CIs, and average-effect CI have landed. | `conformal.py`, `synthetic_control_results.py` | CWZ-2021 | Heavy | Low | | `ContinuousDiD` CGBS-2024 extensions. (a) `covariates=` kwarg — **DONE (reg/dr)**; (b) discrete-treatment saturated regression (`treatment_type="discrete"`) — **DONE**; (c) lowest-dose-as-control per Remark 3.1 when `P(D=0)=0` (`control_group="lowest_dose"`) — **DONE** (discrete + continuous mass-point, single-cohort; estimand `ATT(d)−ATT(d_L)`; see REGISTRY Note #7). Remaining (all deferred `NotImplementedError`, documented): `estimation_method="ipw"` on the dose curve (scalar-adjustment / degenerate); `covariates=` × `survey_design=` (weighted OR + weighted nuisance IF); multi-cohort **heterogeneous-support** discrete aggregation (support-aware: average each dose only over the cohorts that observe it); **multi-cohort `lowest_dose`** (within-cohort `d_L` reference + support-aware cross-cohort aggregation); and **`covariates=` × `lowest_dose`** (conditional-PT-relative-to-`d_L` estimand). Single-cohort / 2-period / shared-support multi-cohort are supported. | `continuous_did.py` | CGBS-2024 | Heavy | Low | -| **Rust-backend `solve_ols(return_vcov=True)` is run-to-run NONDETERMINISTIC** (~1e-14 rel): 8 identical clustered-vcov calls in one process produced 3 distinct values (observed 2026-07-07, Apple Silicon M4, `maturin --features accelerate`; the pure-Python backend is bit-stable run-to-run). Same-seed reproducibility is a documented library expectation, and per the Python-canonical convention the Rust port should at minimum be deterministic. Likely rayon/threaded-BLAS reduction ordering in the faer SVD solve or the vcov meat; investigate and either force a deterministic reduction path or document the wobble as a platform contract with an explicit note. Repro: `solve_ols(X(400x3), y, cluster_ids=10x40, return_vcov=True)` in a loop, compare `vcov[1,1]` bit patterns. | `rust/src/linalg.rs::solve_ols`, `diff_diff/linalg.py` | wild-bootstrap follow-up | Mid | Medium | | TWFE's HC2/HC2-BM inline full-dummy build (`twfe.py:280-315`) duplicates the dummy-construction logic in `DifferenceInDifferences(fixed_effects=...)` (`estimators.py:478-486`). Extract a shared helper, or delegate TWFE's HC2/HC2-BM path to DiD's `fixed_effects=` branch (with TWFE-specific cluster-default threading), to reduce drift risk on FE naming / survey behavior / result-surface conventions. Substantive refactor — touches both estimators. | `twfe.py::fit`, `estimators.py::DifferenceInDifferences.fit` | follow-up | Heavy | Low | -| `HonestDiD` Δ^SD optimal-FLCI center parity — **LANDED (SE-audit B2b)**: replaced the flat Nelder-Mead affine-estimator optimizer with a faithful port of R `HonestDiD::findOptimalFLCI`'s nested convex program (inner min-worst-case-bias at fixed estimator SD `h` via SLSQP QCQP; outer grid-zoom over `h`). Matches R's center + half-length + optimalVec to ~1e-3 (median ~1e-5) across a stress grid; the prior ~9% intermediate-M center drift is removed (widths always matched, coverage unaffected). Analytical folded-normal cv is more accurate than R's MC `.qfoldednormal`. Golden `honest_flci_golden.json` + `TestHonestFLCIParityR`. | `honest_did.py::_flci_solve` | SE-audit | Done | Low | ### Performance diff --git a/docs/methodology/REGISTRY.md b/docs/methodology/REGISTRY.md index 4470aa40..7109973b 100644 --- a/docs/methodology/REGISTRY.md +++ b/docs/methodology/REGISTRY.md @@ -79,6 +79,7 @@ where τ is the ATT. - Tolerance: `1e-07` (matches R's `qr()` default), relative to largest diagonal element of R in QR decomposition - Controllable via `rank_deficient_action` parameter: "warn" (default), "error", or "silent" - **Note (scale invariance):** the shared `diff_diff/linalg.py` rank check re-detects on column-equilibrated (unit 2-norm) columns when the raw pass reports a deficiency (adopting the higher equilibrated rank and its scale-corrected pivot selection only when a large-scale column inflated the threshold; genuine collinearity keeps the raw selection), and the least-squares solve equilibrates columns then unscales the coefficients — so rank detection and the fit are invariant to per-column scaling; for a well-scaled collinear design the dropped column is unchanged, while a scale-induced under-count adopts the scale-corrected equilibrated selection (which may differ from the raw choice but is guaranteed to retain an identified subset). This covers every covariate outcome-regression fit routed through `solve_ols` — DiD, TwoWayFixedEffects, MultiPeriodDiD, ImputationDiD, TwoStageDiD, TripleDifference, and (as of the OR scale-equilibration change) CallawaySantAnna's `_compute_all_att_gt_covariate_reg` / `_doubly_robust` and StaggeredTripleDifference's `_compute_or` — see the CallawaySantAnna rank-deficiency Note's scope statement. **Certification fast path (2026-07):** the common full-rank case is short-circuited by a Gram/eigenvalue certification on the equilibrated Gram at the documented `_rank_guarded_inv` 1e-10 Gram threshold — two orders stricter than the QR full-rank boundary, so it never reports full rank where the pivoted-QR path would report a deficiency; deficient or uncertifiable designs (including all n < k and non-finite inputs) fall through to the pivoted-QR path verbatim, leaving every drop decision, pivot selection, and rank count unchanged. Certification never SELECTS columns (it only confirms none need dropping), so the `_rank_guarded_inv` note about equilibrated-Gram column selection being reserved for the IF inverse still holds. + - **Note (run-to-run determinism of the clustered vcov, 2026-07):** the Rust backend's clustered `compute_robust_vcov` / `solve_ols(return_vcov=True)` previously aggregated cluster scores in `HashMap` iteration order — SipHash-randomized per call — so identical inputs produced ~1e-14-level distinct vcov values across calls (the Python backend was always bit-stable). Fixed in #653: rows accumulate in first-appearance order (ascending for the factorized 0..G-1 ids the dispatcher passes, matching NumPy's groupby order); repeated identical calls are bit-identical (a 50-call probe during the fix; the shipped regression asserts 20 repeats per id-layout), NumPy parity unchanged (~1e-14 cross-backend GEMM tolerance). Regression: `tests/test_rust_backend.py::TestClusterVcovDeterminism` (contiguous + non-contiguous unsorted ids, plus a raw-kernel path). - **Note (covariate-name collision guard):** `DifferenceInDifferences`, `MultiPeriodDiD`, and `TwoWayFixedEffects` build their `coefficients` dict by zipping a variable-name list (structural terms + user covariate column names, appended verbatim) with the coefficient vector. A covariate whose name equals a reserved structural name — `const`, the treatment/time column names, the `{treatment}:{time}` interaction (DiD), the `period_{p}` dummies / `{treatment}:period_{p}` interactions (MultiPeriodDiD), `ATT` (TwoWayFixedEffects), any fixed-effect / unit / time dummy name, or an internal `_`-prefixed working column (`_treat_time`, `_did_treatment`, `_treatment_post`) — would silently overwrite that structural coefficient (dict last-write-wins) with no error. `fit()` now calls `utils.validate_covariate_names()` before building the design, raising `ValueError` on such a collision (case-sensitive, so `Const` is allowed) and on duplicate covariate names. Reserved fixed-effect/unit/time dummy names are taken from the same `pd.get_dummies(..., drop_first=True)` call used to build them (exact match, including for `Categorical` columns). The TwoWayFixedEffects guard fires on both variance paths — the within-transform path exposes only `{"ATT": att}`, but a `_treatment_post`-named covariate would still clobber the internal interaction column. The influence-function estimators (CallawaySantAnna, etc.) key results by `(g, t)` and are unaffected. **Reference implementation(s):** diff --git a/tests/test_wild_bootstrap.py b/tests/test_wild_bootstrap.py index ba5851cd..7a5953a7 100644 --- a/tests/test_wild_bootstrap.py +++ b/tests/test_wild_bootstrap.py @@ -1507,11 +1507,12 @@ def test_multi_chunk_count_invariant(self, monkeypatch): many = du.wild_bootstrap_se(X, y, resid, cl, 1, n_bootstrap=499, seed=11) # p-value: strict-count statistic with a 1e-9 relative tie guard — - # exact equality expected. se / CI endpoints tolerate the ambient - # Rust-backend run-to-run vcov wobble (~1e-14 rel; see the TODO row - # on rust solve_ols nondeterminism — the pure-Python backend is - # bit-stable and the chunked precompute itself is deterministic - # numpy). + # exact equality expected. se / CI endpoints keep small tolerances + # for historical cross-run robustness: the Rust clustered-vcov + # run-to-run wobble was fixed in #653 (first-appearance cluster + # aggregation; see the REGISTRY determinism note and + # tests/test_rust_backend.py::TestClusterVcovDeterminism) — the + # chunked precompute itself is deterministic numpy. assert many.p_value == one.p_value np.testing.assert_allclose(many.se, one.se, rtol=1e-12) np.testing.assert_allclose(many.ci_lower, one.ci_lower, rtol=1e-6)