feat(synthetic_control): ADH-2015 §4 tail diagnostics (regression_weights + sparse_synthetic_control)

CHANGELOG.md

@@ -8,6 +8,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Added
+- **`SyntheticControl` ADH-2015 §4 tail diagnostics** (two opt-in `SyntheticControlResults`
+  methods, closing the last two ADH-2015 §4 checklist items). `regression_weights()` reports the
+  implied donor weights `W^reg = X0a'(X0a X0a')^{-1} X1a` of the regression counterfactual
+  (intercept-augmented so they sum to 1 at full row rank) and flags donors outside `[0, 1]` — the
+  extrapolation an OLS counterfactual incurs but the simplex-constrained synthetic control cannot;
+  pure linear algebra, min-norm least-squares with a rank-deficient warning. `sparse_synthetic_control()`
+  exhaustively searches `C(J, l)` size-`l` donor subsets holding `V` fixed at the baseline fit,
+  reporting how fit / ATT degrade as the synthetic is forced sparse (per-size winner table +
+  `get_sparse_synthetic_control_gaps()` overlay); the default `sizes=[1,2,3]` sweep skips over-cap
+  sizes with a warning while an explicitly requested oversized `l` raises (`max_subsets` guard). Both
+  are purely additive, surface under `estimator_native_diagnostics`, and leave the analytical
+  inference contract unchanged (`se`/`t_stat`/`p_value`/`conf_int` stay NaN). No behavior change to
+  any existing fit.
 - **Rust `demean_map` kernel for FE absorption** (optional backend acceleration). When the Rust
   extension is available, the method-of-alternating-projections sweeps in
   `demean_by_groups`/`within_transform` run in a compiled kernel, rayon-parallel across the

TODO.md

@@ -28,7 +28,6 @@ The `Origin` column (Actionable tables) and the `PR` column (Deferred tables) bo
 
 | Issue | Location | Origin | Effort | Priority |
 |-------|----------|--------|--------|----------|
-| `SyntheticControl` remaining ADH-2015 §4 items: the regression-weight `W^reg = X_0'(X_0 X_0')^{-1} X_1` extrapolation diagnostic (flag implied OLS weights outside `[0,1]`) and sparse-SC subset search (`l < J`, holding `V` fixed). LOO, in-time placebo, CV `V`-selection, and inverse-variance `V` have landed; these two are the deferred tail. | `synthetic_control.py`, `synthetic_control_results.py` | ADH-2015 | Mid | Low |
 | `SyntheticControl` conformal (CWZ 2021) extensions: (a) one-sided / signed-`t` variants (§7); (b) covariates in the conformal proxy (`X_jt`, eqs 4/6 — current proxy is outcomes-only); (c) AR / innovation-permutation path (Lemmas 5-7) for time-series proxies. The joint test, pointwise CIs, and average-effect CI have landed. | `conformal.py`, `synthetic_control_results.py` | CWZ-2021 | Heavy | Low |
 | `ContinuousDiD` CGBS-2024 extensions (matches R `contdid` v0.1.0 deferral set): (a) `covariates=` kwarg; (b) discrete-treatment saturated regression (integer dose currently warned, not routed to per-level coefficients); (c) lowest-dose-as-control per Remark 3.1 when `P(D=0)=0`. REGISTRY `## ContinuousDiD` → Implementation Checklist marks these `[ ]`. | `continuous_did.py` | CGBS-2024 | Heavy | Low |
 | `ImputationDiD` LOO conservative-variance refinement (BJS 2024 Supp. Appendix A.9) — a finite-sample improvement to the auxiliary-model residuals reducing overfit of `tau_tilde_g` to `epsilon`. Asymptotic Theorem-3 variance is implemented and matches R `didimputation` (which also omits LOO by default). | `imputation.py` | imputation-validation | Mid | Low |

diff_diff/diagnostic_report.py

@@ -2532,6 +2532,107 @@ def _scm_native(self, r: Any) -> Dict[str, Any]:
                 ),
             }
 
+        # Regression-weight extrapolation diagnostic (ADH 2015 §4): opt-in, surfaced once
+        # the user has run results.regression_weights() (pure linear algebra — no refits).
+        if getattr(r, "_regw_df", None) is not None:
+            regw_status = getattr(r, "_regw_status", None)
+            if regw_status == "ran":
+                out["regression_weights"] = {
+                    "status": "ran",
+                    # Headline: how many donors an OLS/regression counterfactual would push
+                    # outside [0,1] (extrapolate). The SC simplex never does.
+                    "n_extrapolating": _to_python_scalar(getattr(r, "_regw_n_extrapolating", None)),
+                    # True if W^reg is a non-unique min-norm solution (not full row rank);
+                    # weight_sum then need not equal 1.
+                    "rank_deficient": bool(getattr(r, "_regw_rank_deficient", False)),
+                    "weight_sum": _to_python_float(getattr(r, "_regw_weight_sum", None)),
+                }
+            else:
+                _regw_reasons = {
+                    "treated_fit_nonconverged": (
+                        "regression_weights() was run but the treated unit's own SCM fit "
+                        "did not converge at fit time, so the synthetic control it is "
+                        "compared against is not a valid optimum."
+                    ),
+                    "too_few_donors": (
+                        "regression_weights() was run but fewer than 2 donors are available "
+                        "(W^reg is trivially [1] with a single donor)."
+                    ),
+                }
+                out["regression_weights"] = {
+                    "status": "infeasible",
+                    "reason_code": regw_status,
+                    "reason": _regw_reasons.get(
+                        regw_status, "regression_weights() produced no table."
+                    ),
+                }
+        else:
+            out["regression_weights"] = {
+                "status": "not_run",
+                "reason": (
+                    "Call results.regression_weights() to flag donors an OLS/regression "
+                    "counterfactual would weight outside [0,1] (ADH 2015 §4 extrapolation "
+                    "diagnostic; opt-in, no refit)."
+                ),
+            }
+
+        # Sparse-SC subset search (ADH 2015 §4): opt-in, surfaced once the user has run
+        # results.sparse_synthetic_control() (exhaustive subset search, V held fixed).
+        if getattr(r, "_sparse_df", None) is not None:
+            sparse_status = getattr(r, "_sparse_status", None)
+            if sparse_status == "ran":
+                # Compact per-size summary: how far the ATT / fit move as the synthetic is
+                # forced sparse (the baseline row is dropped — it is the full-fit reference).
+                records = r._sparse_df.to_dict("records")
+                per_size = [
+                    {
+                        "size": _to_python_scalar(rec["size"]),
+                        "att": _to_python_float(rec["att"]),
+                        "delta_att": _to_python_float(rec["delta_att"]),
+                        "pre_rmspe": _to_python_float(rec["pre_rmspe"]),
+                        "n_failed": _to_python_scalar(rec["n_failed"]),
+                        "status": rec["status"],
+                    }
+                    for rec in records
+                    if rec["status"] != "baseline"
+                ]
+                out["sparse_synthetic_control"] = {
+                    "status": "ran",
+                    # Headline: the largest baseline-relative ATT swing across searched sizes.
+                    "max_abs_delta_att": _to_python_float(
+                        getattr(r, "_sparse_max_abs_delta_att", None)
+                    ),
+                    "sizes": per_size,
+                }
+            else:
+                _sparse_reasons = {
+                    "treated_fit_nonconverged": (
+                        "sparse_synthetic_control() was run but the treated unit's own SCM "
+                        "fit did not converge at fit time, so the baseline ATT is not a "
+                        "valid reference for the sparse deltas."
+                    ),
+                    "too_few_donors": (
+                        "sparse_synthetic_control() was run but fewer than 2 donors are "
+                        "available (a sparse subset must be smaller than the pool)."
+                    ),
+                }
+                out["sparse_synthetic_control"] = {
+                    "status": "infeasible",
+                    "reason_code": sparse_status,
+                    "reason": _sparse_reasons.get(
+                        sparse_status, "sparse_synthetic_control() produced no valid subsets."
+                    ),
+                }
+        else:
+            out["sparse_synthetic_control"] = {
+                "status": "not_run",
+                "reason": (
+                    "Call results.sparse_synthetic_control() to test how the fit / ATT "
+                    "degrade when the synthetic is forced to a few donors (ADH 2015 §4 "
+                    "sparse subset search; opt-in, V held fixed)."
+                ),
+            }
+
         # Test-inversion confidence set (Firpo & Possebom 2018 §4): opt-in, surfaced once
         # the user has run results.confidence_set() (it reuses the in-space placebo
         # reference set — no refits). The analytical conf_int stays NaN; this is a SEPARATE

diff_diff/guides/llms-full.txt

@@ -617,7 +617,7 @@ scm.fit(
 ) -> SyntheticControlResults
 ```
 
-**Inference:** NONE analytical — `se`/`t_stat`/`p_value`/`conf_int` are always NaN. `att` is the mean post-period gap. Significance via in-space placebo permutation inference: `results.in_space_placebo()` reassigns treatment to each donor, refits against the other J-1 donors (the real treated unit is excluded from every placebo pool), and sets `placebo_p_value = rank/(n_placebos+1)` from the post/pre RMSPE-ratio. The permutation `placebo_p_value` is a SEPARATE field from the (NaN) `p_value`; `is_significant` stays bound to `p_value`. **ADH-2015 §4 robustness (opt-in, analytical inference unchanged):** `results.leave_one_out()` drops each reportably-weighted donor (weight > 1e-6) and re-fits (per-drop ATT/`delta_att` — large `delta_att` ⇒ single-donor dependence); `results.in_time_placebo()` backdates the intervention and checks for a spurious pre-period gap (TRUNCATE windowing — predictor windows in the held-out region are dropped). **Confidence sets by test inversion (Firpo-Possebom 2018 §4, opt-in, also non-analytical):** `results.test_sharp_null(effect, gamma=0.1)` tests `H_0: α_1t = f(t)` by re-ranking the in-space placebo gaps (no refits; `test_sharp_null(0)` == `placebo_p_value`), and `results.confidence_set(family="constant"|"linear", gamma=0.1)` inverts it into a confidence set for the effect path (constant-effect interval / linear-slope set, strict `p>gamma`), surfaced on `effect_confidence_set` / `get_confidence_set_df()` — the analytical `conf_int` still stays NaN. **Conformal inference (Chernozhukov-Wüthrich-Zhu 2021, opt-in, also non-analytical):** unlike the Firpo path (which re-ranks the cross-unit placebo gaps), the conformal layer fits its OWN time-permutation-invariant constrained-LS proxy (eqs 3–4, no V-matrix) under the null on ALL periods and permutes residuals over time for the single treated unit. `results.conformal_test(effect, q=1, scheme="moving_block")` gives a joint sharp-null p-value (eqs 1–2; `q∈{1,2,∞}`); `results.conformal_confidence_intervals(alpha=0.1)` gives pointwise per-period CIs (Algorithm 1 — each period `t` uses `Z=(pre, t)`, the other post-periods dropped); `results.conformal_average_effect(alpha=0.1)` gives an average-effect CI by collapsing into `T*`-blocks (Appendix A.1). `scheme="moving_block"` (default, serial-dependence-robust) or `"iid"` (finer). Surfaced on `conformal_inference` / `get_conformal_grid_df()`; `conf_int` stays NaN. Predictor periods must lie within the pre window; `post_periods` must be a contiguous suffix cross-checked against `D` (no anticipation).
+**Inference:** NONE analytical — `se`/`t_stat`/`p_value`/`conf_int` are always NaN. `att` is the mean post-period gap. Significance via in-space placebo permutation inference: `results.in_space_placebo()` reassigns treatment to each donor, refits against the other J-1 donors (the real treated unit is excluded from every placebo pool), and sets `placebo_p_value = rank/(n_placebos+1)` from the post/pre RMSPE-ratio. The permutation `placebo_p_value` is a SEPARATE field from the (NaN) `p_value`; `is_significant` stays bound to `p_value`. **ADH-2015 §4 robustness (opt-in, analytical inference unchanged):** `results.leave_one_out()` drops each reportably-weighted donor (weight > 1e-6) and re-fits (per-drop ATT/`delta_att` — large `delta_att` ⇒ single-donor dependence); `results.in_time_placebo()` backdates the intervention and checks for a spurious pre-period gap (TRUNCATE windowing — predictor windows in the held-out region are dropped); `results.regression_weights()` computes the implied regression-counterfactual weights `W^reg = X0a'(X0a X0a')^{-1}X1a` (intercept-augmented, sums to 1 at full row rank) and flags donors outside `[0,1]` — the extrapolation an OLS counterfactual incurs but the simplex-constrained SC cannot (pure linear algebra, no refit; min-norm + `_regw_rank_deficient` when `k+1>J`); `results.sparse_synthetic_control(sizes=None, max_subsets=50000)` exhaustively searches `C(J,l)` size-`l` donor subsets holding `V` FIXED at the baseline (default sweep `[1,2,3]` skips over-cap sizes with a warning; an explicitly-requested oversize `l` raises), reporting how fit/ATT degrade as the synthetic is forced sparse (+ `get_sparse_synthetic_control_gaps()`). **Confidence sets by test inversion (Firpo-Possebom 2018 §4, opt-in, also non-analytical):** `results.test_sharp_null(effect, gamma=0.1)` tests `H_0: α_1t = f(t)` by re-ranking the in-space placebo gaps (no refits; `test_sharp_null(0)` == `placebo_p_value`), and `results.confidence_set(family="constant"|"linear", gamma=0.1)` inverts it into a confidence set for the effect path (constant-effect interval / linear-slope set, strict `p>gamma`), surfaced on `effect_confidence_set` / `get_confidence_set_df()` — the analytical `conf_int` still stays NaN. **Conformal inference (Chernozhukov-Wüthrich-Zhu 2021, opt-in, also non-analytical):** unlike the Firpo path (which re-ranks the cross-unit placebo gaps), the conformal layer fits its OWN time-permutation-invariant constrained-LS proxy (eqs 3–4, no V-matrix) under the null on ALL periods and permutes residuals over time for the single treated unit. `results.conformal_test(effect, q=1, scheme="moving_block")` gives a joint sharp-null p-value (eqs 1–2; `q∈{1,2,∞}`); `results.conformal_confidence_intervals(alpha=0.1)` gives pointwise per-period CIs (Algorithm 1 — each period `t` uses `Z=(pre, t)`, the other post-periods dropped); `results.conformal_average_effect(alpha=0.1)` gives an average-effect CI by collapsing into `T*`-blocks (Appendix A.1). `scheme="moving_block"` (default, serial-dependence-robust) or `"iid"` (finer). Surfaced on `conformal_inference` / `get_conformal_grid_df()`; `conf_int` stays NaN. Predictor periods must lie within the pre window; `post_periods` must be a contiguous suffix cross-checked against `D` (no anticipation).
 
 **Usage:**
 
@@ -1360,7 +1360,7 @@ Returned by `SyntheticControl.fit()`.
 | `effect_confidence_set` | `dict \| None` | Test-inversion confidence-set summary (Firpo-Possebom 2018 §4): `{family, parameter, gamma, lower, upper, contiguous, status ("ran"/"empty"/"unbounded"), ...}`; None until `confidence_set()` runs. SEPARATE from the always-NaN analytical `conf_int` (a permutation set at level 1−gamma, possibly a set/unbounded). |
 | `conformal_inference` | `dict \| None` | CWZ (2021) conformal-inference summary of the most recent run: `{kind ("joint"/"pointwise"/"average"), scheme, status, n_perms, ...}` (joint adds `joint_p_value`/`proxy_converged`; average/pointwise add `alpha` + CI fields); None until a `conformal_*()` method runs. SEPARATE from the always-NaN analytical `conf_int`. |
 
-**Methods:** `in_space_placebo()` (opt-in permutation inference; refits one synthetic control per donor), `get_placebo_df()` (per-unit RMSPE-ratio table incl. the treated row), `leave_one_out()` (ADH-2015 §4 donor robustness; drops each reportably-weighted donor (weight > 1e-6) → per-drop ATT/`delta_att` table) + `get_leave_one_out_df()`/`get_leave_one_out_gaps()`, `in_time_placebo()` (ADH-2015 §4 backdating placebo; reassigns the intervention earlier, TRUNCATE windowing, placebo ATT ~0 if no real pre-effect) + `get_in_time_placebo_df()`/`get_in_time_placebo_gaps()`, `test_sharp_null(effect, gamma=0.1)` (Firpo-Possebom 2018 §4: test a sharp null α_1t=f(t) by re-ranking the in-space placebo gaps — `effect` is a scalar or a post-period array; `test_sharp_null(0)` is identically `placebo_p_value`), `confidence_set(family="constant"|"linear", gamma=0.1, bounds=None, n_grid=200)` (invert that test for a confidence set of the effect path — a constant-effect interval / linear-slope set; strict p>gamma membership; exact piecewise-constant breakpoint inversion when bounds=None, else a fixed grid; `conf_int` stays NaN) + `get_confidence_set_df()`, `conformal_test(effect, q=1, scheme="moving_block", n_iid=10000, seed=None)` (CWZ 2021 joint sharp-null conformal p-value — fits its own constrained-LS proxy under the null on all periods, permutes residuals over time; `q∈{1,2,∞}`), `conformal_confidence_intervals(alpha=0.1, scheme="moving_block", bounds=None, n_grid=100, seed=None)` (pointwise per-period CIs, Algorithm 1 — each period uses Z=(pre, t)), `conformal_average_effect(alpha=0.1, scheme="moving_block", bounds=None, n_grid=200, seed=None)` (average-effect CI by T*-block collapse, Appendix A.1) + `get_conformal_grid_df()`, `summary()`, `print_summary()`, `to_dict()`, `to_dataframe()`, `get_gap_df()`, `get_weights_df()`
+**Methods:** `in_space_placebo()` (opt-in permutation inference; refits one synthetic control per donor), `get_placebo_df()` (per-unit RMSPE-ratio table incl. the treated row), `leave_one_out()` (ADH-2015 §4 donor robustness; drops each reportably-weighted donor (weight > 1e-6) → per-drop ATT/`delta_att` table) + `get_leave_one_out_df()`/`get_leave_one_out_gaps()`, `in_time_placebo()` (ADH-2015 §4 backdating placebo; reassigns the intervention earlier, TRUNCATE windowing, placebo ATT ~0 if no real pre-effect) + `get_in_time_placebo_df()`/`get_in_time_placebo_gaps()`, `regression_weights()` (ADH-2015 §4 regression-weight extrapolation diagnostic; intercept-augmented `W^reg`, flags donors outside [0,1]; pure linear algebra) + `get_regression_weights_df()`, `sparse_synthetic_control(sizes=None, max_subsets=50000)` (ADH-2015 §4 sparse subset search; exhaustive `C(J,l)` subsets holding V fixed; default-skip vs explicit-raise cap) + `get_sparse_synthetic_control_df()`/`get_sparse_synthetic_control_gaps()`, `test_sharp_null(effect, gamma=0.1)` (Firpo-Possebom 2018 §4: test a sharp null α_1t=f(t) by re-ranking the in-space placebo gaps — `effect` is a scalar or a post-period array; `test_sharp_null(0)` is identically `placebo_p_value`), `confidence_set(family="constant"|"linear", gamma=0.1, bounds=None, n_grid=200)` (invert that test for a confidence set of the effect path — a constant-effect interval / linear-slope set; strict p>gamma membership; exact piecewise-constant breakpoint inversion when bounds=None, else a fixed grid; `conf_int` stays NaN) + `get_confidence_set_df()`, `conformal_test(effect, q=1, scheme="moving_block", n_iid=10000, seed=None)` (CWZ 2021 joint sharp-null conformal p-value — fits its own constrained-LS proxy under the null on all periods, permutes residuals over time; `q∈{1,2,∞}`), `conformal_confidence_intervals(alpha=0.1, scheme="moving_block", bounds=None, n_grid=100, seed=None)` (pointwise per-period CIs, Algorithm 1 — each period uses Z=(pre, t)), `conformal_average_effect(alpha=0.1, scheme="moving_block", bounds=None, n_grid=200, seed=None)` (average-effect CI by T*-block collapse, Appendix A.1) + `get_conformal_grid_df()`, `summary()`, `print_summary()`, `to_dict()`, `to_dataframe()`, `get_gap_df()`, `get_weights_df()`
 
 ### TripleDifferenceResults