Skip to content

Commit 60abfb0

Browse files
igerberclaude
andauthored
feat(diagnostic_report): register SpilloverDiDResults routing (#599)
Route DiagnosticReport(spillover_result) to parallel_trends (event-study on the per-event-time direct-effect dynamics), design_effect (survey fits), and heterogeneity by adding SpilloverDiDResults to _APPLICABILITY / _PT_METHOD. Add the coupled describe_target_parameter branch (aggregation="spillover", headline att). Goodman-Bacon is deliberately excluded: SpilloverDiD identifies the direct effect off far-away control units (Butts Assumption 5), not off the TWFE 2x2 comparisons a Bacon decomposition enumerates. Aggregate-only fits skip parallel_trends with an estimator-accurate remediation (event_study=True constructor kwarg, not aggregate='event_study'). Docs (REGISTRY/REPORTING/spillover.rst/llms-full/CHANGELOG) and TODO updated; 12 new tests (DR routing across event-study/survey/aggregate-only paths, BR smoke, target-parameter branch). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 6518d05 commit 60abfb0

11 files changed

Lines changed: 206 additions & 15 deletions

File tree

CHANGELOG.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
3434
still defers cluster with a warning).
3535

3636
### Changed
37+
- **`DiagnosticReport` now routes `SpilloverDiDResults`.** Previously a fitted
38+
`SpilloverDiD` (Butts 2021) result matched no `_APPLICABILITY` / `_PT_METHOD` entry, so
39+
`DiagnosticReport(spillover_result)` reported every check as `not_applicable`. It now routes
40+
to parallel-trends (event-study joint test on the per-event-time direct-effect dynamics,
41+
populated when `event_study=True`), design-effect (survey-weighted fits), and heterogeneity;
42+
Goodman-Bacon is intentionally excluded because SpilloverDiD identifies off far-away control
43+
units rather than TWFE 2×2 comparisons. Aggregate-only fits now skip parallel-trends with an
44+
estimator-accurate remediation (`SpilloverDiD(..., event_study=True)`). `BusinessReport` and the
45+
`describe_target_parameter` block pick up the routing automatically.
3746
- **`SyntheticControl` in-space / leave-one-out placebo diagnostics now distinguish structural `cv`
3847
infeasibility from solver non-convergence.** Under `v_method="cv"`, an excluded `in_space_placebo()`
3948
/ `leave_one_out()` refit whose pseudo-treated (in-space) or reduced (leave-one-out) donor pool is

TODO.md

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,6 @@ The `Origin` column (Actionable tables) and the `PR` column (Deferred tables) bo
3737
| TWFE's HC2/HC2-BM inline full-dummy build (`twfe.py:280-315`) duplicates the dummy-construction logic in `DifferenceInDifferences(fixed_effects=...)` (`estimators.py:478-486`). Extract a shared helper, or delegate TWFE's HC2/HC2-BM path to DiD's `fixed_effects=` branch (with TWFE-specific cluster-default threading), to reduce drift risk on FE naming / survey behavior / result-surface conventions. Substantive refactor — touches both estimators. | `twfe.py::fit`, `estimators.py::DifferenceInDifferences.fit` | follow-up | Heavy | Low |
3838
| Decide whether to formally deprecate `CallawaySantAnna.cluster=X` in favor of `survey_design=SurveyDesign(psu=X)` (the bare-cluster path already synthesizes a minimal SurveyDesign). Two equivalent paths = redundant surface. Mirrors the question for ImputationDiD / EfficientDiD / TwoStageDiD. | `staggered.py` | follow-up | Mid | Low |
3939
| `HeterogeneousAdoptionDiD` **event-study (Phase 2b)** continuous cluster= threading: Phase 2a static path now threads `cluster=` into `bias_corrected_local_linear` (cluster-robust CCT SE, unweighted + weighted). The per-horizon event-study path still ignores `cluster=` with a `UserWarning` because the `cband` sup-t bootstrap normalizes HC-scale perturbations by the analytical SE and would mix variance families under clustering (mirrors the mass-point `weights= + cluster= + cband=True` `NotImplementedError`). Needs a per-horizon clustered-bootstrap variance-family reconciliation. | `had.py::_fit_event_study` | Phase 2b | Mid | Low |
40-
| `SpilloverDiDResults` not registered in `DiagnosticReport`'s `_APPLICABILITY` / `_PT_METHOD` tables, so `DiagnosticReport(spillover_result)` doesn't route to event-study diagnostics. Decide which diagnostics apply (PT, pre-trends power, heterogeneity, design-effect) and add an end-to-end test. | `diagnostic_report.py` | Wave C | Mid | Low |
4140

4241
### Performance
4342

diff_diff/_reporting_helpers.py

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -635,6 +635,26 @@ def describe_target_parameter(results: Any) -> Dict[str, Any]:
635635
"reference": "REGISTRY.md Sec. SyntheticControl",
636636
}
637637

638+
if name == "SpilloverDiDResults":
639+
return {
640+
"name": "total effect on the treated (Butts spillover-aware ATT)",
641+
"definition": (
642+
"The total effect on the treated ``tau_total`` from Butts (2021) "
643+
"ring-indicator spillover DiD, identified off FAR-AWAY control "
644+
"observations (``d_it > d_bar``, Assumption 5) rather than any "
645+
"not-yet-/never-treated pool. The estimator decomposes into the "
646+
"DIRECT effect on treated units plus per-ring spillover-on-control "
647+
"effects that relax SUTVA within the treated units' spatial "
648+
"neighborhood; ``att`` is the headline total effect, while the "
649+
"per-ring ``spillover_effects`` and (when ``event_study=True``) the "
650+
"per-event-time direct dynamics are available on the result object "
651+
"for disaggregated inference."
652+
),
653+
"aggregation": "spillover",
654+
"headline_attribute": "att",
655+
"reference": "Butts (2021); REGISTRY.md Sec. SpilloverDiD",
656+
}
657+
638658
# Default: unrecognized result class. Fall through with a neutral
639659
# block — agents / downstream consumers can still dispatch on
640660
# ``aggregation="unknown"`` and fall back to generic ATT narration.

diff_diff/diagnostic_report.py

Lines changed: 39 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -65,16 +65,16 @@
6565
"placebo",
6666
)
6767

68-
# Type-level applicability: which checks are *ever* applicable for each of the
69-
# 16 result types. Instance-level applicability further filters by whether
68+
# Type-level applicability: which checks are *ever* applicable for each
69+
# registered result type. Instance-level applicability further filters by whether
7070
# required attributes are present (e.g. ``survey_metadata`` for DEFF) and by
7171
# whether the user disabled a check via ``run_*=False``.
7272
# See ``docs/methodology/REPORTING.md`` for the full matrix and rationale.
7373
#
7474
# Implementation note: The keys are result-class names looked up via
7575
# ``type(results).__name__``. This string-based dispatch mirrors the
7676
# ``_HANDLERS`` pattern in ``diff_diff/practitioner.py`` and avoids circular
77-
# imports across the 16 result modules. Renaming or aliasing any result class
77+
# imports across the result modules. Renaming or aliasing any result class
7878
# requires updating both this table and ``_PT_METHOD`` below; the
7979
# applicability-matrix test parametrized over all result types serves as the
8080
# regression guard.
@@ -131,6 +131,27 @@
131131
"heterogeneity",
132132
}
133133
),
134+
"SpilloverDiDResults": frozenset(
135+
# Butts (2021) ring-indicator spillover DiD is a two-stage-GMM
136+
# estimator, so it inherits TwoStage's diagnostic set MINUS
137+
# ``bacon``. ``bacon`` is excluded because SpilloverDiD identifies
138+
# the direct effect off FAR-AWAY units (Butts Assumption 5), not
139+
# off the TWFE 2x2 comparisons a Goodman-Bacon decomposition
140+
# enumerates: ``bacon_decompose`` on the raw binary treatment
141+
# ignores the ring/distance structure and would pool spillover-
142+
# contaminated in-ring units into the control group — the exact
143+
# SUTVA violation the estimator exists to handle (same rationale
144+
# that excludes bacon for SyntheticControl / TROP / Continuous).
145+
# ``parallel_trends`` routes to ``event_study`` on the per-event-
146+
# time DIRECT-effect dynamics (populated when ``event_study=True``);
147+
# ``design_effect`` is instance-gated on ``survey_metadata`` (Wave
148+
# E.1); ``heterogeneity`` reads ``event_study_effects``.
149+
{
150+
"parallel_trends",
151+
"design_effect",
152+
"heterogeneity",
153+
}
154+
),
134155
"StackedDiDResults": frozenset(
135156
{
136157
"parallel_trends",
@@ -218,6 +239,7 @@
218239
"SunAbrahamResults": "event_study",
219240
"ImputationDiDResults": "event_study",
220241
"TwoStageDiDResults": "event_study",
242+
"SpilloverDiDResults": "event_study",
221243
"StackedDiDResults": "event_study",
222244
"EfficientDiDResults": "hausman",
223245
"ContinuousDiDResults": "event_study",
@@ -263,7 +285,7 @@ class DiagnosticReport:
263285
----------
264286
results : Any
265287
A fitted diff-diff results object (e.g. ``CallawaySantAnnaResults``,
266-
``DiDResults``, ``SyntheticDiDResults``). Any of the 16 result types
288+
``DiDResults``, ``SyntheticDiDResults``). Any registered result type
267289
in the library is accepted.
268290
data : pandas.DataFrame, optional
269291
The underlying panel. Required for checks that need raw data
@@ -703,6 +725,19 @@ def _instance_skip_reason(self, check: str) -> Optional[str]:
703725
# summary emits the "inconclusive" identifying-
704726
# assumption warning rather than silently dropping PT.
705727
if not pre_coefs and n_dropped_undefined == 0:
728+
# SpilloverDiD's event-study switch is the
729+
# ``SpilloverDiD(..., event_study=True)`` constructor
730+
# kwarg, not the ``aggregate='event_study'`` argument
731+
# the generic staggered-estimator message points at
732+
# (SpilloverDiD has no ``aggregate`` kwarg). Emit an
733+
# estimator-accurate remediation for this family.
734+
if name == "SpilloverDiDResults":
735+
return (
736+
"No pre-period event-study coefficients are exposed "
737+
"on this fit. Re-fit with "
738+
"SpilloverDiD(..., event_study=True) to populate the "
739+
"per-event-time direct-effect output."
740+
)
706741
return (
707742
"No pre-period event-study coefficients are exposed on "
708743
"this fit. For staggered estimators, re-fit with "

diff_diff/guides/llms-full.txt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -514,7 +514,7 @@ sp.fit(
514514
- `covariates=` raises `NotImplementedError` (planned follow-up). Gardner two-stage requires covariate effects estimated on the untreated-and-unexposed Omega_0 subsample at stage 1; appending raw covariates only at stage 2 silently biases `tau_total` / `delta_j` on panels with time-varying covariates.
515515
- `survey_design=` is **supported** (Wave E): design-based variance via Binder (1983) Taylor-series linearization for `vcov_type="hc1"` (HC1) and `cluster=<col>` (CR1), and for `vcov_type="conley"` via a panel-aware stratified-Conley sandwich on per-period PSU totals (`conley_lag_cutoff > 0` adds a within-PSU serial Bartlett HAC and requires an effective PSU — supply `survey_design.psu` or inject one via `cluster=<col>`). Accepts `pweight` with strata / PSU / FPC. Replicate-weight variance (BRR / Fay / JK1 / JKn / SDR) is NOT yet supported (`NotImplementedError`): per Gerber (2026) Appendix A the IF-reweighting shortcut does not apply because `gamma_hat` is weight-sensitive, so correct support requires a per-replicate full re-fit.
516516
- `vcov_type="classical"` raises `NotImplementedError` (Wave D restriction). Wave D GMM first-stage correction has not been derived for the homoskedastic meat structure `sigma_hat^2 * (X_10' X_10)`. Use `vcov_type="hc1"`, `vcov_type="conley"`, or pair with `cluster=<col>` for CR1 — all three apply the Wave D GMM correction.
517-
- `event_study=True` SHIPPED (Wave C): emits per-event-time `tau_k` and per-(ring, event-time) `delta_jk` as `att_dynamic: pd.DataFrame` (indexed by event-time `k`) plus MultiIndex `spillover_effects: pd.DataFrame` (levels `(ring_label, event_time)`). TwoStageDiD-compatible `event_study_effects: Dict[int, Dict]` alias also emitted for `plot_event_study` consumption — `_extract_plot_data` prefers the new `reference_period` attribute over the legacy `n_obs==0` heuristic. (DiagnosticReport integration: NOT yet wired; queued as a follow-up.) (schema: `{k: {"effect", "se", "n_obs", "t_stat", "p_value", "conf_int": (low, high)}}` mirroring `two_stage.py:1355-1389`). Reference period `ref_period = -1 - anticipation` (TwoStageDiD `two_stage.py:486` convention); reference row uses `coef=0.0, se=0.0, n_obs=0, conf_int=(0.0, 0.0)`. Scalar `att` field becomes a sample-share-weighted average of post-treatment `tau_k` (`att = sum_{k>=0} w_k * tau_k` with `w_k = n_treated_at_k / total`) with SE from linear-combination inference `Var(att) = w' V_subset w` on the post-treatment vcov block — no separate fit. **Two-clock K_it:** direct-effect clock is `K_direct = t - effective_first_treat(i)` for ever-treated rows; spillover clock is `K_spill = t - earliest-in-range-cohort-onset(i)` (running min across activated cohorts, NaN pre-trigger). `K_spill >= 0` structurally; negative-k spillover cells are rectangularly emitted with `coef = NaN, n_obs = 0`. **`horizon_max` semantics:** bins event-times outside `[-H, +H]` into endpoint pools (no observations dropped — divergence from TwoStageDiD which filters; intentional, per `feedback_no_silent_failures`). With `horizon_max=None`, auto-detects bin set from observed K. **Validation:** `horizon_max < 0` raises `ValueError`; `ref_period < -horizon_max` (i.e., `anticipation > horizon_max - 1`) raises `ValueError` — silently floor-shifting the reference would change identification. **Reduce-to-aggregate:** under constant-tau DGP with `horizon_max=None`, the share-weighted scalar `att` reproduces Wave B's aggregate bit-identically. **Note:** `horizon_max=0` does NOT reduce to Wave B (binning collapses pre-treatment K values to `k=0`, making `D^0 = D_i` ever-treated indicator rather than `D_it`). Per-event-time SEs include the Wave D Gardner GMM first-stage correction (see next bullet).
517+
- `event_study=True` SHIPPED (Wave C): emits per-event-time `tau_k` and per-(ring, event-time) `delta_jk` as `att_dynamic: pd.DataFrame` (indexed by event-time `k`) plus MultiIndex `spillover_effects: pd.DataFrame` (levels `(ring_label, event_time)`). TwoStageDiD-compatible `event_study_effects: Dict[int, Dict]` alias also emitted for `plot_event_study` consumption — `_extract_plot_data` prefers the new `reference_period` attribute over the legacy `n_obs==0` heuristic. (DiagnosticReport integration: WIRED — `SpilloverDiDResults` routes to parallel_trends (event-study on these direct-effect dynamics), design_effect, and heterogeneity; bacon excluded since spillover identifies off far-away controls, not TWFE 2x2 comparisons.) (schema: `{k: {"effect", "se", "n_obs", "t_stat", "p_value", "conf_int": (low, high)}}` mirroring `two_stage.py:1355-1389`). Reference period `ref_period = -1 - anticipation` (TwoStageDiD `two_stage.py:486` convention); reference row uses `coef=0.0, se=0.0, n_obs=0, conf_int=(0.0, 0.0)`. Scalar `att` field becomes a sample-share-weighted average of post-treatment `tau_k` (`att = sum_{k>=0} w_k * tau_k` with `w_k = n_treated_at_k / total`) with SE from linear-combination inference `Var(att) = w' V_subset w` on the post-treatment vcov block — no separate fit. **Two-clock K_it:** direct-effect clock is `K_direct = t - effective_first_treat(i)` for ever-treated rows; spillover clock is `K_spill = t - earliest-in-range-cohort-onset(i)` (running min across activated cohorts, NaN pre-trigger). `K_spill >= 0` structurally; negative-k spillover cells are rectangularly emitted with `coef = NaN, n_obs = 0`. **`horizon_max` semantics:** bins event-times outside `[-H, +H]` into endpoint pools (no observations dropped — divergence from TwoStageDiD which filters; intentional, per `feedback_no_silent_failures`). With `horizon_max=None`, auto-detects bin set from observed K. **Validation:** `horizon_max < 0` raises `ValueError`; `ref_period < -horizon_max` (i.e., `anticipation > horizon_max - 1`) raises `ValueError` — silently floor-shifting the reference would change identification. **Reduce-to-aggregate:** under constant-tau DGP with `horizon_max=None`, the share-weighted scalar `att` reproduces Wave B's aggregate bit-identically. **Note:** `horizon_max=0` does NOT reduce to Wave B (binning collapses pre-treatment K values to `k=0`, making `D^0 = D_i` ever-treated indicator rather than `D_it`). Per-event-time SEs include the Wave D Gardner GMM first-stage correction (see next bullet).
518518
- Stage-2 variance applies the Gardner GMM first-stage uncertainty correction across HC1 / Conley / cluster (Wave D, SHIPPED). The IF outer-product formula `psi_i = gamma_hat' X_{10,i} eps_{10,i} - X_{2,i} eps_{2,i}` is used unconditionally; kernel `K` is path-dependent (identity for HC1, block-indicator for cluster, spatial kernel for Conley). Documented synthesis of Butts (2021) §3.1 + Gardner (2022) §4 + Conley (1999); no reference software combines all three. Point estimates unchanged from Wave B/C; SE values shift upward by 1-few percent.
519519
- Only nearest-treated rings supported; `ring_method="count"` (count of treated neighbors in ring) not yet exposed
520520

@@ -2342,7 +2342,7 @@ DIFF_DIFF_BACKEND=rust pytest # Force Rust (fail if unavailable)
23422342

23432343
## BusinessReport
23442344

2345-
Plain-English stakeholder narrative from any of the 16 fitted result types.
2345+
Plain-English stakeholder narrative from any fitted result type.
23462346
Renders `summary()` (short paragraph), `full_report()` (multi-section
23472347
markdown), and `to_dict()` (stable AI-legible schema — single source of
23482348
truth; prose renders from the dict).

diff_diff/results.py

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -434,9 +434,11 @@ class SpilloverDiDResults(DiDResults):
434434
# TwoStageDiD-compatible alias for ``att_dynamic`` consumable by
435435
# ``plot_event_study`` (wired in Wave C via the ``reference_period``
436436
# attribute fallback in ``_extract_plot_data``). ``DiagnosticReport``
437-
# routing is NOT yet wired — registering ``SpilloverDiDResults`` in
438-
# ``DiagnosticReport``'s applicability/method tables is a planned
439-
# follow-up (see TODO.md).
437+
# routing is now wired: ``SpilloverDiDResults`` is registered in
438+
# ``DiagnosticReport``'s ``_APPLICABILITY`` / ``_PT_METHOD`` tables with
439+
# applicable checks {parallel_trends (event-study on these direct-effect
440+
# dynamics), design_effect, heterogeneity}; bacon is excluded (spillover
441+
# identifies off far-away units, not TWFE 2x2 comparisons).
440442
# Schema mirrors ``two_stage.py:1355-1389``:
441443
# {k: {"effect", "se", "n_obs", "t_stat", "p_value", "conf_int": (low, high)}}
442444
# Reference row uses ``conf_int = (0.0, 0.0)`` (TwoStageDiD parity).

docs/api/spillover.rst

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -196,7 +196,10 @@ and planned follow-up enhancements:
196196
``TwoStageDiD``'s schema for ``plot_event_study`` consumption (the
197197
plotter prefers the new ``reference_period`` attribute over the
198198
legacy ``n_obs==0`` heuristic). ``DiagnosticReport`` routing for
199-
``SpilloverDiDResults`` is queued as a follow-up. Reference
199+
``SpilloverDiDResults`` is wired: parallel-trends (event-study on
200+
the direct-effect dynamics), design-effect, and heterogeneity apply;
201+
Goodman-Bacon is excluded (spillover identifies off far-away
202+
controls, not TWFE 2x2 comparisons). Reference
200203
period ``-1 - anticipation`` (TwoStageDiD parity). ``horizon_max``
201204
bins event-times into endpoint pools (no row drop — divergence
202205
from TwoStageDiD's filtering semantic, intentional per

0 commit comments

Comments
 (0)