Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,33 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Removed
- **`HeterogeneousAdoptionDiD.fit()` no longer accepts the deprecated `survey=` and
`weights=` kwargs** (the pre-scheduled removal for the 3.7.0 minor bump; the
`DeprecationWarning` shipped in a prior release with "will be removed in the next minor
release"). Weighting is now expressed solely through the canonical `survey_design=`,
matching `CallawaySantAnna` / `EfficientDiD` / `ImputationDiD` / `TwoStageDiD` (which all
take `survey_design=` only and compute design-based Binder (1983) Taylor-linearization
variance via the shared `compute_survey_if_variance`).
- **`survey=SurveyDesign(...)`** -> **`survey_design=SurveyDesign(...)`** (pure rename; same
Binder-TSL path, byte-identical output).
- **`fit(weights=<array>)`** -> add the weights as a column on `data` and pass
**`survey_design=SurveyDesign(weights='col')`**. This is a variance-family change: the old
per-row `weights=` array produced a CCT-2014 pweight-robust / 2SLS-sandwich SE with Normal
inference (`variance_formula="pweight"` / `"pweight_2sls"`); the canonical path produces
Binder-TSL with `t(df_survey)` inference (`variance_formula="survey_binder_tsl"` /
`"survey_binder_tsl_2sls"`). Point estimates are unchanged; SEs move ~0.1%+.
- **`fit(weights=<array>, cluster=<col>)`** (the weighted-CR1 2SLS sandwich) has no drop-in
equivalent - migrate to **`survey_design=SurveyDesign(weights='w', psu='cluster_col')`**
(Binder-TSL clustered through the PSU).
- **`cband` is now keyword-only** on `fit()` (it previously followed the removed positional
`survey`/`weights` slots; keyword-only prevents a silent positional misbind).
- Note: this public-kwarg removal in a *minor* bump is the pre-scheduled HAD exception
documented since the deprecation shipped; other pending removals (e.g. `SyntheticDiD`
`lambda_reg`/`zeta`) remain gated on the next major (v4.0.0). The equivalent
`survey=`/`weights=` kwargs on the HAD pretest helpers (`stute_test`, `qug_test`,
`did_had_pretest_workflow`, ...) are unchanged in this release and removed separately.

### Testing
- **CI-locked standard-error parity for flagship and previously-unasserted paths (SE-audit
coverage batch).** These surfaces computed SEs matching R but had no CI assertion pinning them
Expand Down
2 changes: 1 addition & 1 deletion TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ Doable in principle, but no current caller and/or explicitly out of paper scope.
| `solve_ols` residual memory floor is inherent to SVD-based lstsq after the PR-E marshalling slim (rust-side allocator high-water 15.32 -> 7.81 GB on the 2.4M x 130 clustered solve; remaining = one fused equilibrated input copy + thin-SVD U transient + vcov scores block + LAPACK gelsd internals on the python path). Further reduction requires the tall-skinny-QR algorithm change - same trade-off as the parked QR-reuse row below (library-wide last-digit perturbation + loses the second independent collinearity detector); the two land together if ever reopened. Diagnosis + measurements in docs/performance-plan.md. | `rust/src/linalg.rs::solve_ols`, `diff_diff/linalg.py` | PR-E | Low |
| SunAbraham-family fits are SOLVER-bound after the v3.6.x demeaning speedups: on the county-class shape (3.1k units x 60 months, ~100 interaction columns) `solve_ols` = ~60% of fit time (faer SVD 1.24s + pivoted-QR rank detection 0.70s of 3.27s), and the share grows further under the Rust demean kernel; the pivoted QR alone is ~20% of the whole fit (measured attribution in docs/performance-plan.md). Candidate fix is QR-reuse (one factorization for rank detection + solve). **Parked per the 2026-07 correctness-first decision**: the change perturbs every coefficient in the library at the last digits (golden/parity/bit-identity recapture) and removes the SVD's independent singular-value truncation - a second, different near-collinearity detector that is deliberate defense-in-depth for the no-silent-failures contract (the FE-span guard episode showed borderline cases are real). Re-open only on practitioner demand for wide event studies; any implementation gates on fixest/R end-to-end parity, not just internal consistency. | `linalg.py::solve_ols`, `linalg.py::_detect_rank_deficiency` | PR-C attribution | Medium |
| dCDH parity-test SE/CI assertions only cover pure-direction scenarios; mixed-direction SE comparison is structurally apples-to-oranges (cell-count vs obs-count weighting). | `test_chaisemartin_dhaultfoeuille_parity.py` | #294 | Low |
| `HeterogeneousAdoptionDiD` survey-design API consolidation (**scheduled: next minor bump**): drop the deprecated `survey=` / `weights=` kwargs on all 8 HAD surfaces; only `survey_design=` remains. Also fold the legacy back-end `weights=` routing into the unified `_resolve_survey_for_fit` path. DeprecationWarning has shipped; the removal is ~50 LoC gated on the semver bump. | `had.py`, `had_pretests.py` | next minor | Medium |
| `HeterogeneousAdoptionDiD` survey-design API consolidation — **pretest helpers** (`HeterogeneousAdoptionDiD.fit` shipped in 3.7.0): drop the deprecated `survey=` / `weights=` kwargs on the 7 remaining HAD pretest surfaces (`did_had_pretest_workflow`, `qug_test`, `stute_test`, `yatchew_hr_test`, `stute_joint_pretest`, `joint_pretrends_test`, `joint_homogeneity_test`); only `survey_design=` remains. Each helper has its OWN weight back-end (array-in helpers read raw `survey`/`weights` for computation at `stute_test:1750`; data-in helpers route `weights=` through a downstream `make_pweight_design(weights_unit)`), so this needs per-helper tracing, not the mechanical deletion the fit() surface allowed. Deferred from the fit() removal PR to keep that change bounded. | `had_pretests.py` | next minor (fit() done 3.7.0) | Medium |
| `HeterogeneousAdoptionDiD` joint cross-horizon covariance / sup-t bands: per-horizon SEs use independent sandwiches (paper-faithful pointwise CIs per Pierce-Schott Fig 2). Follow-ups (low demand): IF-based stacking for joint cross-horizon inference; analytical H×H covariance on the weighted ES path. (A sup-t band on the unweighted ES path shipped via the clustered band — `cluster=` fires the simultaneous band on the unweighted path.) | `had.py::_fit_event_study` | Phase 2b / 4.5 B | Low |
| `HeterogeneousAdoptionDiD` event-study staggered timing beyond the last cohort: Phase 2b auto-filters to the last cohort (paper App B.2); earlier-cohort effects aren't HAD-identified (redirect to dCDH). Full staggered HAD needs a different identification path (out of paper scope). | `had.py::_validate_had_panel_event_study` | Phase 2b | Low |
| `HeterogeneousAdoptionDiD` survey-aware support-endpoint test (**research, waits on literature**): needs a calibrated support-infimum test under complex sampling (endpoint EVT × survey-aware functional CLT × tail-empirical-process theory). Permanent `NotImplementedError` on `qug_test(survey=...)`; rationale in REGISTRY § "QUG Null Test" Note (Phase 4.5 C0). | `had_pretests.py::qug_test` | Phase 4.5 C0 | Low |
Expand Down
16 changes: 7 additions & 9 deletions diff_diff/guides/llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -779,12 +779,10 @@ had.fit(
unit_col: str,
first_treat_col: str | None = None, # Required on staggered panels (last-cohort auto-filter trigger)
aggregate: str = "overall", # "overall" (single scalar WAS) or "event_study" (per-horizon WAS)
survey: SurveyDesign | None = None, # DEPRECATED alias of survey_design=
weights: np.ndarray | None = None, # DEPRECATED pweight shortcut alias
cband: bool = True, # Simultaneous (sup-t) confidence bands on event-study fits that are weighted/survey OR clustered
*,
survey_design: SurveyDesign | None = None, # Canonical survey-design kwarg (weights, strata, PSU, FPC)
trends_lin: bool = False, # Eq 17 linear-trend detrending. Requires aggregate="event_study"; needs F>=3 (pre-period depth) for the regression; rejects ALL weighting entry paths (survey_design= / survey= / weights= all raise NotImplementedError under trends_lin).
cband: bool = True, # Simultaneous (sup-t) confidence bands on event-study fits that are survey-weighted OR clustered (keyword-only)
survey_design: SurveyDesign | None = None, # Survey-design kwarg (weights, strata, PSU, FPC) — the sole weighting entry
trends_lin: bool = False, # Eq 17 linear-trend detrending. Requires aggregate="event_study"; needs F>=3 (pre-period depth) for the regression; rejects survey_design= (raises NotImplementedError under trends_lin).
covariates: Any | None = None, # NOT IMPLEMENTED — non-None raises NotImplementedError (deferred Appendix B.1 / Theorem 6 covariate-adjusted extension; pre-residualize the outcome on covariates as a workaround)
) -> HeterogeneousAdoptionDiDResults | HeterogeneousAdoptionDiDEventStudyResults
```
Expand Down Expand Up @@ -821,7 +819,7 @@ es = est.fit(data_mp, outcome_col='y', unit_col='unit',

**Staggered panels.** On multi-cohort panels with `aggregate="event_study"`, `fit()` auto-filters to the last treatment cohort plus never-treated units (paper Appendix B.2) and emits a `UserWarning` naming kept/dropped counts. The estimand is then a **last-cohort-only WAS**, not a multi-cohort average. For full multi-cohort staggered support, see `ChaisemartinDHaultfoeuille`.

**Mass-point + survey constraint.** When fitting `design="mass_point"` with `survey_design=` (or the deprecated `survey=` alias), `vcov_type="hc1"` (or `robust=True`) is required: the survey path composes the standard error via Binder-TSL on the HC1-scale influence function, so the default classical sandwich path raises `NotImplementedError`. The same HC1 requirement also fires on the `weights=` shortcut when `aggregate="event_study"` AND `cband=True` — UNLESS `cluster=` is set (a clustered mass-point fit resolves to the CR1 sandwich regardless of `vcov_type` and uses the clustered sup-t band, so there is no classical-vs-HC1 mismatch): the per-horizon IF matrix is HC1-scale and the sup-t bootstrap normalizes by it, so mixing in a classical analytical SE would produce inconsistent variance families. Classical vcov is allowed on `weights=` + `aggregate="overall"` and on `weights=` + `aggregate="event_study"` + `cband=False`. Passing `vcov_type="hc1"` is a safe default on weighted survey + sup-t examples since `vcov_type` is unused on the continuous designs (CCT-2014 robust SE is the only formula there).
**Mass-point + survey constraint.** When fitting `design="mass_point"` with `survey_design=SurveyDesign(...)`, `vcov_type="hc1"` (or `robust=True`) is required: the survey path composes the standard error via Binder-TSL on the HC1-scale influence function, so the default classical sandwich path raises `NotImplementedError` — on both the static path and the event-study path (the event-study rejection fires regardless of `cband`, since the Binder-TSL analytical SE consumes the HC1-scaled IF either way). The one exception is `cluster=`: a clustered mass-point fit resolves to the CR1 sandwich regardless of `vcov_type` (and uses the clustered sup-t band), so there is no classical-vs-HC1 mismatch. Passing `vcov_type="hc1"` is a safe default on weighted survey + sup-t examples since `vcov_type` is unused on the continuous designs (CCT-2014 robust SE is the only formula there).

### StackedDiD

Expand Down Expand Up @@ -1554,10 +1552,10 @@ Single-period results container for `HeterogeneousAdoptionDiD`. The table below
| `inference_method` | `str` | `"analytical_nonparametric"` or `"analytical_2sls"` |
| `vcov_type` | `str | None` | Mass-point only: `"classical"`, `"hc1"`, or `"cr1"` |
| `cluster_name` | `str | None` | Cluster column name when CR1 cluster-robust SE is requested; `None` otherwise |
| `survey_metadata` | `SurveyMetadata | None` | Repo-standard survey metadata when `survey_design=` / `weights=` is supplied |
| `survey_metadata` | `SurveyMetadata | None` | Repo-standard survey metadata when `survey_design=` is supplied |
| `bandwidth_diagnostics` | `BandwidthResult | None` | MSE-DPI selector output (continuous designs); `None` on `mass_point` |
| `bias_corrected_fit` | `BiasCorrectedFit | None` | Phase 1c bias-corrected local-linear fit object (continuous designs); `None` on `mass_point` |
| `variance_formula` | `str | None` | HAD-specific SE label on weighted fits, populated on BOTH continuous and mass-point designs: `"pweight"` (continuous, CCT 2014 weighted-robust on the `weights=` shortcut), `"survey_binder_tsl"` (continuous, Binder 1983 TSL on the `survey_design=` path), `"pweight_2sls"` (mass-point + `weights=`; label is applied uniformly across vcov families — classical / HC1 / CR1 — on the weighted 2SLS path, with the actual sandwich resolved via `vcov_type`), or `"survey_binder_tsl_2sls"` (mass-point, Binder 1983 TSL on the `survey_design=` path). `None` on unweighted fits |
| `variance_formula` | `str | None` | HAD-specific SE label on weighted fits, populated on BOTH continuous and mass-point designs: `"survey_binder_tsl"` (continuous, Binder 1983 TSL on the `survey_design=` path) or `"survey_binder_tsl_2sls"` (mass-point, Binder 1983 TSL on the `survey_design=` path; requires `vcov_type="hc1"` / `robust=True` — the mass-point survey path rejects `vcov_type="classical"`, and `cluster=` + `survey_design=` is rejected, so PSU clustering is expressed via `SurveyDesign(weights='<weight_col>', psu='<cluster_col>')`). `None` on unweighted fits |
| `effective_dose_mean` | `float | None` | Weighted denominator used by the β̂-scale rescaling, populated on weighted fits across all designs: weighted `mean(d)` (`continuous_at_zero`), weighted `mean(d − d_lower)` (`continuous_near_d_lower`), or weighted Wald-IV dose gap `mean(d | Z=1, w) − mean(d | Z=0, w)` (`mass_point`). `None` on unweighted fits |

**Methods:** `summary()`, `print_summary()`, `to_dict()`, `to_dataframe()`
Expand Down Expand Up @@ -1590,7 +1588,7 @@ Per-horizon event-study results container for `HeterogeneousAdoptionDiD` with `a
| `bandwidth_diagnostics` | `list[BandwidthResult | None] | None` | Per-horizon MSE-DPI selector output (continuous designs); `None` on `mass_point`; entries can be `None` on degenerate horizons |
| `bias_corrected_fit` | `list[BiasCorrectedFit | None] | None` | Per-horizon Phase 1c bias-corrected local-linear fit objects; `None` on `mass_point`; entries can be `None` on degenerate horizons |
| `filter_info` | `dict | None` | Staggered last-cohort auto-filter metadata (`F_last`, `n_kept`, `n_dropped`, `dropped_cohorts`); `None` when no filter applied |
| `variance_formula` | `str | None` | HAD-specific SE label applied UNIFORMLY across all horizons, populated on BOTH continuous and mass-point designs: `"pweight"` (continuous, CCT 2014 weighted-robust on the `weights=` shortcut), `"survey_binder_tsl"` (continuous, Binder 1983 TSL on the `survey_design=` path), `"pweight_2sls"` (mass-point + `weights=`; label applied uniformly across vcov families — classical / HC1 / CR1 — on the weighted 2SLS path, with the actual sandwich resolved via `vcov_type`), or `"survey_binder_tsl_2sls"` (mass-point, Binder 1983 TSL on the `survey_design=` path). `None` on unweighted fits |
| `variance_formula` | `str | None` | HAD-specific SE label applied UNIFORMLY across all horizons, populated on BOTH continuous and mass-point designs: `"survey_binder_tsl"` (continuous, Binder 1983 TSL on the `survey_design=` path) or `"survey_binder_tsl_2sls"` (mass-point, Binder 1983 TSL on the `survey_design=` path; requires `vcov_type="hc1"` / `robust=True` — the mass-point survey path rejects `vcov_type="classical"`, and `cluster=` + `survey_design=` is rejected, so PSU clustering is expressed via `SurveyDesign(weights='<weight_col>', psu='<cluster_col>')`). `None` on unweighted fits |
| `effective_dose_mean` | `float | None` | Weighted denominator used by the β̂-scale rescaling, populated on weighted fits across all designs: weighted `sum(w·d)/sum(w)` (`continuous_at_zero`), weighted `sum(w·(d − d_lower))/sum(w)` (`continuous_near_d_lower`), or weighted Wald-IV dose gap (`mass_point`). Scalar (not per-horizon) because the β̂-scale denominator is computed once on the fit sample. `None` on unweighted fits |
| `cband_low` | `np.ndarray | None` | Simultaneous (sup-t) band lower bounds; `None` when `cband=False` or on unweighted, unclustered fits (a clustered fit produces the band even when unweighted) |
| `cband_high` | `np.ndarray | None` | Simultaneous (sup-t) band upper bounds |
Expand Down
Loading
Loading