diff --git a/CHANGELOG.md b/CHANGELOG.md index f86d5967..0670fb63 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Testing +- **CI-locked standard-error parity for flagship and previously-unasserted paths (SE-audit + coverage batch).** These surfaces computed SEs matching R but had no CI assertion pinning them + (the latent-risk pattern that once hid the CallawaySantAnna reg-method gap): + - **New `fixest` DiD/TWFE golden** (`benchmarks/data/fixest_did_twfe_golden.json`, generated by + `benchmarks/R/generate_fixest_did_twfe_golden.R`): the flagship 2×2 DiD and TWFE classical/iid + ATT **and SE** now match `fixest::feols` to ~1e-16 in CI without R at test time (the prior + live-`Rscript` tests only asserted `att` at rtol=1e-3 and skipped in CI). The TWFE assertion + also locks the D4 full-K within-transform rescale. + - **CR2 backbone**: `TwoWayFixedEffects(vcov_type="hc2_bm")`'s default CR2-clustered-at-unit SE + and the one-way HC2-BM SE (previously only their Satterthwaite DOF was asserted); the + `MultiPeriodDiD` average-effect CR2 SE (previously only checked finite). + - **StackedDiD** event-study coefficient point estimates; **PlaceboTests** leave-one-out + `t_stat`/`p_value`; **DIDHAD** QUG order-statistic test (`t_stat`/`p_value`, previously + unasserted against the golden). All at ~1e-10. + ### Added - **`ContinuousDiD` discrete-treatment saturated regression** (`treatment_type="discrete"`) for multi-valued / discrete dose (CGBS 2024 Eq. 4.1). Each distinct dose level gets its own effect diff --git a/TODO.md b/TODO.md index 8cd58779..eb171609 100644 --- a/TODO.md +++ b/TODO.md @@ -66,6 +66,7 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m | Issue | Location | Origin | Effort | Priority | |-------|----------|--------|--------|----------| +| SE-audit CI-lock follow-up ("fiddly bits" deferred from the G2+C1-C3/C5 coverage batch). The `fixest_did_twfe_golden.json` stores `cluster_unit` ATT+SE for DiD and TWFE but only the ATT is asserted (the cluster SE carries the documented CR1 DOF-convention difference vs fixest; the hetero SE collapses to iid on these balanced 2-group designs). Also deferred, each needing plumbing or convention work: surface the 2SLS intercept SE for estimatr `se_intercept` parity (C4); surface StackedDiD intercept SEs (`se_cr1/cr2_intercept`, C1); add a `df` field + `boundary_gap` computation for PlaceboTests (C3); assert Yatchew `p`/`sigma2_lin/diff` through the N/(N-1) convention shift + event-map (C5); assert `dof_hc2_bm`/`dof_per_coef` via CI-inversion (C2). Plus the tolerance-tightenings C6-C8. | `tests/test_fixest_did_twfe_parity.py`, `tests/test_estimatr_iv_robust_parity.py`, `tests/test_methodology_stacked_did.py`, `tests/test_methodology_placebo.py`, `tests/test_did_had_parity.py` | SE-audit | Mid | Low | | Render `docs/methodology/REPORTING.md` and `REGISTRY.md` as in-site Sphinx pages so cross-refs can use `:doc:` instead of off-site `blob/main` URLs (stable-docs readers can otherwise land on a different revision than their package version). Two paths: (a) add `myst-parser` to `conf.py` + docs extras and link with `:doc:`, or (b) convert both to `.rst`. **Note:** REGISTRY.md is ~4.5k lines of LaTeX-heavy markdown — high risk under the `-W` (warnings-as-errors) Sphinx build; budget multiple rounds. | `docs/conf.py`, `docs/api/business_report.rst`, `docs/api/diagnostic_report.rst`, tutorials 18 & 19 | follow-up | Mid | Low | | `ImputationDiD` covariate-path variance lacks a dedicated parity anchor — only the no-covariate staggered panel is R-parity'd, though the covariate path shares the same validated projection code. Add a small dense-design **hand-calc** for the covariate projection (no external tooling), or a covariate (time-varying X) R `didimputation` golden asserting overall/ES SE parity (the golden variant needs local R). | `tests/test_methodology_imputation.py`, `benchmarks/R/generate_didimputation_golden.R` | imputation-validation | Mid | Low | | Add true half-sample BRR replicate-weight regressions per estimator family (current tests use Fay-like 0.5/1.5 perturbations; `test_survey_phase6.py` covers true BRR at the helper level). | `tests/test_replicate_weight_expansion.py` | #253 | Mid | Low | diff --git a/benchmarks/R/generate_fixest_did_twfe_golden.R b/benchmarks/R/generate_fixest_did_twfe_golden.R new file mode 100644 index 00000000..7dd5e87b --- /dev/null +++ b/benchmarks/R/generate_fixest_did_twfe_golden.R @@ -0,0 +1,117 @@ +# Generate a committed fixest golden for the flagship 2x2 DiD and TWFE +# standard-error parity paths (SE-audit item G2). +# +# The core DiD / TWFE fixest SE parity was previously only checked by +# skip-guarded live-Rscript tests that never run in CI -- and those tests only +# assert `att` (rtol=1e-3), never the SE. This golden materializes deterministic +# panels and fixest's feols() ATT + SE so tests can assert machine-precision SE +# parity WITHOUT R at test time. +# +# Scope (this golden): the classical / iid SE, which Python matches to machine +# precision on both the 2x2 DiD path and the within-transform TWFE path (the +# latter also locks the SE-audit D4 full-K rescale). The cluster-robust ATT is +# stored too; its SE carries the documented CR1 small-sample DOF-convention +# difference vs fixest and is left to a follow-up. (`hetero`/HC1 collapses to iid +# on these balanced 2-group designs, so it is not a distinct target here.) +# +# Regenerate: Rscript benchmarks/R/generate_fixest_did_twfe_golden.R +# Output: benchmarks/data/fixest_did_twfe_golden.json +# Environment: R 4.5.2, fixest 0.14.2. + +suppressMessages(library(fixest)) +suppressMessages(library(jsonlite)) + +set.seed(12345) + +fit_att <- function(model, vcov_spec) { + ct <- coeftable(model, vcov = vcov_spec) + idx <- which(rownames(ct) == "treated:post") + ci <- confint(model, vcov = vcov_spec)[idx, ] + list( + att = unbox(ct[idx, "Estimate"]), + se = unbox(ct[idx, "Std. Error"]), + t_stat = unbox(ct[idx, "t value"]), + p_value = unbox(ct[idx, "Pr(>|t|)"]), + ci_lower = unbox(ci[[1]]), + ci_upper = unbox(ci[[2]]) + ) +} + +# --------------------------------------------------------------------------- +# Scenario 1: 2x2 DiD (200 units x 2 periods), outcome ~ treated * post +# --------------------------------------------------------------------------- +n_units_did <- 200 +did_rows <- list() +i <- 1 +for (unit in 0:(n_units_did - 1)) { + is_treated <- as.integer(unit < n_units_did %/% 2) + for (period in c(0, 1)) { + y <- 10.0 + period * 2.0 + if (is_treated == 1 && period == 1) y <- y + 3.0 + y <- y + rnorm(1, 0, 1) + did_rows[[i]] <- data.frame(unit = unit, outcome = y, treated = is_treated, post = period) + i <- i + 1 + } +} +did <- do.call(rbind, did_rows) +did_m <- feols(outcome ~ treated * post, data = did) + +did_golden <- list( + data = list(unit = did$unit, outcome = did$outcome, treated = did$treated, post = did$post), + n_obs = unbox(nrow(did)), + iid = fit_att(did_m, "iid"), + cluster_unit = fit_att(did_m, ~unit) +) + +# --------------------------------------------------------------------------- +# Scenario 2: TWFE (50 units x 4 periods), outcome ~ treated:post | unit + post +# --------------------------------------------------------------------------- +n_units_twfe <- 50 +n_periods_twfe <- 4 +twfe_rows <- list() +i <- 1 +for (unit in 0:(n_units_twfe - 1)) { + is_treated <- as.integer(unit < n_units_twfe %/% 2) + unit_effect <- unit * 0.2 + for (period in 0:(n_periods_twfe - 1)) { + post <- as.integer(period >= n_periods_twfe %/% 2) + y <- 5.0 + unit_effect + period * 1.5 + if (is_treated == 1 && post == 1) y <- y + 2.5 + y <- y + rnorm(1, 0, 1) + twfe_rows[[i]] <- data.frame( + unit = unit, period = period, outcome = y, treated = is_treated, post = post + ) + i <- i + 1 + } +} +twfe <- do.call(rbind, twfe_rows) +twfe_m <- feols(outcome ~ treated:post | unit + post, data = twfe) + +twfe_golden <- list( + data = list( + unit = twfe$unit, period = twfe$period, outcome = twfe$outcome, + treated = twfe$treated, post = twfe$post + ), + n_obs = unbox(nrow(twfe)), + iid = fit_att(twfe_m, "iid"), + cluster_unit = fit_att(twfe_m, ~unit) +) + +# --------------------------------------------------------------------------- +golden <- list( + meta = list( + generator = unbox("benchmarks/R/generate_fixest_did_twfe_golden.R"), + r_version = unbox(paste(R.version$major, R.version$minor, sep = ".")), + fixest_version = unbox(as.character(packageVersion("fixest"))), + description = unbox(paste( + "fixest feols() ATT + SE golden for the flagship 2x2 DiD", + "(outcome ~ treated*post) and TWFE (| unit + post) paths, SE-audit G2." + )) + ), + did = did_golden, + twfe = twfe_golden +) + +out <- "benchmarks/data/fixest_did_twfe_golden.json" +writeLines(toJSON(golden, pretty = TRUE, digits = 16, auto_unbox = FALSE), out) +cat("Wrote", out, "\n") diff --git a/benchmarks/data/fixest_did_twfe_golden.json b/benchmarks/data/fixest_did_twfe_golden.json new file mode 100644 index 00000000..949abfeb --- /dev/null +++ b/benchmarks/data/fixest_did_twfe_golden.json @@ -0,0 +1,59 @@ +{ + "meta": { + "generator": "benchmarks/R/generate_fixest_did_twfe_golden.R", + "r_version": "4.5.2", + "fixest_version": "0.14.2", + "description": "fixest feols() ATT + SE golden for the flagship 2x2 DiD (outcome ~ treated*post) and TWFE (| unit + post) paths, SE-audit G2." + }, + "did": { + "data": { + "unit": [0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13, 14, 14, 15, 15, 16, 16, 17, 17, 18, 18, 19, 19, 20, 20, 21, 21, 22, 22, 23, 23, 24, 24, 25, 25, 26, 26, 27, 27, 28, 28, 29, 29, 30, 30, 31, 31, 32, 32, 33, 33, 34, 34, 35, 35, 36, 36, 37, 37, 38, 38, 39, 39, 40, 40, 41, 41, 42, 42, 43, 43, 44, 44, 45, 45, 46, 46, 47, 47, 48, 48, 49, 49, 50, 50, 51, 51, 52, 52, 53, 53, 54, 54, 55, 55, 56, 56, 57, 57, 58, 58, 59, 59, 60, 60, 61, 61, 62, 62, 63, 63, 64, 64, 65, 65, 66, 66, 67, 67, 68, 68, 69, 69, 70, 70, 71, 71, 72, 72, 73, 73, 74, 74, 75, 75, 76, 76, 77, 77, 78, 78, 79, 79, 80, 80, 81, 81, 82, 82, 83, 83, 84, 84, 85, 85, 86, 86, 87, 87, 88, 88, 89, 89, 90, 90, 91, 91, 92, 92, 93, 93, 94, 94, 95, 95, 96, 96, 97, 97, 98, 98, 99, 99, 100, 100, 101, 101, 102, 102, 103, 103, 104, 104, 105, 105, 106, 106, 107, 107, 108, 108, 109, 109, 110, 110, 111, 111, 112, 112, 113, 113, 114, 114, 115, 115, 116, 116, 117, 117, 118, 118, 119, 119, 120, 120, 121, 121, 122, 122, 123, 123, 124, 124, 125, 125, 126, 126, 127, 127, 128, 128, 129, 129, 130, 130, 131, 131, 132, 132, 133, 133, 134, 134, 135, 135, 136, 136, 137, 137, 138, 138, 139, 139, 140, 140, 141, 141, 142, 142, 143, 143, 144, 144, 145, 145, 146, 146, 147, 147, 148, 148, 149, 149, 150, 150, 151, 151, 152, 152, 153, 153, 154, 154, 155, 155, 156, 156, 157, 157, 158, 158, 159, 159, 160, 160, 161, 161, 162, 162, 163, 163, 164, 164, 165, 165, 166, 166, 167, 167, 168, 168, 169, 169, 170, 170, 171, 171, 172, 172, 173, 173, 174, 174, 175, 175, 176, 176, 177, 177, 178, 178, 179, 179, 180, 180, 181, 181, 182, 182, 183, 183, 184, 184, 185, 185, 186, 186, 187, 187, 188, 188, 189, 189, 190, 190, 191, 191, 192, 192, 193, 193, 194, 194, 195, 195, 196, 196, 197, 197, 198, 198, 199, 199], + "outcome": [10.585528817843855, 15.709466017509524, 9.8906966853189466, 14.546502826537237, 10.605887455840394, 13.182044032296272, 10.630098551068391, 14.723815894774784, 9.7158402560566284, 14.080677997525871, 9.8837521936479984, 16.817312043704217, 10.370627864257955, 15.520216457554957, 9.2494680054976683, 15.816899839520582, 9.1136424787567876, 14.668422410057447, 11.120712651669557, 15.298723699267294, 10.779621924555324, 16.455785082476865, 9.3556715707686973, 13.446862594770312, 8.4022904833036876, 16.805097518810825, 9.5183526363053641, 15.620379801298423, 10.612123492650849, 14.837689023081875, 10.81187317855386, 17.196833546347534, 12.049190337406193, 16.632445639480466, 10.254271192814056, 15.491188279272558, 9.6759134212628819, 13.337949756141366, 11.767733850872967, 15.025801048647809, 11.128510833590179, 12.619641938602957, 8.939734447847469, 15.937140540182908, 10.854451720330553, 16.460729403104089, 8.5869012221108054, 15.567403253424482, 10.583187653435685, 13.693201166535575, 9.4596139261378944, 16.947692664732038, 10.053590270357454, 15.351662840555237, 9.3290234610060629, 15.277953694806143, 10.691171272874884, 15.823795328706087, 12.14506502048029, 12.653056021636626, 10.14959198061521, 13.657468518270456, 10.553303075503898, 16.58996284269541, 9.4131204056129008, 13.1676226937906, 10.888139432652018, 16.593488472356274, 10.516854670662386, 13.704328319705176, 10.054615575329375, 14.215350626808455, 8.9506471810437347, 17.33051196240471, 11.402705382867431, 15.942600850691525, 10.826258287152429, 14.188459509808919, 10.476248280753653, 16.021258407481092, 10.645383074206077, 16.043143551551442, 9.6956308873063382, 17.477110916962971, 10.971220673037445, 16.867099184566719, 10.672042469014787, 14.692046619162662, 10.536523716710873, 15.824870065351675, 9.0360985209861973, 14.144917488900209, 11.886946942025663, 14.608180628004817, 9.019367051050013, 15.687332100557041, 9.494956482479104, 17.157719816506415, 9.4002024361751637, 14.305453306729248, 10.223925407521289, 13.843776669759226, 10.422418527786999, 13.675244744151072, 10.141084312860285, 14.463952001509053, 9.6883939154218517, 16.556109642601257, 9.55196670870553, 15.321123537493968, 8.7698277531198094, 13.675941307845846, 11.261242274388296, 16.319231719642335, 9.9192462386319651, 14.494910193728508, 9.9478464066696759, 15.628860625108436, 12.180002397484838, 14.930982693480784, 11.544863602037596, 16.321452017162631, 10.322151575362803, 16.530955119735157, 9.578760306661426, 13.841178977181862, 8.1546317105785331, 16.157325287057148, 7.8764501189852432, 13.80396848188917, 11.642191990767341, 15.883654833162014, 10.524875891035917, 13.815340934369228, 12.655788272142795, 13.952086289781688, 8.9888774763809618, 15.668921652514333, 10.129177293676193, 14.577423132946485, 8.8597358551518735, 13.706284709967461, 9.4053012309291066, 13.499185919779162, 10.015855693065559, 15.540169567692219, 8.4527080329543018, 15.849652929928439, 10.89601318433245, 15.138690996809729, 8.3806716776338614, 15.548397923014402, 10.195282153811252, 14.19350200523653, 9.8913757647928726, 14.749053376147687, 11.699346668533179, 14.655701203735031, 10.067772061404218, 14.349430273145941, 9.512361484796914, 15.303151237257662, 9.7580259803707463, 14.518266432498102, 9.0081971357432735, 14.71935086119557, 10.633017344180171, 13.760181656892142, 11.764314068994318, 14.976320111846992, 10.199920483827434, 16.347192777640011, 10.036073487799698, 15.824581130566735, 8.29732814839398, 15.480950155267681, 12.483550089326254, 15.401364987505605, 10.215177170747044, 13.1842876470972, 9.0882605757671229, 14.950955309146378, 9.5946125233181867, 16.130381797846855, 10.815464735072572, 15.076417519048142, 11.45374735124973, 15.374121083436261, 9.8290959442474257, 14.497787188149491, 10.543522107940634, 14.494813997148377, 10.786795789402014, 15.300949401608442, 11.310223906747371, 15.798433770272112, 10.850860437614436, 14.556432031905311, 9.5532252123904211, 15.01330504177615, 8.563854293772927, 11.370740352737199, 10.24352176562534, 13.058362233345978, 10.831348817915028, 12.105211815952332, 8.2582869055426453, 12.645246993765225, 10.09710422467608, 11.923266305461823, 10.991950677701748, 11.140749238750898, 9.7184203116755956, 14.066247270690713, 9.3884470221833034, 12.315612824664818, 10.660293378144825, 10.277797592854887, 7.8653739516513781, 12.068945598137001, 10.8678217425811, 9.7099558205412464, 9.8498097117916803, 11.731218207208959, 11.79133204114731, 12.672268041698089, 9.7906988637032661, 12.012182506124372, 11.534116864384394, 12.07729181640714, 10.078437532748927, 11.220738929570304, 10.166559668731869, 12.26532456977475, 10.890780709620268, 11.532111630533487, 10.758374556806851, 11.358263642230058, 10.627671823247429, 12.248330126822982, 9.2999241832757917, 11.432598414880816, 9.7386060741105442, 10.936114964213415, 9.8936313460112491, 12.771103739675585, 12.747403542248726, 11.916065242924187, 10.543567634250284, 12.752861210635389, 9.1913258723421318, 13.001119847390726, 10.456052504632618, 10.565749692568925, 9.7346951837219233, 12.64176916364174, 9.5849789712884235, 11.540424323776062, 9.207505984280715, 10.841460866524615, 10.710889998448437, 13.267601752238235, 9.8568489442493608, 11.484971093656805, 11.482891184308192, 11.837411090073731, 10.041709174402316, 12.483039896138864, 8.8198728260721317, 11.336426260857207, 9.3653501057869359, 11.298036968554108, 10.576850378871315, 9.8869196292742298, 10.260909682674749, 13.147127190289652, 10.014793648556202, 11.688260756098042, 9.043803891895843, 12.473413758540708, 8.4861359203227682, 12.164280997670588, 9.1291347814346473, 13.593328994316586, 10.646597522791781, 12.357369685182645, 10.102393063199646, 11.324733174804951, 10.972085023739425, 12.755869929249423, 9.5717143769328565, 11.286075236284876, 9.8096159344327845, 12.39986481171624, 9.0221550854958465, 12.183736909549099, 7.8496894702562603, 11.377033474254096, 9.2345606842298746, 12.464309418783944, 10.522282168524832, 12.009793759932338, 9.5594738037970917, 13.199489534501403, 9.8825315106835774, 12.038209787380577, 11.194805632371287, 12.343958352102906, 9.6709270255350113, 13.670857916858999, 9.0819417971811838, 11.912192671362686, 11.320293719834741, 13.730786130498224, 12.162596077876763, 11.684267520667424, 9.4249038957633147, 10.593643564402115, 12.267859853558702, 11.229146071029321, 10.380315696994392, 12.605136772754966, 11.019674136817583, 12.474943044148217, 7.8140535598874141, 12.933192166975154, 10.475435421232726, 12.390281600028404, 9.2726718491568114, 12.986555948076592, 11.42398359867007, 12.484730511200338, 10.349236445934427, 12.860124249616192, 10.404611144608053, 12.367044921399527, 8.4808009504920232, 13.54980333197868, 10.499881065667667, 12.460872851532372, 12.076709933277439, 11.692349434416993, 10.952370885178272, 12.532788293060056, 9.9044522916246862, 11.857936621567884, 8.8169742890216511, 12.54531840953441, 7.4181445574192439, 12.77890003053602, 10.292940302703855, 11.913292094651597, 8.5336405045397647, 10.916820551264202, 11.057737367126522, 11.639679199443577, 10.350593765015391, 12.028257656193359, 10.473048261023495, 11.080845272328023, 9.6241765636604555, 10.187166235687776, 10.288600211104963, 11.810377418722897, 10.017860213211348, 12.650430242426131, 10.310254985421182, 13.668357168923887, 10.672614533611014, 11.72248201197386, 9.8539736080471627, 13.701431831675949, 10.471357880571047, 12.581700078573641, 10.666020696010317, 11.221097774841079, 11.163324847825363, 10.035455804799652, 10.769170668260006, 14.259771993410174, 9.5227277086169853, 11.897419491204099, 10.368696176156456, 11.464567006017447, 10.506601941315227, 11.849442902410214, 10.904243556201491, 14.242036137467217, 8.8048771093804632, 11.581477433908654, 10.798251394698955, 12.498172163391981, 10.11956022337119, 11.632797280755458, 10.262331084814878, 12.262704124824337, 10.640738877993305, 12.307089830771924, 9.9668705992232489, 10.625249473174474], + "treated": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], + "post": [0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1] + }, + "n_obs": 400, + "iid": { + "att": 2.9475876082926851, + "se": 0.20319954322261985, + "t_stat": 14.505877137053348, + "p_value": 1.531281441716454e-38, + "ci_lower": 2.54810287226349, + "ci_upper": 3.3470723443218802 + }, + "cluster_unit": { + "att": 2.9475876082926851, + "se": 0.20788458904501431, + "t_stat": 14.178961614390902, + "p_value": 5.3153950882205083e-32, + "ci_lower": 2.537648232476283, + "ci_upper": 3.3575269841090871 + } + }, + "twfe": { + "data": { + "unit": [0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 6, 6, 6, 6, 7, 7, 7, 7, 8, 8, 8, 8, 9, 9, 9, 9, 10, 10, 10, 10, 11, 11, 11, 11, 12, 12, 12, 12, 13, 13, 13, 13, 14, 14, 14, 14, 15, 15, 15, 15, 16, 16, 16, 16, 17, 17, 17, 17, 18, 18, 18, 18, 19, 19, 19, 19, 20, 20, 20, 20, 21, 21, 21, 21, 22, 22, 22, 22, 23, 23, 23, 23, 24, 24, 24, 24, 25, 25, 25, 25, 26, 26, 26, 26, 27, 27, 27, 27, 28, 28, 28, 28, 29, 29, 29, 29, 30, 30, 30, 30, 31, 31, 31, 31, 32, 32, 32, 32, 33, 33, 33, 33, 34, 34, 34, 34, 35, 35, 35, 35, 36, 36, 36, 36, 37, 37, 37, 37, 38, 38, 38, 38, 39, 39, 39, 39, 40, 40, 40, 40, 41, 41, 41, 41, 42, 42, 42, 42, 43, 43, 43, 43, 44, 44, 44, 44, 45, 45, 45, 45, 46, 46, 46, 46, 47, 47, 47, 47, 48, 48, 48, 48, 49, 49, 49, 49], + "period": [0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3], + "outcome": [5.6279651131262245, 6.502143950931778, 10.784377722710804, 10.998220913780912, 4.5827780713659703, 7.5281942394628736, 10.615181354325687, 11.765280083063875, 4.1951647108992871, 5.8793131200798943, 9.9287707970392773, 12.239086637823164, 6.804687034813016, 7.1667462166601652, 10.217542316346453, 12.580080262019598, 6.1280164800238275, 6.3693713833065937, 10.168585031890885, 13.886230388998605, 6.5142421022230543, 7.7931683807592087, 10.937345169538663, 13.207142723762983, 7.4280774574073005, 7.2699874306958696, 13.031908351727143, 12.4367328439275, 7.6323188315606947, 8.5090586629259253, 11.077459916125484, 13.352351643527371, 6.7706780106987487, 6.8749286738980171, 12.487332884207625, 14.090667776973891, 9.0391260841210794, 9.1922123089624623, 13.09893258373295, 15.735442768880111, 7.394249899631979, 9.1325214943716944, 12.602591102479783, 15.010210998225613, 8.4146783852656899, 9.2008706583080695, 11.790532049080154, 12.002791488018159, 7.8385453221705701, 7.8705851696591349, 13.613078103934548, 15.428268539119628, 7.0293672745814266, 8.6659913181598753, 14.105317464768261, 15.803544816030929, 9.2961826969564356, 9.1434015677929068, 12.363969480272125, 15.152538646178499, 6.7138113534459247, 9.2274771449301625, 13.656914836911099, 14.11818713176492, 8.0781021746398665, 8.8447493273765012, 12.694271265003227, 14.286047609173576, 6.3217663406183435, 9.7593528763699933, 15.199637649886597, 15.370631216805792, 6.3633781623633157, 10.645030066464795, 14.930842052438663, 15.102420856899126, 8.1989725957437436, 9.7878021859167443, 14.295067620242651, 15.702852087290863, 8.9759646027244884, 10.372609662385674, 14.368114379636948, 16.453742255459208, 7.7936641656413155, 10.910930786953303, 15.452336089487973, 16.338379546739841, 9.5596800123663357, 11.128435318230386, 13.737216283186594, 17.254658519406792, 9.3579041214090033, 9.0840045776390781, 14.872309506695229, 16.824169832616978, 9.0631750783774283, 10.453289259459764, 15.188541139783563, 16.488398343634739, 8.5796761415091911, 9.0330613531390256, 13.484715835481135, 13.562027689235762, 13.530733305570458, 11.537054536681008, 13.420455784800858, 15.576210811679333, 8.251037048084676, 11.666248647236092, 11.401507154115524, 13.66521960043203, 11.446303717678608, 12.053505181858247, 14.003642765259539, 15.867454660414005, 11.667101501944607, 11.04541885100498, 13.642202010336968, 14.787683220302183, 11.413020868399551, 13.064256260947447, 14.726065952100765, 15.084897784841759, 11.241615300165408, 12.924292050515328, 14.986409140414588, 15.276637162766956, 11.032466043449547, 11.132979694069041, 14.216462124123344, 16.734871997278262, 11.01279948883786, 12.741067597413769, 16.714128836079468, 16.745411052385318, 11.487680044369601, 14.485448282859128, 15.353798963944275, 16.246246061281973, 14.127463034858827, 13.312293520555775, 15.098757844939332, 18.410378149361485, 13.821455715607406, 15.793067994802197, 14.507764072066383, 14.412801889133533, 10.74439832512952, 14.469509417915171, 14.9287774133577, 16.866790322625711, 12.97736706776052, 15.257718000728527, 16.99864999962659, 19.218503930639461, 11.712540130492016, 15.51603338663365, 16.477225304631588, 16.573623092352463, 13.112843941668528, 14.565213574781662, 15.369372788914172, 18.01885546276613, 12.397055500143672, 14.018376798691619, 16.162837278146519, 17.412725684773726, 12.373972832653891, 16.298359184599111, 17.23669024580683, 17.570103736182247, 14.134802256130939, 13.892746810706736, 17.228767315048351, 18.676164732804111, 15.036223879089711, 15.775539637979172, 15.252852511609744, 18.355468797889625, 14.680062090936776, 14.149798430206872, 17.245102444487369, 18.192651464047991, 14.931364566537448, 15.901381806056948, 15.624537167131063, 18.72353877743927, 15.201641111029375, 13.344372625287878, 16.873089319050848, 17.587141109364868, 15.400861346328536, 16.81310077052542, 18.63191486258691, 20.781808497950898, 14.573689604108017, 16.102070968117463, 18.843702252321247, 18.858262652405017], + "treated": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], + "post": [0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 1] + }, + "n_obs": 200, + "iid": { + "att": 2.5711552747352178, + "se": 0.35000516029279766, + "t_stat": 7.3460496199093512, + "p_value": 1.2707814512534252e-11, + "ci_lower": 1.8795022045997385, + "ci_upper": 3.262808344870697 + }, + "cluster_unit": { + "att": 2.5711552747352178, + "se": 0.29109428123652853, + "t_stat": 8.8327234180393486, + "p_value": 1.0482182922053883e-11, + "ci_lower": 1.9861794154923553, + "ci_upper": 3.1561311339780804 + } + } +} diff --git a/tests/test_did_had_parity.py b/tests/test_did_had_parity.py index 1bacf8aa..6e955e74 100644 --- a/tests/test_did_had_parity.py +++ b/tests/test_did_had_parity.py @@ -56,7 +56,7 @@ import pytest from diff_diff import HeterogeneousAdoptionDiD -from diff_diff.had_pretests import yatchew_hr_test +from diff_diff.had_pretests import qug_test, yatchew_hr_test FIXTURE_PATH = Path(__file__).parent.parent / "benchmarks" / "data" / "did_had_golden.json" @@ -500,3 +500,41 @@ def test_metadata_n_dgps(self, fixture): def test_all_dgps_present(self, fixture): for name in DGP_NAMES: assert name in fixture["fixtures"], f"missing DGP {name!r}" + + +def _golden_qug(combos): + """First non-NA (qug_t, qug_p) across a DGP's combos. QUG is computed once + on the cross-sectional dose, so every combo repeats the same value (with + ``"NA"`` on placebo rows).""" + for cd in combos.values(): + res = cd["result"] + ts = res.get("qug_t") + ps = res.get("qug_p") + t_list = ts if isinstance(ts, list) else [ts] + p_list = ps if isinstance(ps, list) else [ps] + t_vals = [x for x in t_list if x not in (None, "NA")] + p_vals = [x for x in p_list if x not in (None, "NA")] + if t_vals and p_vals: + return float(t_vals[0]), float(p_vals[0]) + return None, None + + +@pytest.mark.parametrize("dgp_name", DGP_NAMES) +class TestQUGParity: + """QUG order-statistic dose test vs R DIDHAD (SE-audit C5). + + ``qug_test`` was never asserted against this golden. It depends only on the + cross-sectional dose vector (one dose per unit), so it is DGP-level, not + per-combo. Compares to the golden's ``qug_t`` (the order-statistic ratio T, + NOT a per-horizon treatment-effect t-stat) and ``qug_p = 1/(1+T)``. + """ + + def test_qug_t_and_p_match_r(self, fixture, dgp_name): + entry = fixture["fixtures"][dgp_name] + panel = _panel_from_fixture(entry) + dose = panel.groupby("g")["d"].max().to_numpy(dtype=np.float64) + res = qug_test(dose) + r_t, r_p = _golden_qug(entry["combos"]) + assert r_t is not None, f"{dgp_name}: no non-NA qug_t in golden" + np.testing.assert_allclose(res.t_stat, r_t, atol=1e-10, rtol=0) + np.testing.assert_allclose(res.p_value, r_p, atol=1e-10, rtol=0) diff --git a/tests/test_estimators_vcov_type.py b/tests/test_estimators_vcov_type.py index 39c55473..42ce4dd1 100644 --- a/tests/test_estimators_vcov_type.py +++ b/tests/test_estimators_vcov_type.py @@ -649,6 +649,13 @@ def test_multi_period_cluster_hc2_bm_avg_att_uses_clubsandwich_dof(self): unit="unit", ) assert np.isfinite(res.avg_att) and np.isfinite(res.avg_se) + # SE-audit C2: pin the average-effect SE (was only checked finite). The + # R golden carries the contrast `c_avg` and the CR2 vcov, so the expected + # avg SE is sqrt(c_avg' Vcr2 c_avg). + c_avg = np.asarray(d["c_avg"], dtype=np.float64) + vcov_cr2 = np.asarray(d["vcov_cr2"]).reshape(d["vcov_cr2_shape"], order="F") + expected_avg_se = float(np.sqrt(c_avg @ vcov_cr2 @ c_avg)) + np.testing.assert_allclose(res.avg_se, expected_avg_se, atol=1e-8, rtol=0) # Recover the implied DOF from the reported p_value: # avg_p_value = 2 * (1 - t.cdf(|t|, df)) -> df = root of # `t.sf(|t|, df) * 2 - p` (Satterthwaite-bounded scalar bisection @@ -1681,9 +1688,7 @@ def test_unweighted_cr2_bm_dof_scale_invariant_batch(self): treated = np.asarray(d["treated"], dtype=float) post = np.asarray(d["post"], dtype=float) unit = np.asarray(d["unit"]) - X = np.column_stack( - [np.ones(n) if nm == "(Intercept)" else np.zeros(n) for nm in names] - ) + X = np.column_stack([np.ones(n) if nm == "(Intercept)" else np.zeros(n) for nm in names]) # Rebuild the full design (same as the sibling test). period = np.asarray(d["period"]) tp = treated * post diff --git a/tests/test_fixest_did_twfe_parity.py b/tests/test_fixest_did_twfe_parity.py new file mode 100644 index 00000000..91c53a2c --- /dev/null +++ b/tests/test_fixest_did_twfe_parity.py @@ -0,0 +1,91 @@ +"""CI-locked fixest parity for the flagship 2x2 DiD and TWFE standard errors. + +The core DiD / TWFE ``fixest::feols`` SE parity was previously only checked by +skip-guarded live-``Rscript`` tests that (a) never run in CI and (b) assert only +``att`` (rtol=1e-3), never the SE. This module pins Python's classical/iid ATT +**and SE** against a committed ``fixest`` golden, so the flagship paths are +SE-locked in CI without R at test time (SE-audit item G2). + +The TWFE assertion also locks the SE-audit D4 full-K within-transform rescale +against fixest's iid vcov. + +Regenerate the golden via +``Rscript benchmarks/R/generate_fixest_did_twfe_golden.R``. +""" + +import json +import os + +import numpy as np +import pandas as pd +import pytest + +from diff_diff import DifferenceInDifferences, TwoWayFixedEffects + +_GOLDEN_PATH = os.path.join( + os.path.dirname(__file__), + "..", + "benchmarks", + "data", + "fixest_did_twfe_golden.json", +) + + +def _load_golden(): + if not os.path.exists(_GOLDEN_PATH): + return None + with open(_GOLDEN_PATH) as fh: + return json.load(fh) + + +def _build_df(block): + return pd.DataFrame({k: v for k, v in block["data"].items()}) + + +_SKIP = pytest.mark.skipif( + _load_golden() is None, + reason=( + "fixest_did_twfe_golden.json not present; regenerate via " + "`Rscript benchmarks/R/generate_fixest_did_twfe_golden.R`." + ), +) + + +@_SKIP +class TestFixestDiDTWFEParity: + """Python classical (iid) ATT + SE == fixest feols iid, machine precision.""" + + def test_did_2x2_classical_se_matches_fixest_iid(self): + golden = _load_golden() + df = _build_df(golden["did"]) + res = DifferenceInDifferences(vcov_type="classical").fit( + df, outcome="outcome", treatment="treated", time="post", unit="unit" + ) + exp = golden["did"]["iid"] + np.testing.assert_allclose(res.att, exp["att"], atol=1e-10, rtol=0) + np.testing.assert_allclose(res.se, exp["se"], atol=1e-10, rtol=0) + + def test_twfe_classical_se_matches_fixest_iid(self): + """Also locks the SE-audit D4 full-K within-transform rescale.""" + golden = _load_golden() + df = _build_df(golden["twfe"]) + res = TwoWayFixedEffects(vcov_type="classical").fit( + df, outcome="outcome", treatment="treated", time="post", unit="unit" + ) + exp = golden["twfe"]["iid"] + np.testing.assert_allclose(res.att, exp["att"], atol=1e-10, rtol=0) + np.testing.assert_allclose(res.se, exp["se"], atol=1e-10, rtol=0) + + def test_cluster_att_matches_fixest(self): + """The cluster-robust ATT matches fixest exactly (the SE carries the + documented CR1 DOF-convention difference and is deferred).""" + golden = _load_golden() + for key, est in ( + ("did", DifferenceInDifferences(vcov_type="hc1", cluster="unit")), + ("twfe", TwoWayFixedEffects(vcov_type="hc1", cluster="unit")), + ): + df = _build_df(golden[key]) + res = est.fit(df, outcome="outcome", treatment="treated", time="post", unit="unit") + np.testing.assert_allclose( + res.att, golden[key]["cluster_unit"]["att"], atol=1e-10, rtol=0 + ) diff --git a/tests/test_methodology_placebo.py b/tests/test_methodology_placebo.py index 2250bb1a..a3d21746 100644 --- a/tests/test_methodology_placebo.py +++ b/tests/test_methodology_placebo.py @@ -244,6 +244,8 @@ def test_leave_one_out_matches_r(self, golden, panel): gl = golden["leave_one_out"] assert res.placebo_effect == pytest.approx(gl["mean"], abs=1e-10) assert res.se == pytest.approx(gl["se"], abs=1e-10) + assert res.t_stat == pytest.approx(gl["t_stat"], abs=1e-10) + assert res.p_value == pytest.approx(gl["p_value"], abs=1e-10) assert res.conf_int[0] == pytest.approx(gl["ci_lower"], abs=1e-9) assert res.conf_int[1] == pytest.approx(gl["ci_upper"], abs=1e-9) assert res.leave_one_out_effects is not None diff --git a/tests/test_methodology_stacked_did.py b/tests/test_methodology_stacked_did.py index 3257ef1a..dde79599 100644 --- a/tests/test_methodology_stacked_did.py +++ b/tests/test_methodology_stacked_did.py @@ -240,6 +240,17 @@ def test_hc1_se_matches_clubsandwich_cr1s_unit_cluster(self, goldens, panel): "match clubSandwich CR1S contrast variance at atol=1e-10" ), ) + # SE-audit C1: the event-study POINT estimates (`coef_es`) were loaded + # for the DOF recovery but never pinned to R. They are vcov-independent, + # so this one assertion covers both cluster levels. + py_coef = np.array([res.event_study_effects[h]["effect"] for h in event_times]) + np.testing.assert_allclose( + py_coef, + goldens["unit"]["coef_es"], + atol=1e-10, + rtol=1e-10, + err_msg="StackedDiD event-study coefficients must match R `coef_es`.", + ) def test_hc2_bm_se_matches_clubsandwich_cr2_unit_cluster(self, goldens, panel): """cluster=unit, vcov_type=hc2_bm matches R `vcovCR(type='CR2')`. diff --git a/tests/test_methodology_twfe.py b/tests/test_methodology_twfe.py index da94224f..6b9819c5 100644 --- a/tests/test_methodology_twfe.py +++ b/tests/test_methodology_twfe.py @@ -1296,10 +1296,20 @@ def test_twfe_hc2_bm_dof_matches_singleton_cluster_cr2(self): ) y = data_local["y"].values.astype(np.float64) _, residuals, _ = solve_ols(X, y, vcov_type="hc2") - _, dof_bm_one_way = compute_robust_vcov(X, residuals, vcov_type="hc2_bm", return_dof=True) + vcov_ow, dof_bm_one_way = compute_robust_vcov( + X, residuals, vcov_type="hc2_bm", return_dof=True + ) att_idx = scenario["coef_names"].index("treat_post") dof_R = float(scenario["dof_bm_one_way"][att_idx]) np.testing.assert_allclose(float(dof_bm_one_way[att_idx]), dof_R, atol=1e-10, rtol=0) + # SE-audit C2: the one-way CR2 (HC2-BM) SE was loaded but never asserted. + if "vcov_cr2_one_way" in scenario: + n = len(scenario["coef_names"]) + vcov_cr2_ow = np.array(scenario["vcov_cr2_one_way"]).reshape((n, n), order="F") + se_R = float(np.sqrt(vcov_cr2_ow[att_idx, att_idx])) + np.testing.assert_allclose( + float(np.sqrt(vcov_ow[att_idx, att_idx])), se_R, atol=1e-10, rtol=0 + ) def test_twfe_hc2_bm_clustered_at_unit_dof_matches_clubsandwich(self): """CR2-BM DOF clustered at unit matches clubSandwich @@ -1350,3 +1360,23 @@ def test_twfe_hc2_bm_clustered_at_unit_dof_matches_clubsandwich(self): att_idx = scenario["coef_names"].index("treat_post") dof_R = float(scenario["dof_bm_unit"][att_idx]) np.testing.assert_allclose(float(dof_bm_unit[att_idx]), dof_R, atol=1e-10, rtol=0) + + def test_twfe_hc2_bm_se_matches_clubsandwich_cr2_unit(self): + """The SE ``TwoWayFixedEffects(vcov_type='hc2_bm')`` reports by default + (CR2 clustered at unit) matches clubSandwich ``vcovCR(type='CR2')`` at + atol=1e-10. SE-audit C2: the golden's ``vcov_cr2_unit`` diagonal was + loaded for the DOF test but its SE was never asserted.""" + scenario = _load_twfe_golden_scenario() + if scenario is None: + pytest.skip("twfe_two_period scenario not in golden JSON.") + if "vcov_cr2_unit" not in scenario: + pytest.skip("twfe_two_period scenario does not include vcov_cr2_unit.") + data = self._build_panel(scenario) + res = TwoWayFixedEffects(vcov_type="hc2_bm").fit( + data, outcome="y", treatment="treated", time="post", unit="unit" + ) + n = len(scenario["coef_names"]) + vcov_cr2 = np.array(scenario["vcov_cr2_unit"]).reshape((n, n), order="F") + att_idx = scenario["coef_names"].index("treat_post") + se_R = float(np.sqrt(vcov_cr2[att_idx, att_idx])) + np.testing.assert_allclose(res.se, se_R, atol=1e-10, rtol=0)