diff --git a/CHANGELOG.md b/CHANGELOG.md index 4fa7d9de..347c8718 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,7 +21,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 each a deferred follow-up. `LPDiDResults` gains `survey_metadata` / `n_strata` / `n_psu`, a `"survey_tsl"` `vcov_type`, and a Survey Design block in `summary()`. The non-survey path is byte-for-byte unchanged. Validated against `survey::svyglm` on the stacked long difference - (numeric golden parity is the D2 follow-up). + (full numeric golden parity in Phase D2, below). +- **`LPDiD` complex-survey R-parity validation** (Phase D2). Pins the D1 survey path end-to-end + against `survey::svyglm` (Lumley) goldens on a dedicated staggered-absorbing survey panel: + per-horizon point/SE/df + pooled-post/pre for three variance paths: the variance-weighted full + design (strata+PSU+FPC), the weights-only unit-injected-PSU design, and the direct-covariate + variant (point ~1e-6, SE ~1e-5, df exact via the per-design `n_PSU - n_strata` / `n_PSU - 1` + formula). Each horizon uses a fresh `svydesign` over that horizon's clean sample, matching the + library's per-sample resolution; `svyglm` is the reference implementation of the Binder TSL + sandwich so it anchors the variance directly, and the clean-sample construction is independently + cross-checked against `alexCardazzi/lpdid` (the unweighted variance-weighted event study matches + to <1e-8). New `tests/test_methodology_lpdid.py::TestLPDiDSurveyParityR` + + `benchmarks/R/generate_lpdid_survey_golden.R` + `lpdid_survey_panel.csv` / + `lpdid_survey_golden.json` (own seed, so the absorbing / non-absorbing goldens stay + byte-identical). No estimator change. - **`TROP` non-absorbing (on/off) treatment support** (Athey, Imbens, Qu & Viviano 2025, §2.1 / Eq. 12 / Algorithm 2). New `non_absorbing` parameter (default `False`). The paper supports general assignment patterns ("units moving into and out of treatment"), not only diff --git a/TODO.md b/TODO.md index 2734b50b..837cde40 100644 --- a/TODO.md +++ b/TODO.md @@ -69,7 +69,6 @@ generic sparse-FE, QR+SVD rank-detection redundancy, `check_finite` bypass — m | `ImputationDiD` covariate-path variance lacks a dedicated parity anchor — only the no-covariate staggered panel is R-parity'd, though the covariate path shares the same validated projection code. Add a small dense-design **hand-calc** for the covariate projection (no external tooling), or a covariate (time-varying X) R `didimputation` golden asserting overall/ES SE parity (the golden variant needs local R). | `tests/test_methodology_imputation.py`, `benchmarks/R/generate_didimputation_golden.R` | imputation-validation | Mid | Low | | Add true half-sample BRR replicate-weight regressions per estimator family (current tests use Fay-like 0.5/1.5 perturbations; `test_survey_phase6.py` covers true BRR at the helper level). | `tests/test_replicate_weight_expansion.py` | #253 | Mid | Low | | Port the CI `` extraction into the reviewer-eval harness so `docs/tutorials/*.ipynb` cases (currently guarded out of `verify-corpus`/`run`) can be reviewed with CI-equivalent context. | `tools/reviewer-eval/adapters/ci_prompt.py` | local-review | Mid | Low | -| **`LPDiD` survey-design R-parity (PR-D2)** — pin the stratified-PSU Taylor-linearization standard errors against `survey::svyglm` on the stacked long difference (per-horizon + pooled-post point/SE/df), mirroring `benchmark_survey_estimators.R`. The D1 build ships the survey path validated by pure-Python invariants (reduction/FPC/stratification/lonely-PSU/NaN-consistency); D2 adds the numeric `svyglm` golden + parity test. | `benchmarks/R/`, `tests/test_methodology_lpdid.py` | PR-D1 | Mid | Low | --- diff --git a/benchmarks/R/generate_lpdid_survey_golden.R b/benchmarks/R/generate_lpdid_survey_golden.R new file mode 100644 index 00000000..c82cec05 --- /dev/null +++ b/benchmarks/R/generate_lpdid_survey_golden.R @@ -0,0 +1,303 @@ +#!/usr/bin/env Rscript +# Golden generator: LPDiD COMPLEX-SURVEY path (Dube, Girardi, Jorda & Taylor 2025) +# vs survey::svyglm (Lumley 2004) -- Phase D2, closes Phase D of the LP-DiD salvage. +# +# D1 (#590) added survey support to LPDiD: probability weights + stratified-PSU +# Binder (1983) Taylor-linearization (TSL) sandwich SEs on the variance-weighted +# default path. D1 was smoke-gated against svyglm (~1e-10) on a hand-built single +# horizon. D2 pins the END-TO-END estimator: each LP-DiD horizon builds its own +# clean-control long-difference sample, and the matching svyglm reference must use +# a FRESH svydesign over THAT sample (NOT subset() of a full-panel design -- each +# horizon is a different stacked dataset, not a subpopulation of a fixed design). +# +# Survey variance is validated DIRECTLY against survey::svyglm, the reference +# implementation of the TSL sandwich -- no third-party survey package is needed as +# a gate. The CLEAN-SAMPLE CONSTRUCTION is instead cross-checked independently: +# * the UNWEIGHTED variance-weighted event study (same prep/clean_h recipe as the +# absorbing generator) must match alexCardazzi::lpdid() to <1e-8 (fail-closed, +# pinned commit). Clean-sample selection is weight-independent, so this +# transitively validates the weighted samples the svyglm goldens are built on. +# * every svyglm WLS point must equal weighted feols(.. | time, weights, vcov=~unit) +# on the same sample to <1e-7 (confirms sample/weight wiring before pinning SEs). +# +# Outputs (checked into the repo): +# benchmarks/data/lpdid_survey_panel.csv (staggered absorbing survey panel) +# benchmarks/data/lpdid_survey_golden.json (svyglm goldens: [est, se, df, n_psu]) +# +# Does NOT touch generate_lpdid_golden.R or its panels/goldens (separate seed + +# disjoint output paths => the absorbing / non-absorbing fixtures stay byte-identical). +# +# Usage: +# Rscript benchmarks/R/generate_lpdid_survey_golden.R + +suppressMessages({ + library(dplyr); library(tsibble); library(slider); library(fixest) + library(survey); library(data.table); library(jsonlite); library(lpdid) +}) +setFixest_ssc(ssc(adj = TRUE, cluster.adj = TRUE)) # authors' reghdfe-style convention +options(survey.lonely.psu = "fail") # fail-closed: a lonely PSU is an error + +SEED <- 20260630L +set.seed(SEED) +PRE <- 3L; POST <- 4L +ALEX_SHA <- "ba64563983861be5e616f6020842c1a1cdf17a27" +TOL_XCHECK <- 1e-8 # unweighted clean-sample cross-check vs alexCardazzi +TOL_POINT <- 1e-7 # svyglm WLS point == weighted feols point (same sample) + +# Verify the installed alexCardazzi/lpdid is exactly the pinned commit so the +# clean-sample cross-check gate is meaningful (fail-closed). +.alex_sha <- packageDescription("lpdid")$RemoteSha +if (is.null(.alex_sha) || is.na(.alex_sha)) { + stop(sprintf("installed `lpdid` has no RemoteSha; reinstall via remotes::install_github(\"alexCardazzi/lpdid\", ref = \"%s\")", + ALEX_SHA)) +} +if (.alex_sha != ALEX_SHA) { + stop(sprintf("installed `lpdid` commit %s != pinned ALEX_SHA %s; reinstall the pinned commit", + .alex_sha, ALEX_SHA)) +} + +# ============================================================ +# 1. Staggered absorbing survey panel. Balanced (gap-free). 4 strata; never-treated +# distributed ROUND-ROBIN across all PSUs => every per-horizon clean sample keeps +# all PSUs in every stratum (no lonely PSU; realized df = 14 - 4 = 10 for VW/cov). +# weight (inverse selection prob) is DECOUPLED from fpc (population PSU count) and +# dispersed across strata so the survey point clearly differs from the unweighted. +# ============================================================ +Tt <- 12L +strata_sizes <- c(30L, 30L, 40L, 40L) # 140 units +psu_per_stratum <- c(3L, 3L, 4L, 4L) # 14 PSUs +weight_h <- c(5.0, 10.0, 7.5, 15.0) # dispersed inverse-selection weights +fpc_h <- c(6L, 6L, 8L, 8L) # population PSU count per stratum (f = n_psu/fpc = 0.5) +cohorts <- c(5L, 8L) # ~25% @ g=5, ~25% @ g=8, ~50% never + +time_fe <- rnorm(Tt, 0, 1) +rows <- list(); idx <- 0L; unit <- 0L; global_psu <- 0L +for (h in seq_along(strata_sizes)) { + n_h <- strata_sizes[h]; n_psu_h <- psu_per_stratum[h] + for (i in 0:(n_h - 1L)) { + unit <- unit + 1L + psu_assign <- global_psu + (i %% n_psu_h) + 1L # round-robin within stratum + frac <- i / n_h + g <- if (frac < 0.25) cohorts[1] else if (frac < 0.5) cohorts[2] else 0L + unit_fe <- rnorm(1, 0, 2); eps_prev <- 0 + for (t in 1:Tt) { + eps <- 0.4 * eps_prev + rnorm(1); eps_prev <- eps + treated <- as.integer(g > 0 && t >= g) + eff <- if (treated == 1) { + k <- t - g; base <- if (g == 5L) 2.0 else 3.5; slope <- if (g == 5L) 0.5 else 0.2 + base + slope * k + } else 0 + x <- rnorm(1) + y <- unit_fe + time_fe[t] + eff + 0.7 * x + eps + idx <- idx + 1L + rows[[idx]] <- list(unit = unit, time = t, treat = treated, y = y, x = x, + stratum = h, psu = psu_assign, fpc = fpc_h[h], weight = weight_h[h]) + } + } + global_psu <- global_psu + n_psu_h +} +panel <- rbindlist(rows) +setorder(panel, unit, time) +panel_path <- file.path("benchmarks", "data", "lpdid_survey_panel.csv") +dir.create(dirname(panel_path), recursive = TRUE, showWarnings = FALSE) +fwrite(panel, panel_path) +message(sprintf("Wrote survey panel: %s (%d rows, %d units, %d PSUs, %d strata)", + panel_path, nrow(panel), uniqueN(panel$unit), + uniqueN(panel$psu), uniqueN(panel$stratum))) + +# ============================================================ +# 2. prep + clean_h: duplicated verbatim from generate_lpdid_golden.R (the recipe +# the absorbing feols golden uses, alexCardazzi-cross-checked). Survey columns are +# unit-constant and ride along through fill_gaps (panel is gap-free) into clean_h. +# ============================================================ +td <- panel[treat == 1, .(treat_date = min(time)), by = unit] +prep <- panel %>% + left_join(td, by = "unit") %>% + as_tsibble(key = unit, index = time) %>% + fill_gaps() %>% + group_by(unit) %>% arrange(time, .by_group = TRUE) %>% + mutate(treat_date = if (all(is.na(treat_date))) NA_real_ else suppressWarnings(min(treat_date, na.rm = TRUE))) %>% + mutate( + treat = as.integer(!is.na(treat_date) & time >= treat_date), + tdiff = treat - lag(treat, 1L), + Ly = lag(y, 1L), + obs = !is.na(y) + ) %>% ungroup() %>% as_tibble() +prep$tdiff[is.na(prep$tdiff) | prep$tdiff < 0] <- 0 + +clean_h <- function(h, base, kind) { + p <- prep %>% group_by(unit) %>% arrange(time, .by_group = TRUE) + if (kind == "post") { + p <- p %>% mutate(Dy = lead(y, h) - .data[[base]], Fh = lead(treat, h)) %>% ungroup() %>% + filter(obs & !is.na(Dy) & !is.na(tdiff) & !is.na(Fh) & (tdiff == 1 | Fh == 0)) + } else { + p <- p %>% mutate(Dy = lag(y, h) - .data[[base]]) %>% ungroup() %>% + filter(obs & !is.na(Dy) & !is.na(tdiff) & !is.na(treat) & (tdiff == 1 | treat == 0)) + } + p +} +identified <- function(d) nrow(d) > 0 && length(unique(d$time[d$tdiff == 1])) > 0 && + any(d$tdiff == 0) + +# unweighted feols event study (for the alexCardazzi clean-sample cross-check) ------ +feols_es <- function(formula) function(d) { + if (!identified(d)) return(c(NA_real_, NA_real_)) + m <- feols(formula, data = d, vcov = ~unit, warn = FALSE, notes = FALSE) + c(coef(m)[["tdiff"]], se(m)[["tdiff"]]) +} +post_h <- 0:POST +pre_h <- 2:PRE +run_es <- function(base, fml) { + out <- list() + for (h in post_h) out[[as.character(h)]] <- feols_es(fml)(clean_h(h, base, "post")) + for (h in pre_h) out[[as.character(-h)]] <- feols_es(fml)(clean_h(h, base, "pre")) + out +} + +# ============================================================ +# 3. svyglm goldens. Each horizon: a FRESH svydesign over that clean sample. +# Records c(estimate, SE, df, n_psu) for the tdiff coefficient. +# ============================================================ +des_vw <- function(d) svydesign(ids = ~psu, strata = ~stratum, fpc = ~fpc, weights = ~weight, data = d) +des_nofpc <- function(d) svydesign(ids = ~psu, strata = ~stratum, weights = ~weight, data = d) +des_inject <- function(d) svydesign(ids = ~unit, weights = ~weight, data = d) +fml_main <- Dy ~ tdiff + factor(time) +fml_cov <- Dy ~ tdiff + x + factor(time) + +svy_one <- function(d, formula, design_fn, psu_col, with_cov) { + if (!identified(d)) return(c(NA_real_, NA_real_, NA_real_, NA_real_)) + if (psu_col == "psu") { # no-lonely-PSU guard (strata designs) + chk <- d %>% group_by(stratum) %>% summarise(npsu = n_distinct(psu), .groups = "drop") + if (any(chk$npsu < 2L)) stop(sprintf("lonely PSU in a realized clean sample: %s", deparse(formula))) + } + des <- design_fn(d) + co <- summary(svyglm(formula, design = des))$coefficients + est <- unname(co["tdiff", "Estimate"]); se <- unname(co["tdiff", "Std. Error"]) + # internal point gate: same-sample weighted feols WLS point must match svyglm + ffml <- if (with_cov) (Dy ~ tdiff + x | time) else (Dy ~ tdiff | time) + pt <- unname(coef(feols(ffml, data = d, weights = ~weight, vcov = ~unit, warn = FALSE, notes = FALSE))["tdiff"]) + if (!is.finite(est) || abs(est - pt) > TOL_POINT) + stop(sprintf("svyglm WLS point != feols point (%.10f vs %.10f): %s", est, pt, deparse(formula))) + c(est, se, degf(des), length(unique(d[[psu_col]]))) +} + +svy_run_es <- function(formula, design_fn, psu_col, with_cov = FALSE) { + out <- list() + for (h in post_h) out[[as.character(h)]] <- svy_one(clean_h(h, "Ly", "post"), formula, design_fn, psu_col, with_cov) + for (h in pre_h) out[[as.character(-h)]] <- svy_one(clean_h(h, "Ly", "pre"), formula, design_fn, psu_col, with_cov) + out +} + +survey_vw_es <- svy_run_es(fml_main, des_vw, "psu") +survey_vw_nofpc_es <- svy_run_es(fml_main, des_nofpc, "psu") +survey_inject_es <- svy_run_es(fml_main, des_inject, "unit") +survey_cov_es <- svy_run_es(fml_cov, des_vw, "psu", with_cov = TRUE) +message("svyglm event studies built (vw / vw-nofpc / inject / covariate).") + +# ---- pooled (authors' fixed-composition window-mean recipe, clean-through-F.POST) ---- +pooled_tbl <- prep %>% group_by(unit) %>% arrange(time, .by_group = TRUE) %>% + mutate(ypost = slide_dbl(y, ~ mean(.x, na.rm = FALSE), .before = 0, .after = POST, .complete = TRUE), + ypre = slide_dbl(y, ~ mean(.x, na.rm = FALSE), .before = PRE, .after = -2, .complete = TRUE), + pooly_post = ypost - Ly, pooly_pre = ypre - Ly, + Fpost = lead(treat, POST)) %>% ungroup() +pp <- pooled_tbl %>% filter(obs & !is.na(pooly_post) & !is.na(tdiff) & !is.na(Fpost) & (tdiff == 1 | Fpost == 0)) +ppre <- pooled_tbl %>% filter(obs & !is.na(pooly_pre) & !is.na(tdiff) & !is.na(treat) & (tdiff == 1 | treat == 0)) + +svy_pooled <- function(d, yvar, design_fn, psu_col, with_cov) { + if (psu_col == "psu") { + chk <- d %>% group_by(stratum) %>% summarise(npsu = n_distinct(psu), .groups = "drop") + if (any(chk$npsu < 2L)) stop(sprintf("lonely PSU in pooled sample (%s)", yvar)) + } + des <- design_fn(d) + fml <- as.formula(if (with_cov) sprintf("%s ~ tdiff + x + factor(time)", yvar) + else sprintf("%s ~ tdiff + factor(time)", yvar)) + co <- summary(svyglm(fml, design = des))$coefficients + est <- unname(co["tdiff", "Estimate"]); se <- unname(co["tdiff", "Std. Error"]) + ffml <- as.formula(if (with_cov) sprintf("%s ~ tdiff + x | time", yvar) else sprintf("%s ~ tdiff | time", yvar)) + pt <- unname(coef(feols(ffml, data = d, weights = ~weight, vcov = ~unit, warn = FALSE, notes = FALSE))["tdiff"]) + if (!is.finite(est) || abs(est - pt) > TOL_POINT) + stop(sprintf("pooled svyglm point != feols point (%.10f vs %.10f): %s", est, pt, yvar)) + c(est, se, degf(des), length(unique(d[[psu_col]]))) +} + +survey_vw_pooled <- list(post = svy_pooled(pp, "pooly_post", des_vw, "psu", FALSE), + pre = svy_pooled(ppre, "pooly_pre", des_vw, "psu", FALSE)) +survey_inject_pooled <- list(post = svy_pooled(pp, "pooly_post", des_inject, "unit", FALSE), + pre = svy_pooled(ppre, "pooly_pre", des_inject, "unit", FALSE)) +survey_cov_pooled <- list(post = svy_pooled(pp, "pooly_post", des_vw, "psu", TRUE), + pre = svy_pooled(ppre, "pooly_pre", des_vw, "psu", TRUE)) +message("svyglm pooled rows built (vw / inject / covariate).") + +# ============================================================ +# 4. Independent clean-sample cross-check: UNWEIGHTED VW event study (same recipe) +# vs alexCardazzi::lpdid() at <1e-8 (fail-closed). Weight-independent selection +# => transitively validates the weighted svyglm samples above. +# ============================================================ +vw_unw_es <- run_es("Ly", Dy ~ tdiff | time) +alex_df <- as.data.frame(panel) +alex_fit <- function(...) tryCatch( + lpdid(alex_df, window = c(-PRE, POST), y = "y", unit_index = "unit", + time_index = "time", treat_status = "treat", ...), + error = function(e) list(error = conditionMessage(e))) +alex_es <- function(fit) { + if (!is.null(fit$error)) return(list(error = fit$error)) + setNames(lapply(seq_along(fit$window), + function(i) c(fit$coeftable$Estimate[i], fit$coeftable$`Std. Error`[i])), + as.character(fit$window)) +} +xcheck <- function(name, ours, theirs) { + if (!is.null(theirs$error)) stop(sprintf("alexCardazzi cross-check UNAVAILABLE for %s: %s", name, theirs$error)) + hs <- intersect(names(ours), names(theirs)) + if (length(hs) == 0L) stop(sprintf("alexCardazzi cross-check for %s: no overlapping horizons", name)) + diffs <- sapply(hs, function(h) { + a <- ours[[h]]; b <- theirs[[h]] + if (any(is.na(a)) || any(is.na(b))) NA_real_ else max(abs(a[1:2] - b[1:2])) + }) + if (all(is.na(diffs))) stop(sprintf("alexCardazzi cross-check for %s: all horizons non-comparable", name)) + d <- max(diffs, na.rm = TRUE) + message(sprintf(" [xcheck %s] max|ours - alex| = %.2e over h={%s}", name, d, paste(hs, collapse = ","))) + if (!is.finite(d) || d > TOL_XCHECK) + stop(sprintf("alexCardazzi cross-check FAILED for %s (%.2e > %.0e)", name, d, TOL_XCHECK)) +} +message("Independent clean-sample cross-check (unweighted VW vs alexCardazzi):") +xcheck("survey_clean_sample_unweighted_vw", vw_unw_es, alex_es(alex_fit())) + +# ============================================================ +# 5. Write golden JSON +# ============================================================ +golden <- list( + meta = list( + estimator = "LPDiD (Dube, Girardi, Jorda & Taylor 2025) - complex survey (Phase D2)", + r_version = R.version.string, + survey_version = as.character(packageVersion("survey")), + fixest_version = as.character(packageVersion("fixest")), + lpdid_alexcardazzi_version = as.character(packageVersion("lpdid")), + lpdid_alexcardazzi_commit = ALEX_SHA, + seed = SEED, pre_window = PRE, post_window = POST, + n_strata = uniqueN(panel$stratum), n_psu_total = uniqueN(panel$psu), + record_format = "Each horizon / pooled row is c(estimate, SE, df, n_psu) for the tdiff coefficient.", + se_convention = paste( + "survey::svyglm Binder (1983) Taylor-linearization sandwich. df = degf(design):", + "n_PSU - n_strata (strata present, VW/covariate) or n_PSU - 1 (weights-only inject).", + "Each horizon uses a FRESH svydesign over THAT horizon's clean LP-DiD long-difference", + "sample (NOT subset() of a full design) - matching the library's per-sample resolution."), + clean_sample_xcheck = paste( + "Clean-sample construction independently validated: the UNWEIGHTED VW event study", + "(same prep/clean_h recipe) matches alexCardazzi::lpdid() to <1e-8. Selection is", + "weight-independent, so this transitively validates the weighted svyglm samples."), + point_gate = paste( + "Every svyglm WLS point validated == weighted feols(.. | time, weights, vcov=~unit)", + "on the same sample to <1e-7."), + fpc_note = "fpc is the per-stratum population PSU count (f_h = n_PSU_h / fpc_h); decoupled from the inverse-selection weight." + ), + survey_vw_es = survey_vw_es, + survey_vw_nofpc_es = survey_vw_nofpc_es, + survey_inject_es = survey_inject_es, + survey_cov_es = survey_cov_es, + survey_vw_pooled = survey_vw_pooled, + survey_inject_pooled = survey_inject_pooled, + survey_cov_pooled = survey_cov_pooled +) +golden_path <- file.path("benchmarks", "data", "lpdid_survey_golden.json") +write_json(golden, golden_path, auto_unbox = TRUE, pretty = TRUE, digits = 12, na = "null") +message(sprintf("Wrote survey golden: %s", golden_path)) diff --git a/benchmarks/data/lpdid_survey_golden.json b/benchmarks/data/lpdid_survey_golden.json new file mode 100644 index 00000000..bfcc031b --- /dev/null +++ b/benchmarks/data/lpdid_survey_golden.json @@ -0,0 +1,68 @@ +{ + "meta": { + "estimator": "LPDiD (Dube, Girardi, Jorda & Taylor 2025) - complex survey (Phase D2)", + "r_version": "R version 4.5.2 (2025-10-31)", + "survey_version": "4.5", + "fixest_version": "0.13.2", + "lpdid_alexcardazzi_version": "0.6.1", + "lpdid_alexcardazzi_commit": "ba64563983861be5e616f6020842c1a1cdf17a27", + "seed": 20260630, + "pre_window": 3, + "post_window": 4, + "n_strata": 4, + "n_psu_total": 14, + "record_format": "Each horizon / pooled row is c(estimate, SE, df, n_psu) for the tdiff coefficient.", + "se_convention": "survey::svyglm Binder (1983) Taylor-linearization sandwich. df = degf(design): n_PSU - n_strata (strata present, VW/covariate) or n_PSU - 1 (weights-only inject). Each horizon uses a FRESH svydesign over THAT horizon's clean LP-DiD long-difference sample (NOT subset() of a full design) - matching the library's per-sample resolution.", + "clean_sample_xcheck": "Clean-sample construction independently validated: the UNWEIGHTED VW event study (same prep/clean_h recipe) matches alexCardazzi::lpdid() to <1e-8. Selection is weight-independent, so this transitively validates the weighted svyglm samples.", + "point_gate": "Every svyglm WLS point validated == weighted feols(.. | time, weights, vcov=~unit) on the same sample to <1e-7.", + "fpc_note": "fpc is the per-stratum population PSU count (f_h = n_PSU_h / fpc_h); decoupled from the inverse-selection weight." + }, + "survey_vw_es": { + "0": [2.624112557921, 0.1849191460497, 10, 14], + "1": [2.593986037014, 0.2126289257184, 10, 14], + "2": [3.579941778181, 0.1691561916473, 10, 14], + "3": [4.008222869028, 0.250331727805, 10, 14], + "4": [3.882316481629, 0.1803068275608, 10, 14], + "-2": [-0.05518985932683, 0.1164062358211, 10, 14], + "-3": [-0.2829634020839, 0.1941367665957, 10, 14] + }, + "survey_vw_nofpc_es": { + "0": [2.624112557921, 0.2615151642859, 10, 14], + "1": [2.593986037014, 0.3007027105038, 10, 14], + "2": [3.579941778181, 0.239222980387, 10, 14], + "3": [4.008222869028, 0.3540225245542, 10, 14], + "4": [3.882316481629, 0.2549923609249, 10, 14], + "-2": [-0.05518985932683, 0.164623277443, 10, 14], + "-3": [-0.2829634020839, 0.2745508482749, 10, 14] + }, + "survey_inject_es": { + "0": [2.624112557921, 0.2261150339971, 139, 140], + "1": [2.593986037014, 0.2465200278538, 139, 140], + "2": [3.579941778181, 0.1967625151687, 139, 140], + "3": [4.008222869028, 0.2635749111488, 139, 140], + "4": [3.882316481629, 0.2716230812718, 139, 140], + "-2": [-0.05518985932683, 0.2152559315853, 139, 140], + "-3": [-0.2829634020839, 0.2495231661023, 139, 140] + }, + "survey_cov_es": { + "0": [2.647898017603, 0.1936410644549, 10, 14], + "1": [2.593494237233, 0.2131448643148, 10, 14], + "2": [3.577209310476, 0.1711510921869, 10, 14], + "3": [4.007497877233, 0.2516503255828, 10, 14], + "4": [3.878643429977, 0.1799400438737, 10, 14], + "-2": [-0.05792998040707, 0.1173984619161, 10, 14], + "-3": [-0.2864951132323, 0.1950637839557, 10, 14] + }, + "survey_vw_pooled": { + "post": [3.359136633265, 0.1779718319795, 10, 14], + "pre": [-0.1690766307054, 0.1518511686767, 10, 14] + }, + "survey_inject_pooled": { + "post": [3.359136633265, 0.1905486610436, 139, 140], + "pre": [-0.1690766307054, 0.2138738768941, 139, 140] + }, + "survey_cov_pooled": { + "post": [3.370150201558, 0.1834087011598, 10, 14], + "pre": [-0.1727322382533, 0.1528219754753, 10, 14] + } +} diff --git a/benchmarks/data/lpdid_survey_panel.csv b/benchmarks/data/lpdid_survey_panel.csv new file mode 100644 index 00000000..dff7c047 --- /dev/null +++ b/benchmarks/data/lpdid_survey_panel.csv @@ -0,0 +1,1681 @@ +unit,time,treat,y,x,stratum,psu,fpc,weight +1,1,0,-1.38626389953886,-1.3135499317869,1,1,6,5 +1,2,0,1.12729593622699,2.10825588110722,1,1,6,5 +1,3,0,-1.21881080327747,1.49137770159704,1,1,6,5 +1,4,0,-1.07685444495189,0.157923277640533,1,1,6,5 +1,5,1,-0.80615518377343,0.0228751186514433,1,1,6,5 +1,6,1,0.655033773694395,0.987848529455189,1,1,6,5 +1,7,1,3.22984629410565,0.0451957901985605,1,1,6,5 +1,8,1,2.08925127477244,0.218259959101494,1,1,6,5 +1,9,1,4.14578490328655,-0.2039275374331,1,1,6,5 +1,10,1,2.3566511206348,-0.787016666607241,1,1,6,5 +1,11,1,3.98547563311419,0.90799639790466,1,1,6,5 +1,12,1,6.24454570407614,0.974621980928353,1,1,6,5 +2,1,0,1.25164077259406,-1.55445230626851,1,2,6,5 +2,2,0,-0.237565938676358,0.129068981598153,1,2,6,5 +2,3,0,-1.02189202967346,-0.752766329777366,1,2,6,5 +2,4,0,2.79652418352618,1.2350919400775,1,2,6,5 +2,5,1,2.40888423295779,2.01540704862181,1,2,6,5 +2,6,1,1.58939356599588,-0.755153677013692,1,2,6,5 +2,7,1,5.15664308477853,0.55882290430771,1,2,6,5 +2,8,1,4.96645578210498,-0.580875408329384,1,2,6,5 +2,9,1,4.76102840531726,-1.19630439789651,1,2,6,5 +2,10,1,4.93174878597698,-0.0940123267129393,1,2,6,5 +2,11,1,5.26766490827512,1.46654688561033,1,2,6,5 +2,12,1,8.2175849582371,-0.0923233310359709,1,2,6,5 +3,1,0,0.788940970658124,0.00133718011086981,1,3,6,5 +3,2,0,0.933234199427541,0.385467184500205,1,3,6,5 +3,3,0,-1.27201248727308,0.200017723326383,1,3,6,5 +3,4,0,-0.954614330684006,0.987357960186799,1,3,6,5 +3,5,1,2.59383074009449,0.671754012399562,1,3,6,5 +3,6,1,2.32327621752308,-0.039794814666526,1,3,6,5 +3,7,1,4.58078396219799,0.335270273535526,1,3,6,5 +3,8,1,5.52463251674074,0.539498479892719,1,3,6,5 +3,9,1,6.19452557875635,1.22812250585706,1,3,6,5 +3,10,1,2.62377728557775,-0.68094956121789,1,3,6,5 +3,11,1,5.11136923751118,-0.909655896507865,1,3,6,5 +3,12,1,6.09244079385361,-1.30086960258696,1,3,6,5 +4,1,0,-2.65073636560161,-0.987214110161621,1,1,6,5 +4,2,0,-2.93754142648928,-0.922027277662055,1,1,6,5 +4,3,0,-2.92130558676419,1.2822178490463,1,1,6,5 +4,4,0,-2.79970457140895,0.649616688097705,1,1,6,5 +4,5,1,-2.44593293791085,-1.05896607582246,1,1,6,5 +4,6,1,-1.72640074454811,-0.344399403372918,1,1,6,5 +4,7,1,0.0321376702960825,-1.76873818781382,1,1,6,5 +4,8,1,2.23507055897547,1.66185942506711,1,1,6,5 +4,9,1,2.23621835753869,-0.193071537571491,1,1,6,5 +4,10,1,4.11104677196468,0.628879564961648,1,1,6,5 +4,11,1,2.60621240106839,0.838869054152405,1,1,6,5 +4,12,1,2.56208677165306,-0.318581410012423,1,1,6,5 +5,1,0,-0.0335023078258394,-1.01334329641896,1,2,6,5 +5,2,0,-1.65966951744739,-2.52580769416736,1,2,6,5 +5,3,0,-2.41054338990382,-0.0438352637339001,1,2,6,5 +5,4,0,-2.24422162949307,-0.734568970525196,1,2,6,5 +5,5,1,-0.446722415715389,-0.533062023588962,1,2,6,5 +5,6,1,1.18398804931463,0.965996498224262,1,2,6,5 +5,7,1,2.13693689891788,0.513631492024806,1,2,6,5 +5,8,1,2.14945199573916,2.12128040958557,1,2,6,5 +5,9,1,0.376397613672981,-0.210078658411475,1,2,6,5 +5,10,1,1.2687189180338,-1.2059191937365,1,2,6,5 +5,11,1,2.32162255858752,0.207531324837152,1,2,6,5 +5,12,1,4.97554831277228,-0.431300936872998,1,2,6,5 +6,1,0,1.63566067688661,0.0499488490963563,1,3,6,5 +6,2,0,0.447825135656469,0.0947881347327403,1,3,6,5 +6,3,0,-0.0899242442677647,-1.28474530277475,1,3,6,5 +6,4,0,2.9257807547962,1.9548223447702,1,3,6,5 +6,5,1,2.57612643749468,-1.56905959400362,1,3,6,5 +6,6,1,2.24491708155965,-1.22967545430202,1,3,6,5 +6,7,1,3.94064373305185,-2.09491069032714,1,3,6,5 +6,8,1,4.26553799631438,-0.967887977555318,1,3,6,5 +6,9,1,8.27434484517681,2.21916070920348,1,3,6,5 +6,10,1,6.20157953316336,0.118817049122968,1,3,6,5 +6,11,1,6.60330327042829,-0.274061652004027,1,3,6,5 +6,12,1,6.21723588291474,-0.777381016939534,1,3,6,5 +7,1,0,1.23959202327413,0.680407768892421,1,1,6,5 +7,2,0,-0.275285027456687,0.233859993819247,1,1,6,5 +7,3,0,-0.6152031314642,-0.36125847307684,1,1,6,5 +7,4,0,1.02245447881433,-1.33139480797428,1,1,6,5 +7,5,1,1.47184161199386,-1.41187059816485,1,1,6,5 +7,6,1,1.05446698739402,-0.696505781524492,1,1,6,5 +7,7,1,4.63259684238379,0.526307493104944,1,1,6,5 +7,8,1,3.80999658197122,0.827263719579844,1,1,6,5 +7,9,1,3.95247889806476,-0.150414696394337,1,1,6,5 +7,10,1,7.24494476326603,1.07489688411623,1,1,6,5 +7,11,1,5.30231185834874,-0.627009316691287,1,1,6,5 +7,12,1,7.80685983202879,1.15897332766322,1,1,6,5 +8,1,0,-0.407949653748393,-1.61856566264516,1,2,6,5 +8,2,0,-2.29258413839261,0.255293778809889,1,2,6,5 +8,3,0,-1.86818764761784,-0.178970961479152,1,2,6,5 +8,4,0,0.156062337238618,2.21024583549293,1,2,6,5 +8,5,1,3.02597103195531,2.49403705471386,1,2,6,5 +8,6,1,0.733756410143204,1.66052823115347,1,2,6,5 +8,7,1,2.13833667049091,-2.25012943099084,1,2,6,5 +8,8,1,1.1814848341121,0.543436532276745,1,2,6,5 +8,9,1,1.71457534803177,-0.770771127545265,1,2,6,5 +8,10,1,3.86908367655823,0.600680431862429,1,2,6,5 +8,11,1,3.40681403377368,0.942792628529139,1,2,6,5 +8,12,1,4.08034657712946,0.485965469697245,1,2,6,5 +9,1,0,0.495353495784443,-0.437769992864355,1,3,6,5 +9,2,0,-0.505103627914989,0.997846380179685,1,3,6,5 +9,3,0,0.756596853888305,1.88793413337212,1,3,6,5 +9,4,0,-0.627131785424889,-0.372174427395204,1,3,6,5 +9,5,0,-2.05957409746247,-0.177531513091506,1,3,6,5 +9,6,0,-0.637612132252226,1.72297685463312,1,3,6,5 +9,7,0,-0.637789350970227,-0.602072800203596,1,3,6,5 +9,8,1,2.68722122049789,-0.606995234612834,1,3,6,5 +9,9,1,2.33583785232604,-1.0175812520208,1,3,6,5 +9,10,1,1.58074767247119,-3.00717568436488,1,3,6,5 +9,11,1,3.01504269160305,-1.07701598846107,1,3,6,5 +9,12,1,4.15056113541715,-1.04411031581758,1,3,6,5 +10,1,0,3.65457287876064,1.68372275142753,1,1,6,5 +10,2,0,-0.82398671063205,-0.755796601388162,1,1,6,5 +10,3,0,-0.184173139636434,0.0932790705083469,1,1,6,5 +10,4,0,0.464667365612425,0.0665021573332019,1,1,6,5 +10,5,0,0.575566154544876,0.127734562272416,1,1,6,5 +10,6,0,0.259325970721236,-0.303910459774781,1,1,6,5 +10,7,0,2.82289021637001,-1.58162630710967,1,1,6,5 +10,8,1,4.47437475861707,0.738242722079021,1,1,6,5 +10,9,1,2.28875987812734,0.259188774267147,1,1,6,5 +10,10,1,3.9774474079429,0.789701372281211,1,1,6,5 +10,11,1,4.17409193441835,-0.691096758861132,1,1,6,5 +10,12,1,4.67079695940293,0.740413778047699,1,1,6,5 +11,1,0,0.0654361925804636,0.257696802097512,1,2,6,5 +11,2,0,-0.850240881440555,-0.28636683275855,1,2,6,5 +11,3,0,1.66355262924643,1.43854518966656,1,2,6,5 +11,4,0,-0.076944869490975,-1.50579415580442,1,2,6,5 +11,5,0,0.408573895732068,-1.16600926679119,1,2,6,5 +11,6,0,1.27597736266345,-0.701539277561293,1,2,6,5 +11,7,0,1.31263968212445,-1.20482832449922,1,2,6,5 +11,8,1,4.90254114660797,1.71342776967427,1,2,6,5 +11,9,1,3.70456588859809,0.207283305029853,1,2,6,5 +11,10,1,5.862638542933,-0.491672228743349,1,2,6,5 +11,11,1,5.98381148072377,0.372963300665636,1,2,6,5 +11,12,1,6.13579550389972,0.31617074761432,1,2,6,5 +12,1,0,-0.585139606913581,0.792836553282479,1,3,6,5 +12,2,0,-3.13762139044697,0.124444928679665,1,3,6,5 +12,3,0,-2.89030032287327,0.110722518495738,1,3,6,5 +12,4,0,-3.03251460558629,-0.382326616628464,1,3,6,5 +12,5,0,-3.03939524218956,1.0703375966054,1,3,6,5 +12,6,0,-2.37201312190822,0.501593270805825,1,3,6,5 +12,7,0,-1.86480902936143,-1.36272482527483,1,3,6,5 +12,8,1,0.326356940317275,-1.69635133051764,1,3,6,5 +12,9,1,0.741623865014607,-1.00028576662632,1,3,6,5 +12,10,1,-0.605469239542062,-2.47893587929364,1,3,6,5 +12,11,1,3.47284113992295,0.716348691552106,1,3,6,5 +12,12,1,3.09012754300084,-0.645595263719482,1,3,6,5 +13,1,0,-1.15299928507375,-1.25241497147032,1,1,6,5 +13,2,0,-0.410025119096197,0.597115401946667,1,1,6,5 +13,3,0,-1.15191711369501,-1.26510765372216,1,1,6,5 +13,4,0,-0.67524540428486,-1.29069017409734,1,1,6,5 +13,5,0,1.32335673147987,0.225389506505008,1,1,6,5 +13,6,0,-1.8982620845786,-1.62519302159512,1,1,6,5 +13,7,0,0.096667515751835,-1.22735032358374,1,1,6,5 +13,8,1,2.58593417371968,-0.659059182380201,1,1,6,5 +13,9,1,3.39664498088635,0.447569393719704,1,1,6,5 +13,10,1,6.17420144154308,0.102377719538326,1,1,6,5 +13,11,1,4.48887751278829,0.207575560573841,1,1,6,5 +13,12,1,5.89600855881732,0.597016971805994,1,1,6,5 +14,1,0,-3.44702922053276,0.537034735503177,1,2,6,5 +14,2,0,-3.61768474465986,-0.127667046602619,1,2,6,5 +14,3,0,-5.09987878406957,-0.830360895090306,1,2,6,5 +14,4,0,-5.94020887652718,-1.06730086576799,1,2,6,5 +14,5,0,-5.78467731948828,-0.451454700979868,1,2,6,5 +14,6,0,-4.7270389863068,0.952635357202591,1,2,6,5 +14,7,0,-3.77947259025181,-0.187617825903624,1,2,6,5 +14,8,1,-0.168589889275237,0.845968584445856,1,2,6,5 +14,9,1,0.027090116608848,1.25965199191368,1,2,6,5 +14,10,1,1.26127555423013,0.808472340198076,1,2,6,5 +14,11,1,-0.2378411630764,-0.801138014676762,1,2,6,5 +14,12,1,1.65326672552633,-0.00323928047906444,1,2,6,5 +15,1,0,-0.0785155648047304,0.239116533784949,1,3,6,5 +15,2,0,-0.724660650516545,-0.488545280346356,1,3,6,5 +15,3,0,0.922086461221053,0.517260651236455,1,3,6,5 +15,4,0,-1.72739257700572,-2.32131270549492,1,3,6,5 +15,5,0,-1.91632322109723,-2.33011675114673,1,3,6,5 +15,6,0,-1.5070258677308,-0.044880119291432,1,3,6,5 +15,7,0,-0.587570716240283,1.11948225601019,1,3,6,5 +15,8,1,0.134564744109702,1.36522938160032,1,3,6,5 +15,9,1,1.4310553944824,0.957264268418843,1,3,6,5 +15,10,1,1.62261252225556,0.728873308470783,1,3,6,5 +15,11,1,0.792075871176063,-1.20167742167458,1,3,6,5 +15,12,1,3.31441069558489,-0.612911529445006,1,3,6,5 +16,1,0,-5.43613378122228,-1.33245547055729,1,1,6,5 +16,2,0,-2.64041196760651,1.18134492554398,1,1,6,5 +16,3,0,-2.37613876686453,-0.035490479682145,1,1,6,5 +16,4,0,-1.89144764796255,0.212934146255184,1,1,6,5 +16,5,0,-4.4940365938487,-2.48301565937672,1,1,6,5 +16,6,0,-5.84552869833554,-0.313539046482724,1,1,6,5 +16,7,0,-3.92754584450337,-0.197270831166509,1,1,6,5 +16,8,0,-3.31570714848221,1.42450751790401,1,1,6,5 +16,9,0,-2.01452518173679,1.8316022535107,1,1,6,5 +16,10,0,-4.40599908794041,1.08043582253755,1,1,6,5 +16,11,0,-4.57746392923311,0.281456020736204,1,1,6,5 +16,12,0,-3.37455753382129,-0.480308135663231,1,1,6,5 +17,1,0,2.16155982328588,-1.45619212619908,1,2,6,5 +17,2,0,0.371180368006273,0.315743338549528,1,2,6,5 +17,3,0,-1.4234505263942,-1.2327804528081,1,2,6,5 +17,4,0,-0.768316060322765,-0.711604341159136,1,2,6,5 +17,5,0,1.93610473529701,0.578791701295279,1,2,6,5 +17,6,0,0.272967275921764,-0.233484399611085,1,2,6,5 +17,7,0,-0.511699800956326,0.957084629354032,1,2,6,5 +17,8,0,-1.73353671087592,-0.787469873082387,1,2,6,5 +17,9,0,-0.522791650395184,-1.58075410774737,1,2,6,5 +17,10,0,2.73097610549319,2.18940096931699,1,2,6,5 +17,11,0,1.19367301656062,0.223516344055554,1,2,6,5 +17,12,0,1.09287211826552,-1.51391978279101,1,2,6,5 +18,1,0,1.55306542191228,-0.408726946329875,1,3,6,5 +18,2,0,0.255511317238573,-1.90312831937017,1,3,6,5 +18,3,0,0.269544580043871,-0.371519046934758,1,3,6,5 +18,4,0,0.596080225203016,0.14047555142251,1,3,6,5 +18,5,0,-1.00612819878017,0.620181243368829,1,3,6,5 +18,6,0,-1.65016039924534,-0.594938851426038,1,3,6,5 +18,7,0,2.53086654751632,1.37874997766545,1,3,6,5 +18,8,0,-0.716393464057449,0.15050261927685,1,3,6,5 +18,9,0,-0.465040207684104,0.550865835071096,1,3,6,5 +18,10,0,-0.0110896185234469,-0.0330821290921424,1,3,6,5 +18,11,0,-1.67620766228039,-1.35597460031463,1,3,6,5 +18,12,0,1.09269488266065,0.0546329785590719,1,3,6,5 +19,1,0,1.94167207921793,-0.370776958850933,1,1,6,5 +19,2,0,-1.80940586245081,-0.46801900175922,1,1,6,5 +19,3,0,-1.48174699541597,2.39822839015917,1,1,6,5 +19,4,0,-1.99386342118969,-1.46749745429816,1,1,6,5 +19,5,0,-2.14383331638513,1.31332487028057,1,1,6,5 +19,6,0,-3.38561513218151,0.286622536793052,1,1,6,5 +19,7,0,-0.968979096429558,-1.81610827792953,1,1,6,5 +19,8,0,-1.35854927532848,0.498024198601236,1,1,6,5 +19,9,0,-2.03176282789413,-0.180194429507006,1,1,6,5 +19,10,0,-1.21101919942936,0.135189156977945,1,1,6,5 +19,11,0,0.985924735093641,2.58948236899122,1,1,6,5 +19,12,0,-1.9377140839464,-2.61951260293939,1,1,6,5 +20,1,0,2.82586370939532,1.22132215488224,1,2,6,5 +20,2,0,2.47326328906921,1.26574689128373,1,2,6,5 +20,3,0,1.13575255720677,0.492063717251712,1,2,6,5 +20,4,0,0.620608720215882,-0.891024308106338,1,2,6,5 +20,5,0,2.96654369802829,1.58294910881992,1,2,6,5 +20,6,0,1.04538713168091,-0.405083769400705,1,2,6,5 +20,7,0,0.61765703782352,-1.02505326708456,1,2,6,5 +20,8,0,1.06657652426972,0.847970926767299,1,2,6,5 +20,9,0,0.217597762150536,0.372120561041929,1,2,6,5 +20,10,0,-0.14231979028228,1.16175883723693,1,2,6,5 +20,11,0,0.832421618869553,-1.00471755356139,1,2,6,5 +20,12,0,0.577322192011873,-0.871436769930674,1,2,6,5 +21,1,0,8.21551145371209,2.01242181424621,1,3,6,5 +21,2,0,2.96966748037019,-0.821202069471798,1,3,6,5 +21,3,0,3.34289465644626,1.3734533884894,1,3,6,5 +21,4,0,4.42308840097668,0.177986500359851,1,3,6,5 +21,5,0,5.57231556179919,0.621463102888414,1,3,6,5 +21,6,0,2.97344301515656,0.0346831445605076,1,3,6,5 +21,7,0,1.6430474591317,0.569861970322157,1,3,6,5 +21,8,0,2.00931489224037,-0.244977585341447,1,3,6,5 +21,9,0,2.70715867524022,1.23591155109293,1,3,6,5 +21,10,0,1.93123482330749,-0.0118804655778225,1,3,6,5 +21,11,0,3.29383659766589,0.916820077681654,1,3,6,5 +21,12,0,2.88908750936815,0.0557631324577134,1,3,6,5 +22,1,0,0.980348132095585,-0.160051982765437,1,1,6,5 +22,2,0,-1.41243849875281,-0.645813878698489,1,1,6,5 +22,3,0,-3.11209538770158,0.0714674615644384,1,1,6,5 +22,4,0,-2.57003571773271,0.638637418889642,1,1,6,5 +22,5,0,-1.75537745276258,0.39334647646042,1,1,6,5 +22,6,0,-2.91635837002449,-1.68728915080945,1,1,6,5 +22,7,0,-0.910182934614133,0.456366355379913,1,1,6,5 +22,8,0,-0.312336515664553,-1.03179378188589,1,1,6,5 +22,9,0,1.17387881890032,2.37604910827253,1,1,6,5 +22,10,0,1.45897485131013,0.417315144877294,1,1,6,5 +22,11,0,0.255761544755975,0.378003239705332,1,1,6,5 +22,12,0,2.41877279428373,2.48809277844857,1,1,6,5 +23,1,0,3.39601071946489,-0.0829573486832863,1,2,6,5 +23,2,0,0.889929361876933,-0.0749944362368327,1,2,6,5 +23,3,0,1.86166799899182,0.704014529367211,1,2,6,5 +23,4,0,5.03588942074996,-0.313549755194671,1,2,6,5 +23,5,0,2.35664635183183,-0.615938409181139,1,2,6,5 +23,6,0,1.50067367468957,0.759748456259199,1,2,6,5 +23,7,0,2.13378126181527,-0.89595076209293,1,2,6,5 +23,8,0,1.27208424058629,-0.771824122298255,1,2,6,5 +23,9,0,2.84999558452354,0.908355565313566,1,2,6,5 +23,10,0,1.9568834715449,0.619858822983568,1,2,6,5 +23,11,0,1.01395593077525,0.477819257540968,1,2,6,5 +23,12,0,1.43611888864761,-1.61930781967232,1,2,6,5 +24,1,0,-2.74352709652206,0.264318157796839,1,3,6,5 +24,2,0,-3.00492749499502,0.295441774369184,1,3,6,5 +24,3,0,-3.63035090112717,-0.617100066340453,1,3,6,5 +24,4,0,-2.17177305339127,-1.41170746339888,1,3,6,5 +24,5,0,-1.63996687159069,1.12531666435794,1,3,6,5 +24,6,0,-4.18949640253611,-1.33590345578705,1,3,6,5 +24,7,0,-1.81976745622425,-1.62824038640261,1,3,6,5 +24,8,0,-2.83003471663392,-0.841121574890683,1,3,6,5 +24,9,0,-3.00234189986784,-0.0869294768139208,1,3,6,5 +24,10,0,-2.20094525237781,-1.00780922924579,1,3,6,5 +24,11,0,-1.37372055702777,0.575632443615413,1,3,6,5 +24,12,0,-2.548603165079,-1.2807093038317,1,3,6,5 +25,1,0,1.2534286660554,-0.790992544888179,1,1,6,5 +25,2,0,-0.430903972048297,0.591615809664827,1,1,6,5 +25,3,0,1.05058381108547,0.665325880566864,1,1,6,5 +25,4,0,0.845015412740891,0.229514239811065,1,1,6,5 +25,5,0,1.16227461636785,-1.44849179902118,1,1,6,5 +25,6,0,3.33385528938716,0.401199861661556,1,1,6,5 +25,7,0,3.46666387406927,-1.73617485468836,1,1,6,5 +25,8,0,0.111547928675555,-1.10679830993932,1,1,6,5 +25,9,0,1.16502963298915,-0.838616990698747,1,1,6,5 +25,10,0,3.89065106381815,1.21781681141148,1,1,6,5 +25,11,0,4.34422067964982,2.43465746696409,1,1,6,5 +25,12,0,0.0679151192571044,-1.66533311083068,1,1,6,5 +26,1,0,-0.665946438490313,-1.5124402563176,1,2,6,5 +26,2,0,-3.77276971853452,-1.24367147913353,1,2,6,5 +26,3,0,-2.34294006485323,-0.178541500388894,1,2,6,5 +26,4,0,-2.24098234305548,-0.218978427935179,1,2,6,5 +26,5,0,-0.998973724965991,0.0755112689626314,1,2,6,5 +26,6,0,-0.310926019222321,1.01633140327153,1,2,6,5 +26,7,0,-0.0977268960734357,-0.603855865066818,1,2,6,5 +26,8,0,-3.37409165019545,-1.50793436218323,1,2,6,5 +26,9,0,-0.158699871337861,0.395485388974682,1,2,6,5 +26,10,0,-2.70070874095882,-0.801383740796891,1,2,6,5 +26,11,0,-2.57695172319339,-0.594740562717965,1,2,6,5 +26,12,0,0.491941349695948,-0.225861813060612,1,2,6,5 +27,1,0,-1.95956709691088,0.415406782938761,1,3,6,5 +27,2,0,-4.2241590927531,-0.105824090283357,1,3,6,5 +27,3,0,-3.94698374788565,-0.806736558488228,1,3,6,5 +27,4,0,-0.720048568443123,1.08573678062099,1,3,6,5 +27,5,0,-3.99145280477241,-0.470475035120836,1,3,6,5 +27,6,0,-5.96033396993074,-0.382660762732075,1,3,6,5 +27,7,0,-3.43873688547095,-1.01388666345135,1,3,6,5 +27,8,0,-4.36909214469171,-0.70128627559675,1,3,6,5 +27,9,0,-4.7474252543793,-0.152477232499445,1,3,6,5 +27,10,0,-0.156797244016247,1.06104915638731,1,3,6,5 +27,11,0,-0.78518604646396,2.38030837148088,1,3,6,5 +27,12,0,-1.27876090463372,1.14569684948204,1,3,6,5 +28,1,0,-1.15406063092233,-1.23110808291999,1,1,6,5 +28,2,0,-3.7171006964466,-0.144915243727403,1,1,6,5 +28,3,0,-3.72645018167272,-1.46509229753814,1,1,6,5 +28,4,0,-1.75285433933232,-0.564729469005041,1,1,6,5 +28,5,0,-0.994229160578726,0.907623428761966,1,1,6,5 +28,6,0,-2.3966343403666,0.757926394329095,1,1,6,5 +28,7,0,-0.237544371760184,-0.886585343000575,1,1,6,5 +28,8,0,0.171761508248356,1.05748628750005,1,1,6,5 +28,9,0,-1.26749222017235,-0.231303269228853,1,1,6,5 +28,10,0,-0.67813026719101,2.12047295632347,1,1,6,5 +28,11,0,-6.38213407273758,-2.47693673003283,1,1,6,5 +28,12,0,-2.27171344775368,-0.435751014352884,1,1,6,5 +29,1,0,-0.12013439400619,0.783983275699386,1,2,6,5 +29,2,0,-3.61637344470972,-0.652439449794423,1,2,6,5 +29,3,0,-4.07760172642668,-0.769547437891371,1,2,6,5 +29,4,0,-1.49493550450862,-0.0374695909724359,1,2,6,5 +29,5,0,0.0951344209522429,1.86799448838253,1,2,6,5 +29,6,0,-1.60068571860658,-0.248766530653324,1,2,6,5 +29,7,0,-0.59975695889198,-0.684510869450771,1,2,6,5 +29,8,0,-3.04638789068391,0.195080349597133,1,2,6,5 +29,9,0,-2.74987823478165,0.787878913752439,1,2,6,5 +29,10,0,-2.56828704259925,0.343223036773658,1,2,6,5 +29,11,0,-2.03921941258,1.6826320949307,1,2,6,5 +29,12,0,-1.72370603638696,-0.961505039241722,1,2,6,5 +30,1,0,2.6896019843196,0.752138801572183,1,3,6,5 +30,2,0,-1.63564567315057,-1.21102313148031,1,3,6,5 +30,3,0,0.398355669313215,-0.868009244593217,1,3,6,5 +30,4,0,-0.469851705788858,-0.10929289743688,1,3,6,5 +30,5,0,-1.86569200929322,0.29723125465101,1,3,6,5 +30,6,0,-0.390610936251865,0.000658697789163809,1,3,6,5 +30,7,0,1.08402712405558,-0.44296841400422,1,3,6,5 +30,8,0,-0.469437042143857,-0.247991460346322,1,3,6,5 +30,9,0,-0.530745101507294,-0.620871459809177,1,3,6,5 +30,10,0,-2.14711379951786,-1.81009115272168,1,3,6,5 +30,11,0,1.70827576109975,0.764694220632458,1,3,6,5 +30,12,0,3.83223538233109,0.803357557231321,1,3,6,5 +31,1,0,4.62930439963058,1.22568414944988,2,4,6,10 +31,2,0,1.57450580151212,-0.249641557670862,2,4,6,10 +31,3,0,1.35712877138807,0.902312296077946,2,4,6,10 +31,4,0,2.67698212721147,0.0517289706619815,2,4,6,10 +31,5,1,5.44045777918077,0.56293104315589,2,4,6,10 +31,6,1,4.28124080273308,0.314447358356507,2,4,6,10 +31,7,1,7.40960473046884,1.94753632375463,2,4,6,10 +31,8,1,5.02350392411332,0.586084965656706,2,4,6,10 +31,9,1,6.51986226534334,1.8579359169226,2,4,6,10 +31,10,1,5.60159315772154,-0.172795621965769,2,4,6,10 +31,11,1,3.19515845509325,0.690112194107857,2,4,6,10 +31,12,1,5.68413660564671,-1.26450044020499,2,4,6,10 +32,1,0,-2.6658206127894,-4.30076982239583,2,5,6,10 +32,2,0,-2.63998910479441,-0.218796700793246,2,5,6,10 +32,3,0,-1.7266782347371,0.583069911912448,2,5,6,10 +32,4,0,-1.56773449092408,1.88957727958623,2,5,6,10 +32,5,1,-0.639181799474288,-0.658827504901617,2,5,6,10 +32,6,1,-0.0703134847375397,0.701595566123563,2,5,6,10 +32,7,1,1.56186774709568,0.0108916676853855,2,5,6,10 +32,8,1,2.2965857160184,-0.18596466192014,2,5,6,10 +32,9,1,2.81255135608302,0.961238964884225,2,5,6,10 +32,10,1,4.83419866794936,0.365044593905504,2,5,6,10 +32,11,1,5.26434134246583,0.629387893168104,2,5,6,10 +32,12,1,7.04590022032492,1.16406279351265,2,5,6,10 +33,1,0,0.597223799100332,-0.925004301218189,2,6,6,10 +33,2,0,-0.830143919633204,-1.17214082770913,2,6,6,10 +33,3,0,-1.56594602152452,-0.551141333529353,2,6,6,10 +33,4,0,-2.1864513669775,-0.19795746181197,2,6,6,10 +33,5,1,0.690190676784445,-0.0136787665212247,2,6,6,10 +33,6,1,-0.421164242898429,-1.15959500237879,2,6,6,10 +33,7,1,5.24254438359047,-0.090430242198213,2,6,6,10 +33,8,1,2.05812624561025,-2.24066512080142,2,6,6,10 +33,9,1,3.45505833988679,0.559395910412377,2,6,6,10 +33,10,1,3.67029378949547,-0.821065259608642,2,6,6,10 +33,11,1,5.73323932464537,0.176288007352978,2,6,6,10 +33,12,1,7.436780504573,1.92187100763958,2,6,6,10 +34,1,0,2.63444612352067,-0.779871630015671,2,4,6,10 +34,2,0,0.146066346084456,-1.33284420656638,2,4,6,10 +34,3,0,1.09547517410882,-0.355506650174012,2,4,6,10 +34,4,0,2.14618069934555,-1.42964426914964,2,4,6,10 +34,5,1,2.97225441924941,-0.732991194362542,2,4,6,10 +34,6,1,3.08165104691079,-0.7997516723862,2,4,6,10 +34,7,1,6.79385587028673,-0.587361240865307,2,4,6,10 +34,8,1,3.08956516250014,-1.54489479943175,2,4,6,10 +34,9,1,5.08553716139949,1.49211944700944,2,4,6,10 +34,10,1,5.53326309734336,0.228079189287925,2,4,6,10 +34,11,1,6.48895891786052,0.19184530509723,2,4,6,10 +34,12,1,6.89149433272659,-0.382280002644462,2,4,6,10 +35,1,0,0.885137163042625,2.11172305350165,2,5,6,10 +35,2,0,-3.79138420259872,-0.682161690216382,2,5,6,10 +35,3,0,-2.80969902764538,0.360450427121997,2,5,6,10 +35,4,0,-1.81291933595354,-0.543333440051568,2,5,6,10 +35,5,1,-0.0195694640649012,0.467282131199635,2,5,6,10 +35,6,1,0.268808823802571,0.74970282010405,2,5,6,10 +35,7,1,0.698956750040447,-1.61792263516921,2,5,6,10 +35,8,1,-0.311071056557315,0.935833392272969,2,5,6,10 +35,9,1,2.44271413639742,0.259107528927551,2,5,6,10 +35,10,1,3.00716617777686,0.89483563080272,2,5,6,10 +35,11,1,3.21577825470312,-0.0503822126031728,2,5,6,10 +35,12,1,4.80979653865087,0.851222899291525,2,5,6,10 +36,1,0,1.01763700845361,-1.05735514720405,2,6,6,10 +36,2,0,0.120074405533847,1.97009307628859,2,6,6,10 +36,3,0,-1.76121207726578,2.23567555585398,2,6,6,10 +36,4,0,-0.97160509238988,0.464608887010028,2,6,6,10 +36,5,1,1.55374357789742,1.23732611864284,2,6,6,10 +36,6,1,-3.0745330820963,-0.550650789250409,2,6,6,10 +36,7,1,3.23724969031362,1.35777622033321,2,6,6,10 +36,8,1,2.73299205610633,0.507360046625812,2,6,6,10 +36,9,1,0.861586472079809,-0.869053159782339,2,6,6,10 +36,10,1,1.43868430395039,0.08320212960353,2,6,6,10 +36,11,1,2.1065885309621,0.332504836415244,2,6,6,10 +36,12,1,4.13103586592907,-0.393039083886114,2,6,6,10 +37,1,0,-5.64785006060874,0.00387137006311778,2,4,6,10 +37,2,0,-7.04779658831036,-1.53541652403686,2,4,6,10 +37,3,0,-5.59807110908441,-0.0879236516428552,2,4,6,10 +37,4,0,-3.22842780990842,2.35174307333807,2,4,6,10 +37,5,1,-3.37347175077879,0.360917864670179,2,4,6,10 +37,6,1,-3.24586521897027,0.191370033692879,2,4,6,10 +37,7,1,0.78712512695284,0.923617620426934,2,4,6,10 +37,8,1,-2.8727352676969,-1.57342783558103,2,4,6,10 +37,9,1,-3.99821063416704,-1.43318729016443,2,4,6,10 +37,10,1,2.03637849382853,0.680851494769348,2,4,6,10 +37,11,1,0.811888514290342,0.796775290313884,2,4,6,10 +37,12,1,-0.291914655731162,0.810276446855439,2,4,6,10 +38,1,0,6.15121884158326,-0.263944822731625,2,5,6,10 +38,2,0,3.51359664910244,0.387752723925554,2,5,6,10 +38,3,0,4.24905741354882,0.141027869392039,2,5,6,10 +38,4,0,4.76537427247278,-1.09168047162578,2,5,6,10 +38,5,1,5.37899434213594,0.090767535119336,2,5,6,10 +38,6,1,5.69827206769291,-0.849194883303862,2,5,6,10 +38,7,1,7.30979967805388,0.121520359216735,2,5,6,10 +38,8,1,5.20490522661376,-1.43845159301199,2,5,6,10 +38,9,1,7.58096444626164,0.0212184034382198,2,5,6,10 +38,10,1,9.64039474275125,1.83105698848031,2,5,6,10 +38,11,1,6.24938764115095,0.352170618470425,2,5,6,10 +38,12,1,8.39055364425701,-0.483112676038422,2,5,6,10 +39,1,0,1.83011005676955,-0.610031348281283,2,6,6,10 +39,2,0,2.05699525769468,1.12775853566937,2,6,6,10 +39,3,0,0.0171818269524303,-1.32003899366432,2,6,6,10 +39,4,0,1.96214831172979,-0.855623868156108,2,6,6,10 +39,5,0,0.735775845751993,-1.33063545158513,2,6,6,10 +39,6,0,0.791118288115341,0.500391456608603,2,6,6,10 +39,7,0,1.04079199803234,-1.81501958471656,2,6,6,10 +39,8,1,2.40675045481726,-1.98543174040841,2,6,6,10 +39,9,1,1.76139422290566,-1.42424327653279,2,6,6,10 +39,10,1,6.62397631854989,0.947146320471241,2,6,6,10 +39,11,1,4.10716507236302,-0.495674664236506,2,6,6,10 +39,12,1,4.68858845181312,-1.34896804599038,2,6,6,10 +40,1,0,1.34705208350361,0.991351433843041,2,4,6,10 +40,2,0,-1.10493568748841,-0.866052492026707,2,4,6,10 +40,3,0,-2.80080024113126,-1.4874503897157,2,4,6,10 +40,4,0,0.753330796160008,1.22224415516848,2,4,6,10 +40,5,0,0.770585825515131,1.31912540056937,2,4,6,10 +40,6,0,-0.782123992921549,-0.162023898888156,2,4,6,10 +40,7,0,2.49147077317513,2.90703493258581,2,4,6,10 +40,8,1,1.35601050775493,-1.15680206818065,2,4,6,10 +40,9,1,3.69250051605495,0.493394471554285,2,4,6,10 +40,10,1,3.66960239732568,-0.208876880043165,2,4,6,10 +40,11,1,4.13296640801305,0.368228835086566,2,4,6,10 +40,12,1,5.4549061010279,-0.213378441672674,2,4,6,10 +41,1,0,3.8776301636341,-0.0934086419481362,2,5,6,10 +41,2,0,4.55092237428491,0.223131395262917,2,5,6,10 +41,3,0,1.17229660703989,-1.07010688123241,2,5,6,10 +41,4,0,1.13521339666238,-0.553353296576916,2,5,6,10 +41,5,0,2.00399597382308,-0.25498097506778,2,5,6,10 +41,6,0,2.56180042878894,-0.874891282859066,2,5,6,10 +41,7,0,4.27498733478503,0.162051680043711,2,5,6,10 +41,8,1,3.65198242649501,-0.635004062529328,2,5,6,10 +41,9,1,4.94553018729598,0.990765381727307,2,5,6,10 +41,10,1,6.76719911685584,1.17783035330695,2,5,6,10 +41,11,1,3.09802651194082,-1.3577137211389,2,5,6,10 +41,12,1,6.8449059115133,-1.56126999326562,2,5,6,10 +42,1,0,1.80092201238215,-0.209763112063332,2,6,6,10 +42,2,0,1.11399413453327,-0.524491113838345,2,6,6,10 +42,3,0,2.25477112329967,1.66938021135657,2,6,6,10 +42,4,0,0.759073206727878,0.187373841229865,2,6,6,10 +42,5,0,-0.954772217593505,-0.799610367231592,2,6,6,10 +42,6,0,-1.47859949036146,-0.216304703629375,2,6,6,10 +42,7,0,1.0022334343266,-1.60232723363584,2,6,6,10 +42,8,1,4.01562269347261,0.637600739512775,2,6,6,10 +42,9,1,5.364296099207,1.07311755369469,2,6,6,10 +42,10,1,4.69003253032622,-0.166459367297593,2,6,6,10 +42,11,1,4.34818231484735,0.259704937188028,2,6,6,10 +42,12,1,5.15021613842362,-1.28454928217865,2,6,6,10 +43,1,0,5.35941425355595,0.0755449741711561,2,4,6,10 +43,2,0,1.58311662965847,-0.856829096877277,2,4,6,10 +43,3,0,3.7973138970102,0.263089052659923,2,4,6,10 +43,4,0,4.27592915106978,-0.204217532183614,2,4,6,10 +43,5,0,2.96122976724267,-0.0102021595262646,2,4,6,10 +43,6,0,2.16955444919076,0.889930608061938,2,4,6,10 +43,7,0,2.9187483536401,-0.926468188074741,2,4,6,10 +43,8,1,5.57943140577293,-0.229310085015384,2,4,6,10 +43,9,1,6.35657527774658,0.713597616820147,2,4,6,10 +43,10,1,7.5267678687919,-0.161797502747793,2,4,6,10 +43,11,1,6.00561688740712,1.14443934679838,2,4,6,10 +43,12,1,7.46112489069362,-0.874440605586912,2,4,6,10 +44,1,0,0.737434607133817,0.961288009705673,2,5,6,10 +44,2,0,-1.67574943641202,-0.765010157102516,2,5,6,10 +44,3,0,-1.43338241855225,-0.366255395930872,2,5,6,10 +44,4,0,-1.44927853938578,-0.175510794323996,2,5,6,10 +44,5,0,-1.50546501402123,-0.196452211119201,2,5,6,10 +44,6,0,-3.24573262680896,-1.29684706720484,2,5,6,10 +44,7,0,-0.428405175115839,-0.674160434574383,2,5,6,10 +44,8,1,1.08427542607446,0.428405040079777,2,5,6,10 +44,9,1,-0.34469089146202,-1.22255911403312,2,5,6,10 +44,10,1,0.372576938454066,-1.99607624407189,2,5,6,10 +44,11,1,2.8435722565843,0.660559043444207,2,5,6,10 +44,12,1,2.83052301642067,0.376968116163029,2,5,6,10 +45,1,0,-3.46499378112937,-0.832866995826521,2,6,6,10 +45,2,0,-3.83262333902293,0.103217214301721,2,6,6,10 +45,3,0,-5.39229950342605,0.356569641358976,2,6,6,10 +45,4,0,-4.89461165396551,0.109945956098481,2,6,6,10 +45,5,0,-4.00520379003736,1.14854813103154,2,6,6,10 +45,6,0,-6.54157118561463,-0.718643686054332,2,6,6,10 +45,7,0,-3.94190480798119,-1.16717301972229,2,6,6,10 +45,8,1,-1.41613374010738,-0.804140660652461,2,6,6,10 +45,9,1,1.05118206831315,1.67059710935496,2,6,6,10 +45,10,1,0.887454889292409,-1.53709714181161,2,6,6,10 +45,11,1,0.065592822527655,-0.226626796491908,2,6,6,10 +45,12,1,4.15107006778227,0.868910340790676,2,6,6,10 +46,1,0,2.28814412797963,-0.48876950607341,2,4,6,10 +46,2,0,0.955131895075971,0.377865495480243,2,4,6,10 +46,3,0,2.48008854251602,0.578113728676265,2,4,6,10 +46,4,0,0.119580745794421,-0.212784490065249,2,4,6,10 +46,5,0,0.184394974233475,-0.701448712171642,2,4,6,10 +46,6,0,-0.614209734394238,-1.30123032625959,2,4,6,10 +46,7,0,5.94513011536437,2.14585153608781,2,4,6,10 +46,8,0,3.51860426914898,0.227583678840495,2,4,6,10 +46,9,0,0.328993258762623,0.167271080672443,2,4,6,10 +46,10,0,1.32967653944064,-0.609388909960726,2,4,6,10 +46,11,0,0.900547524313149,0.782276623034455,2,4,6,10 +46,12,0,3.48875069697673,0.102270408925818,2,4,6,10 +47,1,0,0.550111082846736,0.292966679919445,2,5,6,10 +47,2,0,-0.937858935055821,-1.6262095591201,2,5,6,10 +47,3,0,-1.12454320849378,-0.597249992271243,2,5,6,10 +47,4,0,-0.875162638974692,0.773956368014307,2,5,6,10 +47,5,0,-1.12175561222632,-0.430407950883949,2,5,6,10 +47,6,0,-1.04861829209457,0.187145388313954,2,5,6,10 +47,7,0,0.373456562313112,-0.386163021913562,2,5,6,10 +47,8,0,0.671049464743089,1.29005702950842,2,5,6,10 +47,9,0,-1.95158835267807,-0.0844400003007866,2,5,6,10 +47,10,0,-2.07961706821486,0.460290539284762,2,5,6,10 +47,11,0,0.769702161034667,0.12568136147485,2,5,6,10 +47,12,0,0.163216951151574,0.807210843301256,2,5,6,10 +48,1,0,2.73429159291996,-0.209501058619523,2,6,6,10 +48,2,0,2.12993633039594,0.139291098582805,2,6,6,10 +48,3,0,1.86016908115008,0.792912315300734,2,6,6,10 +48,4,0,3.76528540052586,1.27559673871557,2,6,6,10 +48,5,0,1.9546992042884,0.648898505811777,2,6,6,10 +48,6,0,0.321641525764133,-0.173957772890344,2,6,6,10 +48,7,0,3.60242245198644,0.332248614236929,2,6,6,10 +48,8,0,1.42956205853225,-0.652272469403883,2,6,6,10 +48,9,0,0.556302616067611,-0.418833670320762,2,6,6,10 +48,10,0,0.892464014015608,-0.32074003926927,2,6,6,10 +48,11,0,1.74363803674417,0.135661783814361,2,6,6,10 +48,12,0,3.29827750512419,-1.03494257593257,2,6,6,10 +49,1,0,1.19106105686346,0.119465666463643,2,4,6,10 +49,2,0,-2.26323258584129,-1.22115474847692,2,4,6,10 +49,3,0,-0.619589661122987,1.02294326395076,2,4,6,10 +49,4,0,1.7746951501938,1.53588152000959,2,4,6,10 +49,5,0,1.43762694217356,0.701451175986176,2,4,6,10 +49,6,0,1.30944654537829,0.917786699669186,2,4,6,10 +49,7,0,1.16227051122745,0.303052169460591,2,4,6,10 +49,8,0,1.73001633494617,1.42004551397111,2,4,6,10 +49,9,0,-0.919528678054176,0.0270253865827736,2,4,6,10 +49,10,0,-0.0204265997163205,1.10470372380987,2,4,6,10 +49,11,0,-2.43758046967728,0.105300053320683,2,4,6,10 +49,12,0,-1.17954183612224,0.237084444389962,2,4,6,10 +50,1,0,-1.2237390338933,-0.913637514535184,2,5,6,10 +50,2,0,-5.52032687372575,-1.77589518640937,2,5,6,10 +50,3,0,-4.95970452716219,-0.125447284949749,2,5,6,10 +50,4,0,-4.87285580918724,-2.1499110281147,2,5,6,10 +50,5,0,-2.25810741930194,0.496785447066993,2,5,6,10 +50,6,0,-2.79573227559087,-0.665689133494011,2,5,6,10 +50,7,0,-2.61259466908699,-1.05062661584569,2,5,6,10 +50,8,0,-4.1030880956717,-0.671297592694655,2,5,6,10 +50,9,0,-3.16285759909069,-0.293437735148757,2,5,6,10 +50,10,0,-4.3156248417843,-2.27211669827077,2,5,6,10 +50,11,0,-1.0350411131232,2.88452517371576,2,5,6,10 +50,12,0,0.425930965062977,0.582097541327285,2,5,6,10 +51,1,0,1.42851820207103,1.20715061484251,2,6,6,10 +51,2,0,0.252391625936328,0.694394230020628,2,6,6,10 +51,3,0,0.630601048523339,0.779726256290475,2,6,6,10 +51,4,0,1.80843712623179,0.305813792122512,2,6,6,10 +51,5,0,1.09851093712324,1.77398252523838,2,6,6,10 +51,6,0,0.168243854504139,0.865146007250579,2,6,6,10 +51,7,0,2.51236236223871,0.807792758236615,2,6,6,10 +51,8,0,0.584349863814407,-1.42169484360022,2,6,6,10 +51,9,0,2.77217549452801,1.87840278641197,2,6,6,10 +51,10,0,0.245683308921452,-1.78198327163948,2,6,6,10 +51,11,0,0.141613450392089,-0.304555243273393,2,6,6,10 +51,12,0,2.76246062322894,0.564077274760085,2,6,6,10 +52,1,0,0.6049500590238,0.275983561647424,2,4,6,10 +52,2,0,0.13426878488851,-0.881207546101581,2,4,6,10 +52,3,0,1.08314731343338,-0.433375094644175,2,4,6,10 +52,4,0,0.818737844584329,-1.48820633554211,2,4,6,10 +52,5,0,0.585020135695307,0.250031988808035,2,4,6,10 +52,6,0,-2.93387248259616,-1.41210275073953,2,4,6,10 +52,7,0,-0.628551868259646,-0.263431982561321,2,4,6,10 +52,8,0,-1.87342002376069,-0.00218017847454671,2,4,6,10 +52,9,0,-0.459877245674715,-0.467124050631303,2,4,6,10 +52,10,0,0.979378054237498,1.02742781923724,2,4,6,10 +52,11,0,-3.40661005116716,-0.475676189081214,2,4,6,10 +52,12,0,-0.263264854267651,1.62198849182792,2,4,6,10 +53,1,0,-4.9636266208748,0.278530361995648,2,5,6,10 +53,2,0,-7.11502960311436,0.212734674207425,2,5,6,10 +53,3,0,-6.36255022949667,0.247184492784305,2,5,6,10 +53,4,0,-6.93178382555849,-1.9971251667526,2,5,6,10 +53,5,0,-3.86057771452394,0.931427655285951,2,5,6,10 +53,6,0,-4.08409629415534,0.345235728627526,2,5,6,10 +53,7,0,-4.46038488386243,-0.297179881733295,2,5,6,10 +53,8,0,-7.94272032740693,-0.711262659226801,2,5,6,10 +53,9,0,-7.27595902456124,-0.184183885169805,2,5,6,10 +53,10,0,-5.28303944711437,0.178521558658652,2,5,6,10 +53,11,0,-7.90573253725956,-1.94990267171316,2,5,6,10 +53,12,0,-5.42593738184656,0.817936403741587,2,5,6,10 +54,1,0,1.25496834774839,-0.947241703099275,2,6,6,10 +54,2,0,-0.0825294132541066,0.089296550579631,2,6,6,10 +54,3,0,0.0989621364443327,2.04249513347967,2,6,6,10 +54,4,0,0.108519894898947,0.763211532981165,2,6,6,10 +54,5,0,-1.58042287283426,-0.0945099666320701,2,6,6,10 +54,6,0,-4.29226428973034,-1.48057837034857,2,6,6,10 +54,7,0,0.133070646575895,-0.469266797387668,2,6,6,10 +54,8,0,-1.94905653446796,-0.624167476280469,2,6,6,10 +54,9,0,-1.04253990052309,-0.137710058549071,2,6,6,10 +54,10,0,0.270982540747865,-0.122304119613902,2,6,6,10 +54,11,0,-1.23008149012158,-0.20460413645823,2,6,6,10 +54,12,0,0.317983135918715,-0.88436856078177,2,6,6,10 +55,1,0,2.84577542668369,0.621448709606518,2,4,6,10 +55,2,0,0.833383798668622,0.0982977444335708,2,4,6,10 +55,3,0,0.702713134799146,1.25991839311786,2,4,6,10 +55,4,0,3.08242245446904,0.667791835633983,2,4,6,10 +55,5,0,-0.385843475116102,-1.24034463376107,2,4,6,10 +55,6,0,1.81182410203257,0.994508816863761,2,4,6,10 +55,7,0,2.35782012655351,-2.19290614342347,2,4,6,10 +55,8,0,2.04157847335275,-0.688352259543151,2,4,6,10 +55,9,0,0.776569352366177,-0.88646791527235,2,4,6,10 +55,10,0,2.77366125346806,-0.137449955509944,2,4,6,10 +55,11,0,-0.805246887817553,-1.30494355650219,2,4,6,10 +55,12,0,3.36647353801262,0.792541412387489,2,4,6,10 +56,1,0,0.917922479897992,1.87055831854105,2,5,6,10 +56,2,0,-0.855751789640014,-0.186455640547479,2,5,6,10 +56,3,0,-3.40570606741339,-0.452510410649688,2,5,6,10 +56,4,0,-2.6622824702082,-0.627216060454739,2,5,6,10 +56,5,0,-3.582899953463,-0.969539214247689,2,5,6,10 +56,6,0,-3.87923745020641,-0.581521041822565,2,5,6,10 +56,7,0,0.413572610639712,-0.943960323804696,2,5,6,10 +56,8,0,-0.231528843584746,1.63291985333668,2,5,6,10 +56,9,0,-0.87888152171763,2.71086562357487,2,5,6,10 +56,10,0,-3.921570395387,-0.133486741367525,2,5,6,10 +56,11,0,-4.09053146023709,-1.72631671285445,2,5,6,10 +56,12,0,-2.29647315352723,-0.0263500046417098,2,5,6,10 +57,1,0,4.25202604952836,-0.969781400386929,2,6,6,10 +57,2,0,3.27606741077581,-0.425885382557458,2,6,6,10 +57,3,0,5.16448193566541,1.53882004824385,2,6,6,10 +57,4,0,5.15166940979002,1.62014361887431,2,6,6,10 +57,5,0,4.28844107700812,1.11316816395635,2,6,6,10 +57,6,0,3.35976730851577,-0.551711360821471,2,6,6,10 +57,7,0,7.45276736977797,-0.0647390589874436,2,6,6,10 +57,8,0,4.78391763125636,-0.174298166814524,2,6,6,10 +57,9,0,6.62362570723212,1.53586193978532,2,6,6,10 +57,10,0,5.28063003400969,1.68606452326006,2,6,6,10 +57,11,0,4.72668722207607,0.654128832481561,2,6,6,10 +57,12,0,8.55795188306486,1.0641316207577,2,6,6,10 +58,1,0,0.968557491212324,0.781830025820619,2,4,6,10 +58,2,0,-0.941601053467728,0.115794273932226,2,4,6,10 +58,3,0,-2.03284402258048,-1.13784796669886,2,4,6,10 +58,4,0,-0.899208624611643,-0.736440428392409,2,4,6,10 +58,5,0,-1.56241555733021,0.0662679894304959,2,4,6,10 +58,6,0,-1.72737931144763,0.562889443746487,2,4,6,10 +58,7,0,2.55116278510092,0.813400849454336,2,4,6,10 +58,8,0,-0.114649743522915,-1.12562672069473,2,4,6,10 +58,9,0,-2.05281728141087,-1.07393656676804,2,4,6,10 +58,10,0,-1.98790016078581,-1.58539019860405,2,4,6,10 +58,11,0,-2.7722688620159,0.3295910198388,2,4,6,10 +58,12,0,0.416767108558687,2.2034678005628,2,4,6,10 +59,1,0,2.51260361868293,0.643859700095486,2,5,6,10 +59,2,0,0.626539297506655,0.422033709047741,2,5,6,10 +59,3,0,-0.0276949040569273,-0.280055713363277,2,5,6,10 +59,4,0,1.87616160411769,1.58572368121095,2,5,6,10 +59,5,0,0.853401165782878,0.00227303398815268,2,5,6,10 +59,6,0,0.573264036198065,0.650813890393782,2,5,6,10 +59,7,0,2.97503859885853,0.688086919025809,2,5,6,10 +59,8,0,0.00793452121329868,0.176565427835819,2,5,6,10 +59,9,0,-0.200560187725254,-0.0419804906561749,2,5,6,10 +59,10,0,-1.80398364778076,-1.25003542198616,2,5,6,10 +59,11,0,1.48836785315189,2.45295629973547,2,5,6,10 +59,12,0,1.24305689663305,-0.382467345998404,2,5,6,10 +60,1,0,3.2394575438791,-0.0529334643025248,2,6,6,10 +60,2,0,4.11244310828499,2.30453815196841,2,6,6,10 +60,3,0,3.91468219455145,0.518775580401688,2,6,6,10 +60,4,0,3.77845515759354,-0.597182071005081,2,6,6,10 +60,5,0,3.64679874693664,0.49946598819741,2,6,6,10 +60,6,0,2.27553290950247,0.342476729164942,2,6,6,10 +60,7,0,3.13017782603357,-1.76420045039301,2,6,6,10 +60,8,0,1.97728490892632,0.287363937862971,2,6,6,10 +60,9,0,-0.321974843798381,-1.10969933732966,2,6,6,10 +60,10,0,4.25893523068077,1.11994856883698,2,6,6,10 +60,11,0,4.57488016420469,1.48998076336496,2,6,6,10 +60,12,0,2.72065837163731,-1.32536891696064,2,6,6,10 +61,1,0,1.05026485839174,-1.0978742865135,3,7,8,7.5 +61,2,0,-1.49565025125651,-0.684941871653788,3,7,8,7.5 +61,3,0,-0.270390288481742,0.00471315171482735,3,7,8,7.5 +61,4,0,0.679106673348703,0.146851698151307,3,7,8,7.5 +61,5,1,1.97481250148186,0.650401109801506,3,7,8,7.5 +61,6,1,1.23871386102486,-0.140826425471018,3,7,8,7.5 +61,7,1,5.98690698315584,1.83117838300152,3,7,8,7.5 +61,8,1,5.29028968629073,-0.18069463030609,3,7,8,7.5 +61,9,1,5.27523807498567,-1.04320718856848,3,7,8,7.5 +61,10,1,7.73122512656222,-0.961199365668545,3,7,8,7.5 +61,11,1,6.67622201139213,0.751073521197966,3,7,8,7.5 +61,12,1,9.0949420383096,1.79744728027712,3,7,8,7.5 +62,1,0,-2.75039849057242,-0.291775552142069,3,8,8,7.5 +62,2,0,-4.02658373372531,0.336718906825672,3,8,8,7.5 +62,3,0,-5.36917417435456,-0.929866113797397,3,8,8,7.5 +62,4,0,-5.02488294573015,2.2098557103772,3,8,8,7.5 +62,5,1,-1.75150151255685,0.483860921958389,3,8,8,7.5 +62,6,1,-1.68555608657711,0.170938833994252,3,8,8,7.5 +62,7,1,0.459231637627386,-1.13211401647928,3,8,8,7.5 +62,8,1,-0.745946506432926,0.546929561050279,3,8,8,7.5 +62,9,1,-1.04301771213243,0.756052057884154,3,8,8,7.5 +62,10,1,-0.20450539804438,-0.495450282085226,3,8,8,7.5 +62,11,1,-0.159091636904264,-0.0234881277881527,3,8,8,7.5 +62,12,1,1.5926409839379,-0.391656678284422,3,8,8,7.5 +63,1,0,-5.59600006689879,0.155092630668752,3,9,8,7.5 +63,2,0,-6.06526423620119,0.407224193927132,3,9,8,7.5 +63,3,0,-5.33944970468206,0.0825976075006475,3,9,8,7.5 +63,4,0,-5.81776740005756,0.924636308445181,3,9,8,7.5 +63,5,1,-6.09500238662301,-0.193356675256831,3,9,8,7.5 +63,6,1,-5.23975013965662,0.751308262619707,3,9,8,7.5 +63,7,1,-3.44548467352474,0.292974809804221,3,9,8,7.5 +63,8,1,-5.07252282597296,-0.0286711591483,3,9,8,7.5 +63,9,1,-2.18297103702397,1.64049335525045,3,9,8,7.5 +63,10,1,-1.14763958606672,0.998474264857098,3,9,8,7.5 +63,11,1,-1.02851700203427,-0.628945499426337,3,9,8,7.5 +63,12,1,-0.390041651424732,-0.467561620376353,3,9,8,7.5 +64,1,0,0.512188281508519,-0.310232042925711,3,10,8,7.5 +64,2,0,-2.34392030614628,-2.38578499099583,3,10,8,7.5 +64,3,0,0.408207529111278,-0.122122096126638,3,10,8,7.5 +64,4,0,1.84954883752332,1.66736860789238,3,10,8,7.5 +64,5,1,2.16862151640129,0.802065343471153,3,10,8,7.5 +64,6,1,1.75564295220067,-0.582347489737063,3,10,8,7.5 +64,7,1,5.65863983026483,0.845376073502711,3,10,8,7.5 +64,8,1,3.74111848721981,0.243958389613559,3,10,8,7.5 +64,9,1,3.80857963282181,-1.13036599171894,3,10,8,7.5 +64,10,1,4.32145606042632,-0.265875505732601,3,10,8,7.5 +64,11,1,5.10989511170924,-0.0221973675662347,3,10,8,7.5 +64,12,1,3.87799521154341,-2.14105432731753,3,10,8,7.5 +65,1,0,0.651745909215134,2.83856003590043,3,7,8,7.5 +65,2,0,-1.24295155470817,0.595674445088334,3,7,8,7.5 +65,3,0,-1.85249824501286,0.837539701748233,3,7,8,7.5 +65,4,0,-2.43475333144537,-0.285002441505322,3,7,8,7.5 +65,5,1,-1.65581912764179,-0.380992063490334,3,7,8,7.5 +65,6,1,-3.05101232066507,-1.20973740381635,3,7,8,7.5 +65,7,1,1.83473926403851,-0.249643838203791,3,7,8,7.5 +65,8,1,0.340485156360671,0.939356169885019,3,7,8,7.5 +65,9,1,0.0472432958832796,0.423623362736208,3,7,8,7.5 +65,10,1,1.48866333153803,0.548168005573687,3,7,8,7.5 +65,11,1,4.07460413923956,1.46647548415762,3,7,8,7.5 +65,12,1,3.51659076645306,0.976806432382968,3,7,8,7.5 +66,1,0,3.63289840798606,-0.154854181918353,3,8,8,7.5 +66,2,0,0.66451422999007,0.501567307968257,3,8,8,7.5 +66,3,0,0.315649879980107,0.351488951039766,3,8,8,7.5 +66,4,0,1.3714824642386,-1.32510911880125,3,8,8,7.5 +66,5,1,4.02728042778208,0.016622080344879,3,8,8,7.5 +66,6,1,2.09182779004853,-0.232393987726689,3,8,8,7.5 +66,7,1,5.64211019678888,0.85186954090976,3,8,8,7.5 +66,8,1,4.01811714036981,0.330982913218066,3,8,8,7.5 +66,9,1,2.74436599231645,0.661780367099985,3,8,8,7.5 +66,10,1,4.26535221732333,-0.896276805691153,3,8,8,7.5 +66,11,1,4.87659539164704,-0.809386125035862,3,8,8,7.5 +66,12,1,7.47160461619938,1.16409929328737,3,8,8,7.5 +67,1,0,1.70049977772939,-0.387871131139879,3,9,8,7.5 +67,2,0,2.24646438610075,-0.347479632553136,3,9,8,7.5 +67,3,0,2.68741186647599,-2.0574402737734,3,9,8,7.5 +67,4,0,2.27641318105949,0.160528094980677,3,9,8,7.5 +67,5,1,5.77290592377406,0.393931377867569,3,9,8,7.5 +67,6,1,4.4408622639138,0.0249522267709038,3,9,8,7.5 +67,7,1,6.25422702780705,-0.980240025868972,3,9,8,7.5 +67,8,1,4.42153167609815,-0.936518263785264,3,9,8,7.5 +67,9,1,6.61493433727615,1.00861363414453,3,9,8,7.5 +67,10,1,7.47991446992362,1.41793137679492,3,9,8,7.5 +67,11,1,4.82390786096475,0.35365806066863,3,9,8,7.5 +67,12,1,7.88711937217639,1.49532010728255,3,9,8,7.5 +68,1,0,0.848554290497059,2.10064245619043,3,10,8,7.5 +68,2,0,-4.83402351892677,0.198967026305288,3,10,8,7.5 +68,3,0,-5.94856396030859,-1.51182686164139,3,10,8,7.5 +68,4,0,-2.06257671953402,1.42454172679911,3,10,8,7.5 +68,5,1,-1.62195090108143,1.48757186317004,3,10,8,7.5 +68,6,1,-0.701727355793481,-0.703054887832928,3,10,8,7.5 +68,7,1,3.77896033738531,1.04697121094763,3,10,8,7.5 +68,8,1,2.50411719869062,1.69391191503762,3,10,8,7.5 +68,9,1,2.40222029855148,1.68844066373689,3,10,8,7.5 +68,10,1,2.08474138573839,0.0125042396272262,3,10,8,7.5 +68,11,1,3.40952178909183,0.0892655400340264,3,10,8,7.5 +68,12,1,5.26677769834604,0.114145866194755,3,10,8,7.5 +69,1,0,-1.71513951456219,-0.599761929791316,3,7,8,7.5 +69,2,0,-2.7141892777831,0.51516724660808,3,7,8,7.5 +69,3,0,-3.01931889848197,-0.294121485943102,3,7,8,7.5 +69,4,0,-1.82718475910487,-0.354133304933244,3,7,8,7.5 +69,5,1,-0.866412119048173,0.638792669609837,3,7,8,7.5 +69,6,1,-0.817836773692864,-0.946031520570451,3,7,8,7.5 +69,7,1,3.22500795109553,0.27422757730705,3,7,8,7.5 +69,8,1,-0.393562279934702,-1.74028538196609,3,7,8,7.5 +69,9,1,1.84422523469206,0.274825783996721,3,7,8,7.5 +69,10,1,3.04609638423579,-0.0456503480261216,3,7,8,7.5 +69,11,1,4.14349617042585,1.80342402443077,3,7,8,7.5 +69,12,1,5.96592835566127,1.73249234234999,3,7,8,7.5 +70,1,0,0.209400757834089,-0.251570086837695,3,8,8,7.5 +70,2,0,-0.275283982409059,-1.53229144363193,3,8,8,7.5 +70,3,0,-0.314952410484268,1.11826231757194,3,8,8,7.5 +70,4,0,-1.43605927975428,-0.491291862353841,3,8,8,7.5 +70,5,1,4.25953746612045,0.514889017733017,3,8,8,7.5 +70,6,1,2.65991466220465,-0.251598022464193,3,8,8,7.5 +70,7,1,3.69665484233942,0.561351410490886,3,8,8,7.5 +70,8,1,1.88126969697473,-0.392739589117412,3,8,8,7.5 +70,9,1,2.70368103287291,-1.41598638537172,3,8,8,7.5 +70,10,1,5.49400653952046,1.29505797037073,3,8,8,7.5 +70,11,1,5.94338089875575,0.436859806235047,3,8,8,7.5 +70,12,1,7.32763087256214,-1.54329424672373,3,8,8,7.5 +71,1,0,2.87713513981302,-1.09110066338341,3,9,8,7.5 +71,2,0,4.36081173473622,0.502824155618913,3,9,8,7.5 +71,3,0,1.60085232074419,-1.11748328712601,3,9,8,7.5 +71,4,0,2.88283058200991,0.0999108778106402,3,9,8,7.5 +71,5,0,1.82579473692054,0.13258187995232,3,9,8,7.5 +71,6,0,2.87502936209804,0.530986031349087,3,9,8,7.5 +71,7,0,5.57638898687395,0.468418408409433,3,9,8,7.5 +71,8,1,10.3785443959842,2.20777636217524,3,9,8,7.5 +71,9,1,8.81703601546206,1.16212752318811,3,9,8,7.5 +71,10,1,9.29952069020074,0.23461972658481,3,9,8,7.5 +71,11,1,7.34077382468804,-1.07317386072938,3,9,8,7.5 +71,12,1,7.01433588025169,-0.675331105249176,3,9,8,7.5 +72,1,0,-1.09808909841429,0.269620174333303,3,10,8,7.5 +72,2,0,-2.00220909651603,0.548979384043832,3,10,8,7.5 +72,3,0,-1.80222472561411,0.958933289709018,3,10,8,7.5 +72,4,0,-1.32840675239678,-1.58781144802648,3,10,8,7.5 +72,5,0,-2.02543863107575,-0.663504457157167,3,10,8,7.5 +72,6,0,-0.288178040674877,2.17297713167415,3,10,8,7.5 +72,7,0,-1.40101518008643,-0.299150952254327,3,10,8,7.5 +72,8,1,4.159368248292,2.11975041568074,3,10,8,7.5 +72,9,1,0.428086807773928,-0.191549750527506,3,10,8,7.5 +72,10,1,0.715218408773084,-0.0414072362763961,3,10,8,7.5 +72,11,1,1.64650965921908,1.70249736882129,3,10,8,7.5 +72,12,1,2.81412473178248,-0.965231204619628,3,10,8,7.5 +73,1,0,-0.0325266612315499,0.000103594788667179,3,7,8,7.5 +73,2,0,0.390907235629344,-0.780014803560035,3,7,8,7.5 +73,3,0,-0.180550798648557,0.0936912002326627,3,7,8,7.5 +73,4,0,-1.5512524711308,0.62465562686781,3,7,8,7.5 +73,5,0,-2.23472967066054,-1.44959411238848,3,7,8,7.5 +73,6,0,-1.38984286534249,0.569088102792347,3,7,8,7.5 +73,7,0,0.122776700863485,0.391577157997739,3,7,8,7.5 +73,8,1,4.06047514366656,2.0493007179436,3,7,8,7.5 +73,9,1,3.27332369054312,2.55938874585892,3,7,8,7.5 +73,10,1,3.65933640718248,-0.102282708032423,3,7,8,7.5 +73,11,1,4.05326592777104,1.82789400897607,3,7,8,7.5 +73,12,1,4.97820956086896,0.997490887206559,3,7,8,7.5 +74,1,0,3.95309351131709,0.51379797759258,3,8,8,7.5 +74,2,0,1.97982070241258,0.293470561643958,3,8,8,7.5 +74,3,0,1.46144540016877,-0.421960809279661,3,8,8,7.5 +74,4,0,3.45732028798638,0.862556995261492,3,8,8,7.5 +74,5,0,1.25595686636508,-1.01920939682415,3,8,8,7.5 +74,6,0,1.7790180350913,0.659588835148371,3,8,8,7.5 +74,7,0,4.22862136844254,-0.361035842762761,3,8,8,7.5 +74,8,1,4.15025043412358,-1.25414342102528,3,8,8,7.5 +74,9,1,6.79798919014444,1.01154525644387,3,8,8,7.5 +74,10,1,6.28592965166258,-1.17389547620411,3,8,8,7.5 +74,11,1,6.53134970401806,0.521128657319337,3,8,8,7.5 +74,12,1,7.83202575566369,1.0403916390826,3,8,8,7.5 +75,1,0,3.65238788975047,-0.317134441864958,3,9,8,7.5 +75,2,0,-0.238262905907347,0.749239917584653,3,9,8,7.5 +75,3,0,1.74770330958037,1.5028482726054,3,9,8,7.5 +75,4,0,1.82891755167562,1.37927749917471,3,9,8,7.5 +75,5,0,1.00215839581389,0.182019784363409,3,9,8,7.5 +75,6,0,-2.60017184180683,-0.838395465459256,3,9,8,7.5 +75,7,0,0.607487093939373,-0.45152651198091,3,9,8,7.5 +75,8,1,1.03910010163448,-0.0792271275678463,3,9,8,7.5 +75,9,1,3.40995048620754,0.380920218595513,3,9,8,7.5 +75,10,1,2.87908850760332,-1.27437687593643,3,9,8,7.5 +75,11,1,4.89724539097271,0.906637187856262,3,9,8,7.5 +75,12,1,3.77316556608977,-0.202659820470555,3,9,8,7.5 +76,1,0,1.4156317736586,1.52285982109526,3,10,8,7.5 +76,2,0,-1.89329268121184,1.49902272525636,3,10,8,7.5 +76,3,0,-2.44222560491162,-1.23327840804643,3,10,8,7.5 +76,4,0,-0.877504554318529,0.898253163652443,3,10,8,7.5 +76,5,0,0.0528795514560569,0.835725015666567,3,10,8,7.5 +76,6,0,-2.84697900722447,0.302338402370966,3,10,8,7.5 +76,7,0,-0.163716915140311,-1.29344478557119,3,10,8,7.5 +76,8,1,1.26752742771712,-1.27228839215243,3,10,8,7.5 +76,9,1,2.77666169492112,-0.430766211329326,3,10,8,7.5 +76,10,1,2.39933377099111,1.04841095951021,3,10,8,7.5 +76,11,1,3.3609561318413,0.864915054499039,3,10,8,7.5 +76,12,1,3.90826906187586,0.443738795495728,3,10,8,7.5 +77,1,0,1.62634062796674,0.773070828373812,3,7,8,7.5 +77,2,0,-1.54219416446076,0.765840803003751,3,7,8,7.5 +77,3,0,-0.982413840776802,0.974413058421387,3,7,8,7.5 +77,4,0,-0.94538931967622,-0.0277449838923522,3,7,8,7.5 +77,5,0,-0.685309627767665,0.0757575163888658,3,7,8,7.5 +77,6,0,-0.905069349797186,-0.62084875959964,3,7,8,7.5 +77,7,0,1.61565323393045,0.908504878923182,3,7,8,7.5 +77,8,1,3.3996611787419,2.23322981855796,3,7,8,7.5 +77,9,1,1.19851270632177,-1.06283419073364,3,7,8,7.5 +77,10,1,5.41211425742915,0.690047645112829,3,7,8,7.5 +77,11,1,4.50522819196034,-1.09098310411419,3,7,8,7.5 +77,12,1,4.18412445788038,0.023636092734238,3,7,8,7.5 +78,1,0,4.60923749046138,-0.775464500244865,3,8,8,7.5 +78,2,0,4.02566152718213,0.688159478757046,3,8,8,7.5 +78,3,0,3.50130588110788,0.563936400659059,3,8,8,7.5 +78,4,0,3.42362487747479,-1.42322920365403,3,8,8,7.5 +78,5,0,2.21546467566716,-0.274509730981763,3,8,8,7.5 +78,6,0,0.506757954431696,-2.43380219604755,3,8,8,7.5 +78,7,0,6.74211347685299,0.851392556138263,3,8,8,7.5 +78,8,1,4.93585074173242,-1.3377007989528,3,8,8,7.5 +78,9,1,5.1297987294274,-0.399904568798409,3,8,8,7.5 +78,10,1,5.99921620064368,-0.951488068495884,3,8,8,7.5 +78,11,1,5.65812508477719,-0.21500117920543,3,8,8,7.5 +78,12,1,9.11595238647434,1.02373437318102,3,8,8,7.5 +79,1,0,-3.27956035337828,-0.28739176303147,3,9,8,7.5 +79,2,0,-4.55723836280488,0.688765686888582,3,9,8,7.5 +79,3,0,-5.48113257876,-0.627258361964864,3,9,8,7.5 +79,4,0,-4.27710930966322,-0.0661885458802175,3,9,8,7.5 +79,5,0,-3.44453034627218,1.00988898469939,3,9,8,7.5 +79,6,0,-3.60070114910309,0.73502566070646,3,9,8,7.5 +79,7,0,-1.21227230510772,-0.251090085844652,3,9,8,7.5 +79,8,1,0.629190152280529,-1.06787868576452,3,9,8,7.5 +79,9,1,1.31426232183374,0.301699811233701,3,9,8,7.5 +79,10,1,0.297988701375953,0.767086417180156,3,9,8,7.5 +79,11,1,1.69396571558742,0.848228901245653,3,9,8,7.5 +79,12,1,1.97270652202541,0.138619622662611,3,9,8,7.5 +80,1,0,5.58350628596022,-0.613301543455102,3,10,8,7.5 +80,2,0,3.91437240535007,-1.02570727628673,3,10,8,7.5 +80,3,0,3.63941790992906,0.575702945861403,3,10,8,7.5 +80,4,0,3.04507262817115,-0.166153194360328,3,10,8,7.5 +80,5,0,5.57049910523339,0.526185012690439,3,10,8,7.5 +80,6,0,3.57735380252492,0.350790428724876,3,10,8,7.5 +80,7,0,6.02520138149274,-0.236980984040606,3,10,8,7.5 +80,8,1,9.06136412985593,1.69355356853369,3,10,8,7.5 +80,9,1,9.24506102040291,1.71748412685797,3,10,8,7.5 +80,10,1,9.1484494757358,0.865759014164273,3,10,8,7.5 +80,11,1,6.52809787388257,-0.158111717663916,3,10,8,7.5 +80,12,1,8.53761538109411,-0.621502610462533,3,10,8,7.5 +81,1,0,3.0075948161296,0.818810386110166,3,7,8,7.5 +81,2,0,-0.73825256553479,-1.30761815054703,3,7,8,7.5 +81,3,0,-1.26155072695642,-1.57809354603016,3,7,8,7.5 +81,4,0,0.743176749635348,-0.475246365758146,3,7,8,7.5 +81,5,0,-0.370280587775935,0.0836778042687921,3,7,8,7.5 +81,6,0,-0.844757228518761,-1.32754249579534,3,7,8,7.5 +81,7,0,1.87339436310373,-0.614654864352824,3,7,8,7.5 +81,8,0,-2.37240184939225,-1.1597570527926,3,7,8,7.5 +81,9,0,-1.09307016806732,-0.197646604126041,3,7,8,7.5 +81,10,0,-1.55630665469304,-1.90027779952474,3,7,8,7.5 +81,11,0,0.175597362633629,-0.704229353865507,3,7,8,7.5 +81,12,0,2.27595286597544,0.891967607376391,3,7,8,7.5 +82,1,0,3.2871693436322,-0.91714464792246,3,8,8,7.5 +82,2,0,2.68486000568269,0.437582965870395,3,8,8,7.5 +82,3,0,2.04020929399168,-1.30646347773777,3,8,8,7.5 +82,4,0,5.20758145966518,2.33656510378213,3,8,8,7.5 +82,5,0,3.44875231010099,0.122078907701043,3,8,8,7.5 +82,6,0,1.71594285188048,-0.636029215417005,3,8,8,7.5 +82,7,0,4.36763943017132,0.948244240162479,3,8,8,7.5 +82,8,0,-1.12449123742686,-2.07239744143364,3,8,8,7.5 +82,9,0,1.49393329164661,1.3931316580092,3,8,8,7.5 +82,10,0,1.07953595119776,0.249767181805547,3,8,8,7.5 +82,11,0,0.0560844904720318,-1.0865247268932,3,8,8,7.5 +82,12,0,1.08144782369428,0.285033055117248,3,8,8,7.5 +83,1,0,-1.28140507808216,-1.86339954791971,3,9,8,7.5 +83,2,0,-1.99659642874471,-1.18899510052854,3,9,8,7.5 +83,3,0,-5.10417158894476,-2.12621462933979,3,9,8,7.5 +83,4,0,-1.54876281797066,-0.661613072511514,3,9,8,7.5 +83,5,0,-2.25498236060558,-1.44121235762416,3,9,8,7.5 +83,6,0,0.410390343908608,1.18681765314819,3,9,8,7.5 +83,7,0,0.620547801730264,-1.08117976687129,3,9,8,7.5 +83,8,0,0.450275042518289,-0.8808972845246,3,9,8,7.5 +83,9,0,0.60314063974586,1.82555070584826,3,9,8,7.5 +83,10,0,-2.58313385441914,-0.714876686817077,3,9,8,7.5 +83,11,0,-5.27889529196715,-2.41400293286655,3,9,8,7.5 +83,12,0,-1.85072312175316,-1.48013050500177,3,9,8,7.5 +84,1,0,-0.682369344781835,-1.75240457506837,3,10,8,7.5 +84,2,0,-0.767488319067341,0.436085541188598,3,10,8,7.5 +84,3,0,0.512225600925109,-0.40184038059091,3,10,8,7.5 +84,4,0,0.559014035427823,-0.156789577076941,3,10,8,7.5 +84,5,0,2.6499322486856,1.47286625259511,3,10,8,7.5 +84,6,0,-1.51728359830167,-0.201621766008829,3,10,8,7.5 +84,7,0,1.05081863305578,0.699518669801718,3,10,8,7.5 +84,8,0,0.349777687723839,-0.0850399473131071,3,10,8,7.5 +84,9,0,2.33473952031943,-1.53005618227179,3,10,8,7.5 +84,10,0,2.94671541703219,1.71236374065398,3,10,8,7.5 +84,11,0,1.14152346934113,0.548593740588888,3,10,8,7.5 +84,12,0,-1.93056572895849,-0.874372423403139,3,10,8,7.5 +85,1,0,-0.661259397780102,-1.26751301272332,3,7,8,7.5 +85,2,0,0.242820708595447,0.621477960802334,3,7,8,7.5 +85,3,0,-1.19848388037175,1.16317109739696,3,7,8,7.5 +85,4,0,-0.130994122144398,0.118035314657808,3,7,8,7.5 +85,5,0,-2.03486748633015,-0.506260880409483,3,7,8,7.5 +85,6,0,-3.49576108861805,-0.884075539385448,3,7,8,7.5 +85,7,0,-1.51044611055058,-1.18392897485634,3,7,8,7.5 +85,8,0,-0.580638734841894,0.913805136788267,3,7,8,7.5 +85,9,0,-1.96447230767645,1.05248390398328,3,7,8,7.5 +85,10,0,0.250651406151869,-0.72007520291828,3,7,8,7.5 +85,11,0,-1.13680585034288,-0.722776562002148,3,7,8,7.5 +85,12,0,-0.0648649643654291,0.160272451876979,3,7,8,7.5 +86,1,0,0.318959177278616,-0.287178435635268,3,8,8,7.5 +86,2,0,0.844946758599247,-0.171860152956773,3,8,8,7.5 +86,3,0,2.03361137757095,0.430478538543605,3,8,8,7.5 +86,4,0,-0.826419262051611,-0.846202955284537,3,8,8,7.5 +86,5,0,1.36255975878767,-0.330563516328423,3,8,8,7.5 +86,6,0,0.637240713406239,-0.0443442932392742,3,8,8,7.5 +86,7,0,1.77507678238326,-0.557676604314404,3,8,8,7.5 +86,8,0,-1.94788967350079,-1.73975134860952,3,8,8,7.5 +86,9,0,-2.24693461876879,-0.591440770010903,3,8,8,7.5 +86,10,0,-1.28413613661185,-1.51156575906819,3,8,8,7.5 +86,11,0,2.86682019983875,2.28759858264761,3,8,8,7.5 +86,12,0,2.1455539719462,-0.623994298490414,3,8,8,7.5 +87,1,0,1.83730532377807,1.04933089583221,3,9,8,7.5 +87,2,0,-1.49178993157194,0.483508544346974,3,9,8,7.5 +87,3,0,-1.97162296357652,-0.149369206873409,3,9,8,7.5 +87,4,0,-2.29944055281353,-1.74454100953632,3,9,8,7.5 +87,5,0,-0.980971125454151,0.193616090322208,3,9,8,7.5 +87,6,0,-0.97381555942274,-0.296626784774737,3,9,8,7.5 +87,7,0,0.853454058085383,0.397936915521862,3,9,8,7.5 +87,8,0,-2.19776188685549,-0.177372757232997,3,9,8,7.5 +87,9,0,-1.26897496693587,0.26107821240519,3,9,8,7.5 +87,10,0,-1.56791112978867,0.602717485268086,3,9,8,7.5 +87,11,0,-5.10654146734442,-1.12196804017353,3,9,8,7.5 +87,12,0,-2.5000564101411,-1.26028761521876,3,9,8,7.5 +88,1,0,5.61818462735624,-1.04579706965378,3,10,8,7.5 +88,2,0,7.25197210068979,2.22812656248159,3,10,8,7.5 +88,3,0,4.74251122534982,0.682269855767122,3,10,8,7.5 +88,4,0,5.29597450314353,0.817503058062245,3,10,8,7.5 +88,5,0,4.48578698684089,0.631589189382231,3,10,8,7.5 +88,6,0,3.43848476369716,0.919663374749942,3,10,8,7.5 +88,7,0,8.0402089855786,0.986587748536816,3,10,8,7.5 +88,8,0,4.64607217541063,-0.440500304971547,3,10,8,7.5 +88,9,0,4.97719319350276,1.20219994350647,3,10,8,7.5 +88,10,0,5.52191328156402,1.18713278562349,3,10,8,7.5 +88,11,0,4.28796579741219,1.06638949111755,3,10,8,7.5 +88,12,0,5.37466434258456,-0.795587555935783,3,10,8,7.5 +89,1,0,3.74573030679448,1.25170592104992,3,7,8,7.5 +89,2,0,1.36315726620144,-0.217490180687906,3,7,8,7.5 +89,3,0,2.68412428841486,0.717742810508451,3,7,8,7.5 +89,4,0,2.55024188383879,1.18148283588914,3,7,8,7.5 +89,5,0,2.17943183275689,-1.42694963258428,3,7,8,7.5 +89,6,0,2.7329713486954,1.46655507634355,3,7,8,7.5 +89,7,0,0.255503351337889,-0.932724900766283,3,7,8,7.5 +89,8,0,-1.05105470691971,-0.221342110681896,3,7,8,7.5 +89,9,0,0.110561968760842,-0.281520733765174,3,7,8,7.5 +89,10,0,1.98422535549229,1.11175023267327,3,7,8,7.5 +89,11,0,1.1587714594225,2.30433771090425,3,7,8,7.5 +89,12,0,2.02708795242491,0.26284832154657,3,7,8,7.5 +90,1,0,2.45342449399926,-0.147813776071939,3,8,8,7.5 +90,2,0,0.367245352925355,1.20833943893227,3,8,8,7.5 +90,3,0,1.50923888758184,0.91966975967714,3,8,8,7.5 +90,4,0,-0.786834872770234,-0.628039814914898,3,8,8,7.5 +90,5,0,0.766184410346672,-0.354588864156575,3,8,8,7.5 +90,6,0,0.68159947834992,2.11674950029272,3,8,8,7.5 +90,7,0,0.789395889731556,0.297086989805666,3,8,8,7.5 +90,8,0,-2.22464088019144,-1.84539144487562,3,8,8,7.5 +90,9,0,-0.879384090630319,0.329037679115938,3,8,8,7.5 +90,10,0,-2.40937192632522,-1.38513602359462,3,8,8,7.5 +90,11,0,-0.38069600649142,0.266632553717435,3,8,8,7.5 +90,12,0,1.36006091769723,1.78873354277879,3,8,8,7.5 +91,1,0,0.642618903171097,0.834051008121356,3,9,8,7.5 +91,2,0,-1.88659601475385,0.175508977812844,3,9,8,7.5 +91,3,0,-0.54744424821052,1.29484209508369,3,9,8,7.5 +91,4,0,-0.956194537635853,-0.516570125873255,3,9,8,7.5 +91,5,0,-2.08519459437133,-0.446006547734085,3,9,8,7.5 +91,6,0,-1.88264626942973,0.490893373457175,3,9,8,7.5 +91,7,0,-0.665287853804951,-0.75662210543311,3,9,8,7.5 +91,8,0,-3.03524237958642,-0.674185259000274,3,9,8,7.5 +91,9,0,-3.77503490065225,-1.91482614834867,3,9,8,7.5 +91,10,0,-0.717356482591345,2.0935901564303,3,9,8,7.5 +91,11,0,-0.808828233724712,0.223059217223136,3,9,8,7.5 +91,12,0,-0.244112987008033,-0.655377591721005,3,9,8,7.5 +92,1,0,-0.0479434101977105,0.476272282260396,3,10,8,7.5 +92,2,0,-1.06417403756076,-1.26358363347461,3,10,8,7.5 +92,3,0,-0.346573894018492,0.80331016086248,3,10,8,7.5 +92,4,0,0.165964476132389,1.48917166466856,3,10,8,7.5 +92,5,0,1.4312499023266,-0.83691311902797,3,10,8,7.5 +92,6,0,0.752927254343026,0.798837929313084,3,10,8,7.5 +92,7,0,3.8960273399202,1.85562323359864,3,10,8,7.5 +92,8,0,0.0238349675953532,0.168924142246478,3,10,8,7.5 +92,9,0,-1.01848169528382,-0.562818626163668,3,10,8,7.5 +92,10,0,-0.0441358991811903,1.22682639247204,3,10,8,7.5 +92,11,0,-2.78927208434562,-1.28665608363769,3,10,8,7.5 +92,12,0,-0.589172314292196,0.0856294996060305,3,10,8,7.5 +93,1,0,7.48608817740554,-0.542076247221151,3,7,8,7.5 +93,2,0,5.97511754690336,-0.320428440547468,3,7,8,7.5 +93,3,0,5.46053089121121,0.182209921907548,3,7,8,7.5 +93,4,0,4.71281295283018,-0.287314030580537,3,7,8,7.5 +93,5,0,4.42587394851387,-0.499664814292067,3,7,8,7.5 +93,6,0,3.49542874656557,-0.627196457030402,3,7,8,7.5 +93,7,0,6.43933719022521,-0.360653869701599,3,7,8,7.5 +93,8,0,7.53498646854072,2.27180889962923,3,7,8,7.5 +93,9,0,6.9776117830659,1.56049200139089,3,7,8,7.5 +93,10,0,5.30451939792693,-0.372350079357134,3,7,8,7.5 +93,11,0,6.56172930660394,0.518198206370611,3,7,8,7.5 +93,12,0,5.9668113697498,-0.888223571691802,3,7,8,7.5 +94,1,0,1.2657977317881,0.409662918084943,3,8,8,7.5 +94,2,0,-2.43427662254074,0.156546384842692,3,8,8,7.5 +94,3,0,1.99805573561728,1.4063460720504,3,8,8,7.5 +94,4,0,2.30224765024071,1.65984994887822,3,8,8,7.5 +94,5,0,-0.396224413936083,-0.807589817925116,3,8,8,7.5 +94,6,0,-1.33841235545636,-0.18587067236056,3,8,8,7.5 +94,7,0,3.33792125898822,1.00721667078271,3,8,8,7.5 +94,8,0,-0.50005839415035,1.65548012625046,3,8,8,7.5 +94,9,0,1.61684379982285,0.377552611342541,3,8,8,7.5 +94,10,0,1.04728724590643,2.17935263522182,3,8,8,7.5 +94,11,0,-1.79797439725811,-0.639606322518748,3,8,8,7.5 +94,12,0,-0.00393083028647774,-0.110860623558975,3,8,8,7.5 +95,1,0,2.3230078735943,-0.0976325428779391,3,9,8,7.5 +95,2,0,-2.07988942208388,-1.56587542290584,3,9,8,7.5 +95,3,0,-2.1583938970863,0.661364935407346,3,9,8,7.5 +95,4,0,-1.12425078123624,0.118563830368317,3,9,8,7.5 +95,5,0,-0.994837388148586,0.206275426038859,3,9,8,7.5 +95,6,0,-1.38731175766103,0.964106469116961,3,9,8,7.5 +95,7,0,1.52061286281261,1.5553326766782,3,9,8,7.5 +95,8,0,0.00194673613001911,1.20174547780709,3,9,8,7.5 +95,9,0,-3.8704911858654,-1.20057896327192,3,9,8,7.5 +95,10,0,-0.107820696512016,0.618639969053377,3,9,8,7.5 +95,11,0,-1.53301613559891,-0.782195323300171,3,9,8,7.5 +95,12,0,-2.04024231221314,-1.74327661663113,3,9,8,7.5 +96,1,0,1.55568803288655,0.956943780019508,3,10,8,7.5 +96,2,0,-0.614109153529499,0.582186856536501,3,10,8,7.5 +96,3,0,-0.304932390260029,1.09538299978217,3,10,8,7.5 +96,4,0,-0.209343526248331,-0.464815865292816,3,10,8,7.5 +96,5,0,-1.74780411798817,-0.557856293672251,3,10,8,7.5 +96,6,0,-1.03493281927556,0.646123786774245,3,10,8,7.5 +96,7,0,-1.0545327904538,-0.902798139898716,3,10,8,7.5 +96,8,0,-0.950564747977886,-0.39225625986041,3,10,8,7.5 +96,9,0,1.51277930206274,2.17748426615849,3,10,8,7.5 +96,10,0,-0.970801398326797,-0.165506447710334,3,10,8,7.5 +96,11,0,-2.34682246110356,-0.466571636600213,3,10,8,7.5 +96,12,0,0.45332084060403,1.41906653430989,3,10,8,7.5 +97,1,0,1.86123839346717,-0.981734238770132,3,7,8,7.5 +97,2,0,4.4328150043999,1.35128740724808,3,7,8,7.5 +97,3,0,1.02512600370838,-1.20233792179589,3,7,8,7.5 +97,4,0,1.38378932449295,0.568150387144566,3,7,8,7.5 +97,5,0,1.67355193079364,0.628383348703304,3,7,8,7.5 +97,6,0,1.06026800717718,1.33032233688059,3,7,8,7.5 +97,7,0,1.53615277994626,-0.804678846868144,3,7,8,7.5 +97,8,0,-0.53373822186145,-1.39215360526421,3,7,8,7.5 +97,9,0,2.11600127792217,0.773052309791784,3,7,8,7.5 +97,10,0,1.77949924306812,0.414701752225017,3,7,8,7.5 +97,11,0,0.136723824260805,-0.0860489231858821,3,7,8,7.5 +97,12,0,3.34139070310032,1.13980502869125,3,7,8,7.5 +98,1,0,0.18445502762236,0.382138183494791,3,8,8,7.5 +98,2,0,-3.32105530088226,-0.287866671803945,3,8,8,7.5 +98,3,0,0.292671321447887,1.44590676490247,3,8,8,7.5 +98,4,0,-0.914506672081499,1.14639669152249,3,8,8,7.5 +98,5,0,-1.86231065575035,0.173726744519518,3,8,8,7.5 +98,6,0,-1.79630639355966,0.701890932718765,3,8,8,7.5 +98,7,0,0.645951332467728,-0.389343683210168,3,8,8,7.5 +98,8,0,-1.10849859335121,-0.380426674991457,3,8,8,7.5 +98,9,0,-1.55876008875676,-0.0280039290701337,3,8,8,7.5 +98,10,0,-1.23494764602122,0.436445247016977,3,8,8,7.5 +98,11,0,-0.879701549788621,0.370292286744897,3,8,8,7.5 +98,12,0,-1.16285566135497,-0.31705617534142,3,8,8,7.5 +99,1,0,1.36808493052887,0.468469967975404,3,9,8,7.5 +99,2,0,-1.54350465912812,-0.068524064729391,3,9,8,7.5 +99,3,0,-0.184428560874675,1.12994793035624,3,9,8,7.5 +99,4,0,0.768565315801922,-0.274949360168424,3,9,8,7.5 +99,5,0,-0.804470191840947,-0.911515788569356,3,9,8,7.5 +99,6,0,-1.9686040023349,0.436086497526262,3,9,8,7.5 +99,7,0,0.241098274694165,0.360638229920427,3,9,8,7.5 +99,8,0,-1.75486291319697,-0.722452937898593,3,9,8,7.5 +99,9,0,0.596079912936925,0.926178317390592,3,9,8,7.5 +99,10,0,0.0176191353135152,0.350556151189497,3,9,8,7.5 +99,11,0,2.06027721051965,-0.155049000209409,3,9,8,7.5 +99,12,0,4.2260585943161,1.20499859032446,3,9,8,7.5 +100,1,0,1.84242100889917,0.675144126203605,3,10,8,7.5 +100,2,0,0.335549632459587,-0.153004806096296,3,10,8,7.5 +100,3,0,-0.0315831115740504,-1.32832563555485,3,10,8,7.5 +100,4,0,0.509962743841716,-0.103351936347716,3,10,8,7.5 +100,5,0,-0.614085067030215,-0.904887199858005,3,10,8,7.5 +100,6,0,1.91354351258355,0.473581600507013,3,10,8,7.5 +100,7,0,2.23326853382397,0.0519326749749344,3,10,8,7.5 +100,8,0,-1.34153658051805,0.125150788291804,3,10,8,7.5 +100,9,0,-2.88217825207212,0.0661974540578588,3,10,8,7.5 +100,10,0,0.707781101436412,-0.650698543979529,3,10,8,7.5 +100,11,0,0.186653617682782,0.578688707688442,3,10,8,7.5 +100,12,0,-0.678306911960556,-0.466030843933284,3,10,8,7.5 +101,1,0,-4.12233539877087,-0.130832898859255,4,11,8,15 +101,2,0,-4.98986567240415,-1.33239867255592,4,11,8,15 +101,3,0,-4.05463025302717,-0.232941156270798,4,11,8,15 +101,4,0,-4.97714575884022,1.01452476927029,4,11,8,15 +101,5,1,-2.65344642692745,0.726915650386348,4,11,8,15 +101,6,1,-3.23755784831755,0.00546196472548116,4,11,8,15 +101,7,1,1.07413961433793,1.68444519547174,4,11,8,15 +101,8,1,0.103740679145903,0.697010401783869,4,11,8,15 +101,9,1,-1.03588073848461,-0.593064156228788,4,11,8,15 +101,10,1,-0.0552258551020249,0.717810189100407,4,11,8,15 +101,11,1,-0.313394445429308,-1.25706169053077,4,11,8,15 +101,12,1,1.96342267337733,-0.172882677443891,4,11,8,15 +102,1,0,0.449601079595786,1.80047224555515,4,12,8,15 +102,2,0,-2.39445251517033,-0.0627395968561276,4,12,8,15 +102,3,0,-3.0011897954081,1.00503874495999,4,12,8,15 +102,4,0,-2.16277425029355,0.0614690170089316,4,12,8,15 +102,5,1,-0.0437311120564913,0.0600311827822018,4,12,8,15 +102,6,1,-2.68335724829164,-1.0717701070023,4,12,8,15 +102,7,1,0.815229185821719,1.68366731810327,4,12,8,15 +102,8,1,1.84629564418507,0.326010408395149,4,12,8,15 +102,9,1,2.00673630288402,-0.0401741117421287,4,12,8,15 +102,10,1,5.07436605205018,2.95963332678229,4,12,8,15 +102,11,1,1.43900082541318,-0.348562590165045,4,12,8,15 +102,12,1,2.69298540929088,0.783074149650571,4,12,8,15 +103,1,0,3.22811787206364,-1.1980343221548,4,13,8,15 +103,2,0,2.20194973948894,0.454737808449278,4,13,8,15 +103,3,0,2.07796901141504,1.95606814662562,4,13,8,15 +103,4,0,0.514980344309409,0.0177867557704239,4,13,8,15 +103,5,1,3.56584280207704,-0.700085743924675,4,13,8,15 +103,6,1,4.02582667024559,1.14974371902206,4,13,8,15 +103,7,1,5.17425763492923,1.31194915579671,4,13,8,15 +103,8,1,0.898718712122902,-1.4862179036534,4,13,8,15 +103,9,1,4.50110143347668,2.26190869857187,4,13,8,15 +103,10,1,3.8293708852092,-0.537588790169189,4,13,8,15 +103,11,1,4.3602228136691,-2.08144436948879,4,13,8,15 +103,12,1,6.57485816945634,-0.370773463416046,4,13,8,15 +104,1,0,1.6748148908572,-0.0822897900684123,4,14,8,15 +104,2,0,2.51824602552874,-0.152104744516297,4,14,8,15 +104,3,0,1.38678306648701,-1.1849241277785,4,14,8,15 +104,4,0,2.56122835042639,0.296140456413184,4,14,8,15 +104,5,1,4.03545234245049,-0.241627961256247,4,14,8,15 +104,6,1,2.37017513382549,0.538650315623458,4,14,8,15 +104,7,1,6.30073532302911,2.00225885047421,4,14,8,15 +104,8,1,6.17050491987196,1.06860676381156,4,14,8,15 +104,9,1,5.17270273073554,0.497074063077358,4,14,8,15 +104,10,1,8.21810960060855,0.912682087024742,4,14,8,15 +104,11,1,6.46727488555968,0.161173137446732,4,14,8,15 +104,12,1,7.20143125271106,-0.573508682869432,4,14,8,15 +105,1,0,5.2579847628087,0.229289830711741,4,11,8,15 +105,2,0,2.34171181729716,-0.402715287628865,4,11,8,15 +105,3,0,3.09625577606962,0.709102702075593,4,11,8,15 +105,4,0,4.08954842063369,-0.0578461343000415,4,11,8,15 +105,5,1,4.61869577598171,-0.945951784167219,4,11,8,15 +105,6,1,2.31967151134377,-0.0959314386021993,4,11,8,15 +105,7,1,7.79805158455228,1.28661471974361,4,11,8,15 +105,8,1,5.5400012278481,-0.420472082091823,4,11,8,15 +105,9,1,4.10582083870882,-0.493006624220028,4,11,8,15 +105,10,1,6.13000211256374,0.536196043364837,4,11,8,15 +105,11,1,7.0680124736182,-0.0667610670915626,4,11,8,15 +105,12,1,8.58345784237718,1.29025756252417,4,11,8,15 +106,1,0,-0.378640695070692,-0.402962218307162,4,12,8,15 +106,2,0,-3.90982970898268,-1.18085642988503,4,12,8,15 +106,3,0,-1.66190797010933,1.49611827674554,4,12,8,15 +106,4,0,0.499200977534678,0.488358878340677,4,12,8,15 +106,5,1,-0.788410401213817,-1.19511746591295,4,12,8,15 +106,6,1,-0.503483285364508,-0.679442639327253,4,12,8,15 +106,7,1,3.91372725101655,2.06849484931291,4,12,8,15 +106,8,1,1.77842838186651,-0.190743788598591,4,12,8,15 +106,9,1,2.61569439621651,0.934753969031499,4,12,8,15 +106,10,1,2.32043939050635,-0.0889035171979748,4,12,8,15 +106,11,1,3.27821029719273,1.10700116723884,4,12,8,15 +106,12,1,4.98716542892803,0.905050464878154,4,12,8,15 +107,1,0,-0.416603053408224,-0.118918590098563,4,13,8,15 +107,2,0,-1.88374803243746,0.714613993494367,4,13,8,15 +107,3,0,-0.718736623255206,0.482138174107268,4,13,8,15 +107,4,0,-2.09749639686646,0.0329489255039953,4,13,8,15 +107,5,1,1.25459302802195,0.362389312027887,4,13,8,15 +107,6,1,0.460553201703688,0.844092316808621,4,13,8,15 +107,7,1,1.67479387727934,0.0886847746647835,4,13,8,15 +107,8,1,0.604276247105281,1.01292705049834,4,13,8,15 +107,9,1,1.94051947668412,0.755727922004688,4,13,8,15 +107,10,1,2.90784191197117,-1.2654669063245,4,13,8,15 +107,11,1,2.44153992572018,0.400972784479104,4,13,8,15 +107,12,1,5.32851253016372,0.749374645063091,4,13,8,15 +108,1,0,0.44944713273605,-0.440670445777219,4,14,8,15 +108,2,0,-2.54814508414505,-1.5302265999868,4,14,8,15 +108,3,0,-0.834741492258532,0.858760566841236,4,14,8,15 +108,4,0,1.22835892053953,1.21910997177155,4,14,8,15 +108,5,1,0.67346288603752,-0.86466261638371,4,14,8,15 +108,6,1,1.04055565553068,-0.0501256787307594,4,14,8,15 +108,7,1,3.95164734736732,1.04496374246867,4,14,8,15 +108,8,1,1.40833601086861,1.26070907657286,4,14,8,15 +108,9,1,0.76020703209212,1.05296305066072,4,14,8,15 +108,10,1,1.32362420641385,-0.27372035227245,4,14,8,15 +108,11,1,1.25936147820643,-1.59895455708698,4,14,8,15 +108,12,1,3.99515755507188,0.114455811108278,4,14,8,15 +109,1,0,2.36503664290813,1.00798163756683,4,11,8,15 +109,2,0,-1.0566182959576,-1.36546577812692,4,11,8,15 +109,3,0,-0.531099537962635,0.0770628018913173,4,11,8,15 +109,4,0,-1.39684158824135,-0.40531196239218,4,11,8,15 +109,5,1,2.35200885454689,-0.689051675156801,4,11,8,15 +109,6,1,1.58229451718211,-1.25519198149031,4,11,8,15 +109,7,1,3.79142443451985,-0.236698321493625,4,11,8,15 +109,8,1,5.37225652267638,1.41044777428593,4,11,8,15 +109,9,1,5.43097832787634,1.34281888967924,4,11,8,15 +109,10,1,5.77860202357791,-0.417404251975354,4,11,8,15 +109,11,1,4.48819813633794,0.174883342666136,4,11,8,15 +109,12,1,4.69206736386456,-0.327579973720642,4,11,8,15 +110,1,0,1.13526206816996,-0.226532010256694,4,12,8,15 +110,2,0,1.07337970865333,-0.414360339986114,4,12,8,15 +110,3,0,1.47526957241873,0.814283594068896,4,12,8,15 +110,4,0,0.436214578579601,0.103056136097164,4,12,8,15 +110,5,1,2.43616235735957,0.0420388023434178,4,12,8,15 +110,6,1,3.12834740902155,-0.632266864666231,4,12,8,15 +110,7,1,5.31587743444661,0.565245091724471,4,12,8,15 +110,8,1,2.70177286255125,-0.689746708012151,4,12,8,15 +110,9,1,3.7529768527959,-0.218868719581027,4,12,8,15 +110,10,1,6.69486669284887,0.117950115481832,4,12,8,15 +110,11,1,3.66162479722096,0.271484823823822,4,12,8,15 +110,12,1,7.52246911138662,1.42308362019355,4,12,8,15 +111,1,0,-1.64414948948358,-0.0225863999832703,4,13,8,15 +111,2,0,-2.80212079156269,-1.05882968800214,4,13,8,15 +111,3,0,-2.23255529216634,1.28592340345778,4,13,8,15 +111,4,0,-1.57969214674697,0.34178489476182,4,13,8,15 +111,5,0,-0.14867144551175,-0.0768174865573106,4,13,8,15 +111,6,0,-0.152070060829266,0.823502823577843,4,13,8,15 +111,7,0,-0.0692566470903007,-1.44222860912172,4,13,8,15 +111,8,1,0.618646485474864,-0.293556150437869,4,13,8,15 +111,9,1,1.56052035435213,0.356084169233803,4,13,8,15 +111,10,1,2.45971223333319,0.791275567032944,4,13,8,15 +111,11,1,1.4747480162241,0.10896446088627,4,13,8,15 +111,12,1,3.99686324449902,0.0498370022359987,4,13,8,15 +112,1,0,0.852289736338477,-0.464407766619813,4,14,8,15 +112,2,0,-1.90093845899952,-1.09010415558086,4,14,8,15 +112,3,0,-1.87122796829328,-0.349461518652923,4,14,8,15 +112,4,0,-0.213785552631496,-0.956157362093526,4,14,8,15 +112,5,0,-1.0484920875242,0.103971324679902,4,14,8,15 +112,6,0,-2.32365309643809,0.504387758613899,4,14,8,15 +112,7,0,-0.478432766741832,0.921413704213256,4,14,8,15 +112,8,1,1.03277758409829,0.234267516926085,4,14,8,15 +112,9,1,-0.32610435439982,-1.18991535441097,4,14,8,15 +112,10,1,2.088507632238,0.499225131246831,4,14,8,15 +112,11,1,2.25695956682546,0.899250466233598,4,14,8,15 +112,12,1,3.70446377814408,0.00349550386724531,4,14,8,15 +113,1,0,-2.27409927956071,-1.16785451449137,4,11,8,15 +113,2,0,-5.40555987131017,-2.90439490622985,4,11,8,15 +113,3,0,-5.03804578333735,-0.715327184204833,4,11,8,15 +113,4,0,-2.75412973506537,-0.622198536716676,4,11,8,15 +113,5,0,-4.22479433758815,0.401751698686044,4,11,8,15 +113,6,0,-4.6197340604066,-0.479838527664214,4,11,8,15 +113,7,0,0.253845410023647,1.78843628808848,4,11,8,15 +113,8,1,0.302589015470938,-1.15422367970202,4,11,8,15 +113,9,1,0.757233866882821,-0.0461507264084715,4,11,8,15 +113,10,1,1.9098003614267,0.160207929069563,4,11,8,15 +113,11,1,3.75288542086503,0.0259100984045613,4,11,8,15 +113,12,1,4.19082842434993,0.000203921430037288,4,11,8,15 +114,1,0,-1.92853171290431,0.359091116682282,4,12,8,15 +114,2,0,-5.24809522181598,-1.04561205314643,4,12,8,15 +114,3,0,-4.90950811214786,-0.226440044545422,4,12,8,15 +114,4,0,-4.46864875466899,0.304434318169843,4,12,8,15 +114,5,0,-4.00457123916357,0.418841415423042,4,12,8,15 +114,6,0,-5.82859159725368,-0.0728107300109764,4,12,8,15 +114,7,0,-4.72143915532735,-0.246966601652527,4,12,8,15 +114,8,1,-1.09233695781433,0.476737290688364,4,12,8,15 +114,9,1,-3.17888470702284,-1.21467862198599,4,12,8,15 +114,10,1,-2.07370864135068,-0.729152014000332,4,12,8,15 +114,11,1,-2.07547422575902,0.541401118941556,4,12,8,15 +114,12,1,-2.0721639484124,0.535130896472227,4,12,8,15 +115,1,0,-1.91757935057423,-1.3308414520347,4,13,8,15 +115,2,0,-1.72576519625007,-1.78186804859549,4,13,8,15 +115,3,0,-2.30165066914607,-0.254153837631687,4,13,8,15 +115,4,0,9.83841435882038e-05,0.376517317544143,4,13,8,15 +115,5,0,-0.85497609183654,1.59992980322311,4,13,8,15 +115,6,0,-2.21082451970702,-0.0323675173466787,4,13,8,15 +115,7,0,0.168385918785138,-0.683975471583769,4,13,8,15 +115,8,1,2.95409833970508,0.307809707495355,4,13,8,15 +115,9,1,0.754853747301766,-0.896241599333628,4,13,8,15 +115,10,1,0.0951480401129587,-1.38229531053789,4,13,8,15 +115,11,1,1.42210806993005,-0.61790974308177,4,13,8,15 +115,12,1,1.42067221120932,0.141043827094417,4,13,8,15 +116,1,0,-2.71467347497377,-0.301079218710722,4,14,8,15 +116,2,0,-2.96935984902498,-0.278263719532767,4,14,8,15 +116,3,0,-0.113396958811203,0.813676782863289,4,14,8,15 +116,4,0,-2.28856403500964,0.582526183152018,4,14,8,15 +116,5,0,-3.27800505133954,-0.597916356464412,4,14,8,15 +116,6,0,-2.0903497036524,-1.02805020999634,4,14,8,15 +116,7,0,-0.136654673903914,-1.1752020457078,4,14,8,15 +116,8,1,1.18943148899727,-0.316403904634892,4,14,8,15 +116,9,1,0.52855144263424,-0.710620528390908,4,14,8,15 +116,10,1,1.28807044462468,-1.19228900816916,4,14,8,15 +116,11,1,2.27453666764855,0.0901012794684664,4,14,8,15 +116,12,1,2.98434742531587,0.761759498681626,4,14,8,15 +117,1,0,2.77408191872025,-1.84318165917917,4,11,8,15 +117,2,0,5.30249313088979,0.907522918317259,4,11,8,15 +117,3,0,2.07931120663047,-0.0771888232477468,4,11,8,15 +117,4,0,3.01307577382947,0.396153084390139,4,11,8,15 +117,5,0,2.45079764316363,-0.564229658868067,4,11,8,15 +117,6,0,-0.508497966599797,-2.12542509938451,4,11,8,15 +117,7,0,4.46765730575529,-0.0203824565996475,4,11,8,15 +117,8,1,6.23491950420449,-0.992394156850378,4,11,8,15 +117,9,1,6.73357465409218,0.442322803594492,4,11,8,15 +117,10,1,6.44425437709635,-1.19716255047654,4,11,8,15 +117,11,1,6.11706566445874,-0.633496952749165,4,11,8,15 +117,12,1,6.18556239577665,-0.800822668492418,4,11,8,15 +118,1,0,2.11255809909377,0.352192849514561,4,12,8,15 +118,2,0,-0.359608233031671,-1.09462747956242,4,12,8,15 +118,3,0,-0.675824832094101,-0.363956286026996,4,12,8,15 +118,4,0,-2.24827754707585,-1.15949698261289,4,12,8,15 +118,5,0,-1.8676220925804,-1.60395020709504,4,12,8,15 +118,6,0,-0.236668209509073,0.504407045473353,4,12,8,15 +118,7,0,0.205812864925187,-1.18231389172865,4,12,8,15 +118,8,1,2.48106512163931,-1.08754767267516,4,12,8,15 +118,9,1,4.23712881205394,0.203842709808867,4,12,8,15 +118,10,1,4.29723013654603,-0.267490803625227,4,12,8,15 +118,11,1,5.7320421240762,0.813075231340702,4,12,8,15 +118,12,1,6.15702572440454,0.44238400235772,4,12,8,15 +119,1,0,0.242845636465885,-0.268784033886979,4,13,8,15 +119,2,0,-2.21862686549558,-1.47996794897682,4,13,8,15 +119,3,0,-1.27505632359054,-0.370771712593509,4,13,8,15 +119,4,0,-0.122932976674039,0.625213906307649,4,13,8,15 +119,5,0,-2.14301968612835,-0.388081679199303,4,13,8,15 +119,6,0,-2.87238213756994,0.828532840124906,4,13,8,15 +119,7,0,-0.446851271252032,0.599178631176609,4,13,8,15 +119,8,1,-0.239714483566724,0.615839644973678,4,13,8,15 +119,9,1,1.31233236990999,0.327876321989235,4,13,8,15 +119,10,1,2.62454908338158,0.00741212401770021,4,13,8,15 +119,11,1,2.57057329978047,0.059317578208252,4,13,8,15 +119,12,1,3.07526090308198,-0.795777229250948,4,13,8,15 +120,1,0,3.3528811644038,0.271433047143801,4,14,8,15 +120,2,0,-1.28525713153043,0.509089622447105,4,14,8,15 +120,3,0,-1.44075298624627,-0.951836797080414,4,14,8,15 +120,4,0,0.6569902070518,1.63154496400516,4,14,8,15 +120,5,0,-0.207331538086875,0.596953030222945,4,14,8,15 +120,6,0,0.502538525498454,1.76969715309753,4,14,8,15 +120,7,0,1.14009611431002,0.286661500215946,4,14,8,15 +120,8,1,1.84833171372282,-0.776599406722213,4,14,8,15 +120,9,1,2.47275352831436,0.458077830558416,4,14,8,15 +120,10,1,4.57420270936362,-1.10101653352886,4,14,8,15 +120,11,1,3.61663166317648,-0.920789128344997,4,14,8,15 +120,12,1,3.69464401945508,-1.54351925640238,4,14,8,15 +121,1,0,-1.0600602020322,-1.13121752931659,4,11,8,15 +121,2,0,-1.85657793552554,0.888194671533197,4,11,8,15 +121,3,0,-0.623417262479436,0.76235754290946,4,11,8,15 +121,4,0,1.35969238642386,1.11124321918003,4,11,8,15 +121,5,0,-0.0707905436302516,0.101570610309974,4,11,8,15 +121,6,0,-1.19363571366154,-0.387377171547133,4,11,8,15 +121,7,0,4.23753631014307,1.48631140057516,4,11,8,15 +121,8,0,0.735483555807485,0.747224433656291,4,11,8,15 +121,9,0,-0.0232023954439901,0.583318719239443,4,11,8,15 +121,10,0,-1.18608942762419,0.0892974700066731,4,11,8,15 +121,11,0,-0.644032618111192,-1.81000698738742,4,11,8,15 +121,12,0,-0.02056738399732,0.267408737186772,4,11,8,15 +122,1,0,-3.08056656708052,0.145310759880022,4,12,8,15 +122,2,0,-3.12700893824556,0.113835006327305,4,12,8,15 +122,3,0,-1.04419842481571,0.396967089457762,4,12,8,15 +122,4,0,0.135545764301464,0.977016564671623,4,12,8,15 +122,5,0,-1.42218802775512,0.283853083642809,4,12,8,15 +122,6,0,-3.56277596900429,-0.597685696526862,4,12,8,15 +122,7,0,-3.60336801915389,-2.39525707424464,4,12,8,15 +122,8,0,-4.3914808518338,-0.815377354155221,4,12,8,15 +122,9,0,-3.4448566191217,1.83094436896296,4,12,8,15 +122,10,0,-2.16810520772992,1.21497455663955,4,12,8,15 +122,11,0,-2.94672911985112,-0.363391708499243,4,12,8,15 +122,12,0,-1.49824326425028,0.263802260717005,4,12,8,15 +123,1,0,2.0277038175486,0.0295659767701978,4,13,8,15 +123,2,0,1.43628483985488,0.585280548862784,4,13,8,15 +123,3,0,0.475736056296466,-1.12480505974784,4,13,8,15 +123,4,0,2.16853285933966,0.441753947234446,4,13,8,15 +123,5,0,0.984556587820186,0.156783229886456,4,13,8,15 +123,6,0,2.06112244523116,1.25513739272707,4,13,8,15 +123,7,0,1.4876786113248,-0.680917834318016,4,13,8,15 +123,8,0,0.80864241323021,-0.0680987303010995,4,13,8,15 +123,9,0,2.65879076396332,2.06262768109468,4,13,8,15 +123,10,0,0.496609846636935,-0.5151221570367,4,13,8,15 +123,11,0,2.39890629208563,0.881014241503871,4,13,8,15 +123,12,0,3.02088733074367,-0.409376983026605,4,13,8,15 +124,1,0,0.159406231756505,-1.27937364301427,4,14,8,15 +124,2,0,-0.870353748913488,-0.115580004260754,4,14,8,15 +124,3,0,-0.296990291361671,0.993507599776541,4,14,8,15 +124,4,0,0.0605520568953808,0.588847892977911,4,14,8,15 +124,5,0,-0.698767535972994,-0.825178803009419,4,14,8,15 +124,6,0,-1.87652192124571,-1.64349289944471,4,14,8,15 +124,7,0,-0.413868969028433,0.0734267452179088,4,14,8,15 +124,8,0,-1.77004573532126,-0.807631807615683,4,14,8,15 +124,9,0,0.0842720751504509,0.598290188896639,4,14,8,15 +124,10,0,-0.718062257485349,0.55386498406537,4,14,8,15 +124,11,0,-0.196708236358595,1.34033019348136,4,14,8,15 +124,12,0,-0.69449122580841,-1.8826002201915,4,14,8,15 +125,1,0,1.56839349543483,0.231130897870344,4,11,8,15 +125,2,0,-0.794166097874373,1.14857461983794,4,11,8,15 +125,3,0,-0.425502187949611,0.208484732547492,4,11,8,15 +125,4,0,0.357725603624371,-1.23525175591491,4,11,8,15 +125,5,0,-1.58851515867124,-0.192703897948211,4,11,8,15 +125,6,0,-2.43126729351048,0.230924244600075,4,11,8,15 +125,7,0,1.11543079205826,0.456955487624756,4,11,8,15 +125,8,0,-1.45073258315418,0.551023782396379,4,11,8,15 +125,9,0,-0.5746686202886,-0.0895518817419534,4,11,8,15 +125,10,0,1.39128007034366,-1.37709848715224,4,11,8,15 +125,11,0,1.80630925364463,-0.446071198434166,4,11,8,15 +125,12,0,2.66498070296518,1.20867474667679,4,11,8,15 +126,1,0,1.24743148789758,0.216074727019801,4,12,8,15 +126,2,0,0.00799964912559165,-1.01450401791437,4,12,8,15 +126,3,0,1.02157206881559,-0.0135952155271528,4,12,8,15 +126,4,0,2.20118299652645,0.433766337545509,4,12,8,15 +126,5,0,0.775895273064631,1.07270330335157,4,12,8,15 +126,6,0,1.05608515558559,1.1309866395033,4,12,8,15 +126,7,0,1.74805882286997,-0.00138815383030731,4,12,8,15 +126,8,0,-0.826074423918346,-0.161789426858747,4,12,8,15 +126,9,0,3.12783735358829,1.98727357827864,4,12,8,15 +126,10,0,0.884732279785542,0.854110582963726,4,12,8,15 +126,11,0,0.0481346184085252,1.00554517787555,4,12,8,15 +126,12,0,2.3412655530404,-0.233974223660684,4,12,8,15 +127,1,0,4.12187520854169,0.833222388307469,4,13,8,15 +127,2,0,2.78451302222893,0.646231893924009,4,13,8,15 +127,3,0,2.54090584212,-0.49609934702388,4,13,8,15 +127,4,0,2.16992820734511,0.210468931243982,4,13,8,15 +127,5,0,4.61895210613493,0.447136122237439,4,13,8,15 +127,6,0,2.14963006071977,-0.49770757117583,4,13,8,15 +127,7,0,5.73041037096678,-0.0619990560614816,4,13,8,15 +127,8,0,2.63333133432159,-0.440192136698113,4,13,8,15 +127,9,0,2.72423969844466,0.740356560010622,4,13,8,15 +127,10,0,3.03699712494831,0.550464984545119,4,13,8,15 +127,11,0,1.30128797951358,0.124748190600336,4,13,8,15 +127,12,0,5.46439524940469,0.0497453815860016,4,13,8,15 +128,1,0,-0.932974191270767,0.863406305004318,4,14,8,15 +128,2,0,-2.5773742288656,1.43224001799581,4,14,8,15 +128,3,0,-1.55326512539428,0.714185596896415,4,14,8,15 +128,4,0,-3.68776550987517,-1.29388932704077,4,14,8,15 +128,5,0,-0.931229009172763,0.585489992788379,4,14,8,15 +128,6,0,-1.59673709720465,0.0956885251164214,4,14,8,15 +128,7,0,-0.062124508556508,0.784320057052052,4,14,8,15 +128,8,0,-4.03996171711577,0.846848733856535,4,14,8,15 +128,9,0,-2.91989766632239,0.271627280385326,4,14,8,15 +128,10,0,-3.26919116120824,-1.27648485340481,4,14,8,15 +128,11,0,-2.58981889505451,0.0188532182080984,4,14,8,15 +128,12,0,-2.08078260284484,-0.419404093421562,4,14,8,15 +129,1,0,1.1685516423097,0.0832820090111023,4,11,8,15 +129,2,0,-0.483949209200629,-0.153672956015889,4,11,8,15 +129,3,0,-3.75125929930472,-1.04989049749704,4,11,8,15 +129,4,0,-0.621242707143967,0.87735891900983,4,11,8,15 +129,5,0,-0.361355166408239,0.804341838964091,4,11,8,15 +129,6,0,-1.28780170756285,0.142370255542008,4,11,8,15 +129,7,0,0.538845106759985,0.322773353850078,4,11,8,15 +129,8,0,-1.15702537260821,0.199691144585861,4,11,8,15 +129,9,0,-1.46341577557229,-0.00192914797394203,4,11,8,15 +129,10,0,0.666541088767049,0.537847859671586,4,11,8,15 +129,11,0,-0.850703604359227,-0.309077656126097,4,11,8,15 +129,12,0,-0.721415704021324,-2.29586448200558,4,11,8,15 +130,1,0,1.61450550809919,-0.906444975910072,4,12,8,15 +130,2,0,-0.0888450637546805,-1.05150210553651,4,12,8,15 +130,3,0,1.03752395317112,-0.496115286239749,4,12,8,15 +130,4,0,1.01828483972437,-0.150189868594392,4,12,8,15 +130,5,0,1.30824775726166,0.713539305181673,4,12,8,15 +130,6,0,0.0727176125830918,-0.237431690377537,4,12,8,15 +130,7,0,1.93220728466386,0.125576216555124,4,12,8,15 +130,8,0,-0.0375226750460207,0.484363254231166,4,12,8,15 +130,9,0,2.62232998773875,2.10017458364752,4,12,8,15 +130,10,0,-0.270513887893615,0.420943706132718,4,12,8,15 +130,11,0,-1.84740519270983,0.158351534888796,4,12,8,15 +130,12,0,3.93822138204667,1.9492697794737,4,12,8,15 +131,1,0,-0.918527747141863,-0.260949101201396,4,13,8,15 +131,2,0,-2.6974146634198,1.19509138873227,4,13,8,15 +131,3,0,-2.63320355548226,-0.367609366059816,4,13,8,15 +131,4,0,-1.82098789676678,-0.796174696940224,4,13,8,15 +131,5,0,0.810748661962117,1.50165032318135,4,13,8,15 +131,6,0,0.241046835438848,1.74792656473916,4,13,8,15 +131,7,0,0.748475089193749,1.08459757938152,4,13,8,15 +131,8,0,-1.93111106670204,-1.20172120254659,4,13,8,15 +131,9,0,-2.25712834981675,0.0832763594494915,4,13,8,15 +131,10,0,-3.1075565528469,-0.85630756023459,4,13,8,15 +131,11,0,-1.03015555207491,1.40651985989428,4,13,8,15 +131,12,0,0.610524342498919,0.354860215261159,4,13,8,15 +132,1,0,7.28031534670377,0.21079909224224,4,14,8,15 +132,2,0,6.85353154145956,0.224456858796474,4,14,8,15 +132,3,0,2.66069287632978,1.04146185191629,4,14,8,15 +132,4,0,5.66181587661633,1.57606561952563,4,14,8,15 +132,5,0,4.55434501598323,1.10314508089102,4,14,8,15 +132,6,0,3.0844103993844,0.904345260882532,4,14,8,15 +132,7,0,4.41157823169815,-0.904942549731265,4,14,8,15 +132,8,0,4.01054255425014,0.343334706034443,4,14,8,15 +132,9,0,2.15148374667877,-1.33359941098765,4,14,8,15 +132,10,0,6.00438640238982,-0.0681783688766765,4,14,8,15 +132,11,0,3.09921190780103,-1.00549791773323,4,14,8,15 +132,12,0,6.9653708135546,-1.68171284508092,4,14,8,15 +133,1,0,0.715930991083568,0.0689493272482862,4,11,8,15 +133,2,0,0.764940166458471,0.385557995354608,4,11,8,15 +133,3,0,0.337448648481691,0.133795439406697,4,11,8,15 +133,4,0,2.60883111292264,1.10124185333136,4,11,8,15 +133,5,0,-1.49763353631606,-0.575527467366867,4,11,8,15 +133,6,0,-0.701854666484063,-0.682567842237448,4,11,8,15 +133,7,0,2.07461561393987,0.91761938720324,4,11,8,15 +133,8,0,-1.13964065962517,-1.35441682742015,4,11,8,15 +133,9,0,-1.73981863526659,-0.96933572807456,4,11,8,15 +133,10,0,-0.242614027177867,-0.727169479669886,4,11,8,15 +133,11,0,-2.52581063440841,-1.09883885367473,4,11,8,15 +133,12,0,0.978574606059992,0.735747538523928,4,11,8,15 +134,1,0,1.52582888143947,-0.188543289609506,4,12,8,15 +134,2,0,1.53977695634971,-0.309395385910585,4,12,8,15 +134,3,0,1.19072477503354,0.781902097483998,4,12,8,15 +134,4,0,1.02307278202461,0.186799054977248,4,12,8,15 +134,5,0,2.7905025080816,1.4135271578704,4,12,8,15 +134,6,0,2.03422331188375,-0.0983741867274115,4,12,8,15 +134,7,0,4.29881920329992,0.469625336412407,4,12,8,15 +134,8,0,0.345322719254073,-2.06712983169236,4,12,8,15 +134,9,0,0.672723777433229,0.388064860659106,4,12,8,15 +134,10,0,0.876947793298832,-0.782199031664106,4,12,8,15 +134,11,0,3.33604985416053,0.79350560497605,4,12,8,15 +134,12,0,3.60064769984994,2.15654542635504,4,12,8,15 +135,1,0,-2.01164114519903,-0.303718537411362,4,13,8,15 +135,2,0,-5.47146086271221,-1.56704737296407,4,13,8,15 +135,3,0,-3.00365121232252,-0.135122576576881,4,13,8,15 +135,4,0,-3.3411689767946,-1.23251797465687,4,13,8,15 +135,5,0,-0.719764304536301,-0.344517557176024,4,13,8,15 +135,6,0,-2.17646040262853,-1.06124437708116,4,13,8,15 +135,7,0,-1.79363718710398,-1.56087135482426,4,13,8,15 +135,8,0,-4.78495569487239,-0.43741571332793,4,13,8,15 +135,9,0,-3.09912620048905,0.523293643538835,4,13,8,15 +135,10,0,-5.02655499906869,-1.43754394732052,4,13,8,15 +135,11,0,-4.15230021533996,-0.372734703691673,4,13,8,15 +135,12,0,-2.50017594075563,-1.05473546678789,4,13,8,15 +136,1,0,0.432976214401498,-1.09091199750954,4,14,8,15 +136,2,0,0.0789848792945443,-0.396329603497487,4,14,8,15 +136,3,0,0.303024295750972,0.155768916585711,4,14,8,15 +136,4,0,-0.238531111611205,-0.123091459812907,4,14,8,15 +136,5,0,1.369973939758,0.225777192139813,4,14,8,15 +136,6,0,2.88765687423671,0.763293413616878,4,14,8,15 +136,7,0,2.13879538837293,-0.289314596556293,4,14,8,15 +136,8,0,1.32026596485962,-0.431541530739314,4,14,8,15 +136,9,0,-0.137769105334064,-0.190691293259283,4,14,8,15 +136,10,0,1.92452763778636,0.501200863132155,4,14,8,15 +136,11,0,1.14572672014925,-0.252218575791029,4,14,8,15 +136,12,0,1.12915198986992,0.622358309331802,4,14,8,15 +137,1,0,-2.39695376145854,-1.56041783966452,4,11,8,15 +137,2,0,0.350133935562058,1.67764112590093,4,11,8,15 +137,3,0,-2.47374823055531,-1.79336916735057,4,11,8,15 +137,4,0,-2.42331347300019,-1.6285887490902,4,11,8,15 +137,5,0,-2.00881454866982,-0.972472339620504,4,11,8,15 +137,6,0,-2.31379689753255,1.14626980751657,4,11,8,15 +137,7,0,0.226215401480258,-0.495323062587713,4,11,8,15 +137,8,0,-1.67003486128816,1.00869030101638,4,11,8,15 +137,9,0,-3.61878216121674,0.0483508014514602,4,11,8,15 +137,10,0,-2.16342187034336,0.479273429270487,4,11,8,15 +137,11,0,-2.77623992945622,0.540750247251341,4,11,8,15 +137,12,0,-1.55241262576006,0.00267937763287165,4,11,8,15 +138,1,0,-0.677275461084995,-1.36719230765297,4,12,8,15 +138,2,0,-0.462490380712306,0.487965503543606,4,12,8,15 +138,3,0,-1.76019710719793,0.516158057759977,4,12,8,15 +138,4,0,-0.651884670704395,0.809638982439653,4,12,8,15 +138,5,0,-0.779607750749956,1.84534508921484,4,12,8,15 +138,6,0,-2.86042743755234,-0.938364886296405,4,12,8,15 +138,7,0,-1.3301449930694,-1.79822803700857,4,12,8,15 +138,8,0,-1.67808881915926,-0.501000714545932,4,12,8,15 +138,9,0,1.30154680564024,2.04919864379435,4,12,8,15 +138,10,0,-2.01951698018562,0.434359263352257,4,12,8,15 +138,11,0,-2.20837136544348,-1.19473549085794,4,12,8,15 +138,12,0,0.680327952927733,1.17741410494148,4,12,8,15 +139,1,0,-1.12476301456957,-0.578606409521744,4,13,8,15 +139,2,0,-3.43922221064205,-0.604920157934706,4,13,8,15 +139,3,0,-3.25504658411351,-1.49900462466702,4,13,8,15 +139,4,0,-2.78178009365868,-0.366371213110164,4,13,8,15 +139,5,0,-2.98155062743131,-1.23581152940748,4,13,8,15 +139,6,0,-2.29543661692356,0.0203005892035529,4,13,8,15 +139,7,0,-0.0745154478268177,0.287809481883818,4,13,8,15 +139,8,0,0.714453956066579,1.75571348204949,4,13,8,15 +139,9,0,-0.833976852724573,0.356909445067566,4,13,8,15 +139,10,0,-2.64030154034902,0.702955997066231,4,13,8,15 +139,11,0,-1.34752480343306,0.445892766368817,4,13,8,15 +139,12,0,-0.334069391116475,1.28736381489041,4,13,8,15 +140,1,0,-2.62175995286002,-0.261682997611959,4,14,8,15 +140,2,0,-4.00015537052188,0.0747057977756218,4,14,8,15 +140,3,0,-4.73850090995721,-0.584132256082416,4,14,8,15 +140,4,0,-3.25656252955111,-0.333031708377073,4,14,8,15 +140,5,0,-3.12954045554489,0.890463747029763,4,14,8,15 +140,6,0,-5.86836169069293,0.353287683727294,4,14,8,15 +140,7,0,-2.83575935874424,0.127943842219712,4,14,8,15 +140,8,0,-4.55953127921804,0.439684776548471,4,14,8,15 +140,9,0,-3.45318276933482,1.11671382026538,4,14,8,15 +140,10,0,-6.0401061744516,-0.428445680593455,4,14,8,15 +140,11,0,-6.97363197468359,-2.78997966490289,4,14,8,15 +140,12,0,-3.23286102274776,0.37872967135766,4,14,8,15 diff --git a/docs/methodology/REGISTRY.md b/docs/methodology/REGISTRY.md index 378b3aa2..827dff52 100644 --- a/docs/methodology/REGISTRY.md +++ b/docs/methodology/REGISTRY.md @@ -1870,7 +1870,7 @@ The paper specifies no standard-error formula (Section 1 defers to "standard, we 5. **Note:** LP-DiD's per-unit quantities (outcome lags `ylags`, first-difference lags `dylags`, integer-`pmd` premean baselines, treatment-entry detection) are **calendar** quantities (`t-1`, `t-k`), so the estimator requires integer-valued, globally consecutive `time` labels. A unit with an **interior time gap** is handled by reindexing that unit to its complete interior calendar grid `[min_t, max_t]`, computing the features on the grid, then **restricting back to the observed rows** - so a lag/first-difference spanning a gap is NaN and the observation fails closed (never the previous-*observed* row), and no synthetic gap row enters a regression. A gap-free panel skips this entirely and is bit-identical. **Entry = first OBSERVED treated period** (`min(t | D_it=1)`): an unobserved pre-onset gap cannot move a cohort earlier, the only well-defined convention when the true switch falls in an unobserved period. 6. **Note (pooled estimand):** The pooled pre/post ATT (the headline `results.att` is the pooled-post row) is the **unit-equal-weighted average of each unit-event-time's mean long difference** over the window - `mean_h(y_{i,t+h}) - baseline_{i,t}`, one observation per (unit, event-time), regressed on the treatment-switch indicator with event-time fixed effects on the **fixed-composition** sample (only units observing *every* pooled target, with clean controls required through `max(h)`). This equals the mean of the per-horizon event-study coefficients on a balanced panel. **PR-B2 validated it against the authors' runnable R reference**: the pooled estimand matches the authors' own R pooled recipe (`danielegirardi/lpdid`: a `slider` window-mean minus `y_{t-1}` on the clean-through-window-end sample) to ~1e-13 (`tests/test_methodology_lpdid.py::test_pooled`). A prior version of this note speculated the authors used a horizon-**stacked** pooled regression; the authors' R reference in fact uses this same fixed-composition mean-long-difference, so that speculation was incorrect. Unlike the event-study variants (where `alexCardazzi` is a cross-check gate), pooled is anchored to the authors' recipe **only**: `alexCardazzi`'s pooled uses a **laxer** clean-control window, so it differs and is recorded in the golden `meta` for transparency, not as a parity target. 7. **Deviation from R:** `no_composition` is intentionally more faithful to the paper's fixed-composition intent (Section 3.6) than the R packages: it fixes the realized sample across *all* post horizons (every post coefficient shares one sample, even on unbalanced panels) and excludes cohorts with `p_g > T-H`, whereas `alexCardazzi/lpdid` uses a looser per-horizon sample and a stricter `treat_date < T-H` cutoff. It therefore has **no exact R-package anchor** and is validated by the pure-Python tests in `tests/test_lpdid.py` (the R-parity golden omits it; `alexCardazzi`'s looser-semantics value is recorded in the golden `meta`). -8. **Note (survey design):** Complex-survey support (`survey_design=SurveyDesign(...)`, PR-D1) covers the **variance-weighted default path** (`reweight=False`, with or without direct-inclusion covariates): each horizon's long-difference regression is fit by WLS on the survey probability weights, and the SE is the stratified-PSU **Taylor-linearization (Binder 1983 TSL)** sandwich `meat = sum_h (1-f_h)*(n_h/(n_h-1))*sum_j (S_hj - S_h_bar)(S_hj - S_h_bar)'` with `df = n_PSU - n_strata`, reusing the shared `diff_diff/survey.py` helpers (`compute_survey_vcov` / `_compute_stratified_psu_meat`). The design is re-resolved on each realized (post-clean-control) sample so weights/strata/PSU align with the regression rows; with no explicit PSU the unit (LP-DiD's default cluster) is injected as the PSU. Supports pweight + strata + PSU + FPC + lonely-PSU handling. It **rejects** `survey_design` combined with `reweight=True` (the equally-weighted / regression-adjustment IF path has no validated survey reference - the same gap as the RA SE in Deviation #2), replicate-weight designs, and non-pweight (fweight/aweight) types, each a deferred follow-up. The non-survey path is byte-for-byte unchanged (gated on `survey_design is None`). Anchored to `survey::svyglm` on the stacked long difference (numeric golden parity is PR-D2). +8. **Note (survey design):** Complex-survey support (`survey_design=SurveyDesign(...)`, PR-D1) covers the **variance-weighted default path** (`reweight=False`, with or without direct-inclusion covariates): each horizon's long-difference regression is fit by WLS on the survey probability weights, and the SE is the stratified-PSU **Taylor-linearization (Binder 1983 TSL)** sandwich `meat = sum_h (1-f_h)*(n_h/(n_h-1))*sum_j (S_hj - S_h_bar)(S_hj - S_h_bar)'` with `df = n_PSU - n_strata`, reusing the shared `diff_diff/survey.py` helpers (`compute_survey_vcov` / `_compute_stratified_psu_meat`). The design is re-resolved on each realized (post-clean-control) sample so weights/strata/PSU align with the regression rows; with no explicit PSU the unit (LP-DiD's default cluster) is injected as the PSU. Supports pweight + strata + PSU + FPC + lonely-PSU handling. It **rejects** `survey_design` combined with `reweight=True` (the equally-weighted / regression-adjustment IF path has no validated survey reference - the same gap as the RA SE in Deviation #2), replicate-weight designs, and non-pweight (fweight/aweight) types, each a deferred follow-up. The non-survey path is byte-for-byte unchanged (gated on `survey_design is None`). **PR-D2 validated all three survey paths end-to-end against `survey::svyglm`** - per-horizon point/SE/df + pooled for the variance-weighted full design (strata+PSU+FPC), the weights-only unit-injected-PSU design, and the direct-covariate variant (`tests/test_methodology_lpdid.py::TestLPDiDSurveyParityR`; point ~1e-6, SE ~1e-5, df exact via the per-design `n_PSU - n_strata` / `n_PSU - 1` formula). `svyglm` is itself the reference implementation of the Binder TSL sandwich, so it anchors the variance directly (no third-party survey-package gate is needed); the clean-sample construction is independently cross-checked in the generator (the unweighted variance-weighted event study matches `alexCardazzi/lpdid` to <1e-8, and selection is weight-independent). A dedicated survey panel (`benchmarks/data/lpdid_survey_panel.csv`, own seed) keeps the absorbing / non-absorbing goldens byte-identical. ### Implementation Checklist @@ -1889,7 +1889,7 @@ The paper specifies no standard-error formula (Section 1 defers to "standard, we - [x] Non-absorbing R-parity: both modes vs an independent `fixest::feols` Eq. 12/13 reconstruction (point+SE ~1e-13/~1e-15 vw; reweighted point + pinned SE); `alexCardazzi nonabsorbing_lag` recorded as a divergent reference (not a gate); absorbing B2 goldens byte-identical (PR-C2) - [ ] Non-absorbing exit-event dynamics (Appendix C `eta_h`) + the Stata canonical RA/SE - deferred - [x] Survey-design support (PR-D1): pweight + stratified-PSU Taylor-linearization (Binder TSL) variance on the variance-weighted default path; per-sample design re-resolution + unit-as-PSU injection; reweight (incl. RA), replicate-weight, and non-pweight designs rejected; pure-Python invariants (reduction/unit-clustering, FPC-shrinks-SE, stratification, lonely-PSU, NaN-consistency, metadata) (PR-D1) -- [ ] Survey-design R-parity: stratified-PSU TSL SEs vs `survey::svyglm` on the stacked long difference - deferred to PR-D2 +- [x] Survey-design R-parity (PR-D2): all three survey paths (VW strata+PSU+FPC, weights-only inject, direct-covariate) validated end-to-end vs `survey::svyglm` - per-horizon point/SE/df + pooled (point ~1e-6, SE ~1e-5, df exact); clean sample independently `alexCardazzi`-cross-checked (<1e-8); dedicated survey panel keeps the absorbing / non-absorbing goldens byte-identical (PR-D2) --- diff --git a/tests/test_methodology_lpdid.py b/tests/test_methodology_lpdid.py index 6f200819..c5fe91e0 100644 --- a/tests/test_methodology_lpdid.py +++ b/tests/test_methodology_lpdid.py @@ -32,12 +32,14 @@ from __future__ import annotations import json +import warnings from pathlib import Path import numpy as np import pandas as pd import pytest +from diff_diff import SurveyDesign from diff_diff.lpdid import LPDiD _DATA = Path(__file__).parent.parent / "benchmarks" / "data" @@ -55,6 +57,17 @@ NA_PANEL_PATH = _DATA / "lpdid_nonabsorbing_panel.csv" _NONABSORB_FIXTURE_AVAILABLE = NA_GOLDEN_PATH.is_file() and NA_PANEL_PATH.is_file() +# Phase D2: complex-survey R-parity. The merged D1 survey path (pweight + stratified-PSU +# Binder TSL sandwich, fit(survey_design=...)) is pinned against survey::svyglm goldens on +# a dedicated staggered-absorbing survey panel (separate seed -> the absorbing/non-absorbing +# fixtures above stay byte-identical). Each clean LP-DiD horizon uses a fresh svydesign over +# that horizon's sample; the clean-sample construction is independently alexCardazzi-cross- +# checked in the generator, and svyglm supplies the survey variance. Goldens record +# c(estimate, SE, df, n_psu) per horizon / pooled row. +SURVEY_GOLDEN_PATH = _DATA / "lpdid_survey_golden.json" +SURVEY_PANEL_PATH = _DATA / "lpdid_survey_panel.csv" +_SURVEY_FIXTURE_AVAILABLE = SURVEY_GOLDEN_PATH.is_file() and SURVEY_PANEL_PATH.is_file() + # Library regression-adjustment influence-function SE per horizon (reweight=True, # covariates=["x"]). NOT R-parity: the canonical RA SE is Stata teffects only, with # no runnable R analogue (see module docstring / REGISTRY ## LPDiD Deviation 2). @@ -130,6 +143,24 @@ def na_panel() -> pd.DataFrame: return pd.read_csv(NA_PANEL_PATH) +@pytest.fixture(scope="module") +def survey_golden() -> dict: + if not _SURVEY_FIXTURE_AVAILABLE: + pytest.skip( + "Survey LPDiD parity fixture not present. Run " + "`Rscript benchmarks/R/generate_lpdid_survey_golden.R`." + ) + with SURVEY_GOLDEN_PATH.open("r") as f: + return json.loads(f.read()) + + +@pytest.fixture(scope="module") +def survey_panel() -> pd.DataFrame: + if not _SURVEY_FIXTURE_AVAILABLE: + pytest.skip("Survey LPDiD parity fixture not present.") + return pd.read_csv(SURVEY_PANEL_PATH) + + def _es_map(res) -> dict: return { int(r.horizon): (float(r.coefficient), float(r.se)) for r in res.event_study.itertuples() @@ -353,3 +384,189 @@ def test_alex_recorded_as_divergent_reference(self, na_golden: dict) -> None: # (no-off-switch) sub-panel, recorded as committed evidence in meta. mono = _num(na_golden["meta"]["alex_monotone_post_divergence"]) assert mono > 1e-3, f"expected alex to diverge from Eq.13 on the monotone slice, got {mono}" + + +class TestLPDiDSurveyParityR: + """Pin the merged D1 survey path against ``survey::svyglm`` goldens (Phase D2). + + Three structurally-distinct survey variance paths are R-validated end-to-end + through ``LPDiD.fit(survey_design=...)``: + + * **VW** — full stratified-PSU design with FPC vs ``svyglm(ids=~psu, strata, + fpc, weights)``; reported ``n_clusters`` == n_PSU == 14, df == n_PSU - n_strata + == 10 (constant here). + * **inject** — weights-only, the unit injected as PSU, vs ``svyglm(ids=~unit, + weights)``; ``n_clusters`` == n_units == 140, df == n_PSU - 1 == 139. + * **covariate** — direct covariate inclusion vs ``svyglm(Dy ~ tr + x + + factor(time), ...)`` (emits the Appendix B.2.2 homogeneity warning). + + Each clean LP-DiD horizon uses a FRESH ``svydesign`` over that horizon's sample, + matching the library's per-sample ``survey_design.resolve(sample)``. The + clean-sample construction is independently alexCardazzi-cross-checked inside the + generator (the UNWEIGHTED VW event study matches to <1e-8); ``svyglm`` supplies the + survey variance, so it is the reference (no separate survey package gate needed). + + Goldens (``c(estimate, SE, df, n_psu)`` per horizon / pooled row) are generated by:: + + Rscript benchmarks/R/generate_lpdid_survey_golden.R + + There is no per-horizon ``df`` field in the result schema: ``event_study`` / + ``pooled`` rows expose ``n_clusters`` (= realized n_PSU), and df is reconstructed + with the design-specific formula (``n_clusters - n_strata`` with strata, else + ``n_clusters - 1``). Pure-numpy survey path -> backend-agnostic. + """ + + @staticmethod + def _fit(panel: pd.DataFrame, sd: SurveyDesign, **kw): + # The expected per-sample survey warnings (pweight normalization, PSU/cluster + # reconciliation) are not under test here; silence them so the parity signal is + # clean. The covariate variant asserts its homogeneity warning explicitly. + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + return LPDiD(pre_window=3, post_window=4, reweight=False, cluster="unit").fit( + panel, + outcome="y", + unit="unit", + time="time", + treatment="treat", + survey_design=sd, + **kw, + ) + + @staticmethod + def _assert_es(res, golden_es: dict, *, strata: bool) -> None: + """Point (``abs=1e-6``) + SE (``rtol=1e-5``/``abs=1e-7``) + n_PSU + df exact, per + horizon. df is reconstructed from the per-horizon ``n_clusters`` (= realized + n_PSU) with the design-specific formula.""" + rows = {int(r.horizon): r for r in res.event_study.itertuples()} + n_strata = int(res.n_strata) + assert golden_es, "golden variant has no horizons" + for hk, gv in golden_es.items(): + h = int(hk) + g_est, g_se, g_df, g_npsu = (_num(gv[i]) for i in range(4)) + assert h in rows, f"missing horizon {h}" + row = rows[h] + assert np.isfinite(float(row.coefficient)), f"h={h}: non-finite att" + assert float(row.coefficient) == pytest.approx(g_est, abs=1e-6), f"att h={h}" + assert float(row.se) == pytest.approx(g_se, rel=1e-5, abs=1e-7), f"se h={h}" + assert int(row.n_clusters) == int(g_npsu), f"n_psu h={h}" + df_recon = int(row.n_clusters) - (n_strata if strata else 1) + assert df_recon == int(g_df), f"df h={h}: recon {df_recon} != golden {int(g_df)}" + + @staticmethod + def _assert_pooled(res, golden_pooled: dict, *, strata: bool) -> None: + n_strata = int(res.n_strata) + g_post = golden_pooled["post"] + # post == the headline pooled estimate + assert np.isfinite(res.att) and np.isfinite(res.se) + assert float(res.att) == pytest.approx(_num(g_post[0]), abs=1e-6), "pooled-post att" + assert float(res.se) == pytest.approx(_num(g_post[1]), rel=1e-5, abs=1e-7), "pooled-post se" + post_row = res.pooled.loc[res.pooled["window"] == "post"].iloc[0] + # the pooled-post DataFrame row's own coefficient/se (not only the res.att/res.se + # headline that mirrors them) so a pooled-row schema regression cannot slip past + assert float(post_row["coefficient"]) == pytest.approx( + _num(g_post[0]), abs=1e-6 + ), "pooled-post row att" + assert float(post_row["se"]) == pytest.approx( + _num(g_post[1]), rel=1e-5, abs=1e-7 + ), "pooled-post row se" + assert int(post_row["n_clusters"]) == int(_num(g_post[3])), "pooled-post n_psu" + df_recon = int(post_row["n_clusters"]) - (n_strata if strata else 1) + assert df_recon == int(_num(g_post[2])), "pooled-post df" + # pre pooled row: point/SE AND n_psu/df (the df reconstruction has teeth for the + # inject variant where n_strata == 0) + g_pre = golden_pooled["pre"] + pre_row = res.pooled.loc[res.pooled["window"] == "pre"].iloc[0] + assert float(pre_row["coefficient"]) == pytest.approx( + _num(g_pre[0]), abs=1e-6 + ), "pooled-pre att" + assert float(pre_row["se"]) == pytest.approx( + _num(g_pre[1]), rel=1e-5, abs=1e-7 + ), "pooled-pre se" + assert int(pre_row["n_clusters"]) == int(_num(g_pre[3])), "pooled-pre n_psu" + df_recon_pre = int(pre_row["n_clusters"]) - (n_strata if strata else 1) + assert df_recon_pre == int(_num(g_pre[2])), "pooled-pre df" + + def test_vw_event_study_and_pooled( + self, survey_golden: dict, survey_panel: pd.DataFrame + ) -> None: + """Full stratified-PSU + FPC design vs svyglm; df == n_PSU - n_strata == 10.""" + sd = SurveyDesign(weights="weight", strata="stratum", psu="psu", fpc="fpc") + res = self._fit(survey_panel, sd) + # panel-level survey metadata + assert int(res.n_strata) == int(survey_golden["meta"]["n_strata"]) # 4 + assert int(res.n_psu) == int(survey_golden["meta"]["n_psu_total"]) # 14 + assert int(res.survey_metadata.df_survey) == int( + survey_golden["meta"]["n_psu_total"] + ) - int( + survey_golden["meta"]["n_strata"] + ) # 10 + self._assert_es(res, survey_golden["survey_vw_es"], strata=True) + self._assert_pooled(res, survey_golden["survey_vw_pooled"], strata=True) + + def test_inject_weights_only_event_study_and_pooled( + self, survey_golden: dict, survey_panel: pd.DataFrame + ) -> None: + """Weights-only design injects the unit as PSU vs svyglm(ids=~unit); the + ``_inject_cluster_as_psu`` rebind is exercised end-to-end. df == n_PSU - 1.""" + sd = SurveyDesign(weights="weight") + res = self._fit(survey_panel, sd) + assert int(res.n_strata) == 0 # weights-only -> the n_clusters-1 df branch + self._assert_es(res, survey_golden["survey_inject_es"], strata=False) + self._assert_pooled(res, survey_golden["survey_inject_pooled"], strata=False) + + def test_covariate_direct_event_study_and_pooled( + self, survey_golden: dict, survey_panel: pd.DataFrame + ) -> None: + """Direct covariate inclusion (reweight=False + covariates) vs svyglm with x; + the covariate-present full-rank kept-column survey sandwich is R-validated. The + Appendix B.2.2 homogeneity warning (REGISTRY ## LPDiD Deviation 3) is asserted.""" + sd = SurveyDesign(weights="weight", strata="stratum", psu="psu", fpc="fpc") + with pytest.warns(UserWarning, match="direct inclusion"): + res = LPDiD(pre_window=3, post_window=4, reweight=False, cluster="unit").fit( + survey_panel, + covariates=["x"], + outcome="y", + unit="unit", + time="time", + treatment="treat", + survey_design=sd, + ) + self._assert_es(res, survey_golden["survey_cov_es"], strata=True) + self._assert_pooled(res, survey_golden["survey_cov_pooled"], strata=True) + + def test_fpc_shrinks_se(self, survey_golden: dict, survey_panel: pd.DataFrame) -> None: + """A finite FPC shrinks the SE vs no-FPC, and the no-FPC path also matches svyglm. + Both SEs are independently pinned to svyglm, so this checks the FPC mechanism + end-to-end through the library, not just the golden.""" + sd_fpc = SurveyDesign(weights="weight", strata="stratum", psu="psu", fpc="fpc") + sd_nofpc = SurveyDesign(weights="weight", strata="stratum", psu="psu") + res_fpc = self._fit(survey_panel, sd_fpc) + res_nofpc = self._fit(survey_panel, sd_nofpc) + # the no-FPC path is itself svyglm-validated + self._assert_es(res_nofpc, survey_golden["survey_vw_nofpc_es"], strata=True) + se_fpc = { + int(r.horizon): float(r.se) + for r in res_fpc.event_study.itertuples() + if np.isfinite(r.se) + } + se_nofpc = { + int(r.horizon): float(r.se) + for r in res_nofpc.event_study.itertuples() + if np.isfinite(r.se) + } + assert se_fpc and se_fpc.keys() == se_nofpc.keys() + for h in se_fpc: + assert ( + se_fpc[h] < se_nofpc[h] + ), f"FPC should shrink the SE at h={h}: {se_fpc[h]} !< {se_nofpc[h]}" + + def test_weighting_moves_point(self, survey_panel: pd.DataFrame) -> None: + """The survey-weighted headline point differs from the unweighted VW point, so + the probability weights demonstrably flow into the WLS estimate (not ignored).""" + sd = SurveyDesign(weights="weight", strata="stratum", psu="psu", fpc="fpc") + res_survey = self._fit(survey_panel, sd) + res_unweighted = LPDiD(pre_window=3, post_window=4, reweight=False, cluster="unit").fit( + survey_panel, outcome="y", unit="unit", time="time", treatment="treat" + ) + assert abs(float(res_survey.att) - float(res_unweighted.att)) > 1e-3