The per-site empirical error null is the highest-value remaining accuracy item. The tooling already exists (estimate-errors subcommand, monitor --error-table / --error-correction, detect._resolve_e_per_marker); what remains is the data and validation work, not a code build.
Context. The Step 27 bias filter drops whole loci on heuristic thresholds. The principled fix is a per-site error table: fraction-preserving calibration that down-weights (not drops) noisy loci by their own measured background, and regenerates from controls when the panel changes. run9/run10 ran without one (host_err_source=global-fallback), so a 2-3% background site like chr17:7676483 trips against the global floor.
Tasks
Fold in: per-site dropout weighting (cheap, marginal). The same host-free cohort supports a per-site dropout weight w_s = 1 - d_s (scale each marker's log-likelihood by its no-call rate). Dropout is only ~1.6% on the rhAmpSeq panel, so this is a quality-of-fit (CI) refinement, not a point-estimate lever. Worth doing here because it reuses the same cohort and down-weighting machinery; not worth a standalone effort. Implementation: marker_dropouts kwarg through the estimators, an estimate-dropout subcommand, a --dropout-table flag (missing sites default d_s = 0, backwards-compatible). Composes multiplicatively with bias correction.
Evidence: [data]-backed. Was Step 28 (plus Step 15 dropout, Step 24 GoF verification) in the retired overall plan.
The per-site empirical error null is the highest-value remaining accuracy item. The tooling already exists (
estimate-errorssubcommand,monitor --error-table/--error-correction,detect._resolve_e_per_marker); what remains is the data and validation work, not a code build.Context. The Step 27 bias filter drops whole loci on heuristic thresholds. The principled fix is a per-site error table: fraction-preserving calibration that down-weights (not drops) noisy loci by their own measured background, and regenerates from controls when the panel changes. run9/run10 ran without one (
host_err_source=global-fallback), so a 2-3% background site like chr17:7676483 trips against the global floor.Tasks
estimate-errorson host-free samples piled through the SAME admix path (forcedbcftools mpileup), NOT GATK (GATK reassembly hides the artifact). Host-free cohort = fully-reconstituted pure-donor timepoints PNOL (6_MO), GBRI (30_MO), QUDO-TP1 (5_MO); these emit the 7676483 background at full strength with zero host.monitor/run_csv_batch; confirmhost_err_source->per-site/mixedand the 7676483 calls collapse even with the bias filter off.Fold in: per-site dropout weighting (cheap, marginal). The same host-free cohort supports a per-site dropout weight
w_s = 1 - d_s(scale each marker's log-likelihood by its no-call rate). Dropout is only ~1.6% on the rhAmpSeq panel, so this is a quality-of-fit (CI) refinement, not a point-estimate lever. Worth doing here because it reuses the same cohort and down-weighting machinery; not worth a standalone effort. Implementation:marker_dropoutskwarg through the estimators, anestimate-dropoutsubcommand, a--dropout-tableflag (missing sites defaultd_s = 0, backwards-compatible). Composes multiplicatively with bias correction.Evidence:
[data]-backed. Was Step 28 (plus Step 15 dropout, Step 24 GoF verification) in the retired overall plan.