feat: per-sample reference support via manifest ref/ref_name columns

[mrvollger]

Summary

• The manifest can now have four columns: sample, bam, ref, ref_name. Each sample can use its own reference genome.
• The old two-column manifest keeps working; the config then supplies ref and ref_name for all samples. A . cell in the new columns falls back to the config value for that row. Filled manifest cells override config values (with an INFO line).
• Chromosome names and lengths now come only from each sample's BAM header, never from the fasta .fai. A BAM whose contigs are a subset of its fasta now works instead of producing silently empty per-chromosome output.
• A new genome_file rule writes a per-sample chrom.sizes file from the BAM header; all bedtools/bigtools sizes arguments use it. The fasta/fai remain only as true references (CRAM encoding, mosdepth).
• All reference and chromosome lookups are functions of the {sm} wildcard (get_ref(wc), get_ref_name(wc), get_chroms(wc), get_excludes(wc)), backed by an lru_cached pysam header reader. Manifest validation runs at parse time with FIRE:-prefixed errors (duplicate samples, pairing rules, malformed rows, missing ref/fai/bam).
• The trackhub now writes each sample's own ref_name into its genomes.txt; the hg38/GRCh38 auto-excludes are keyed per sample.

Supersedes #44.

Behavior change

Because chromosome lists are needed to build the DAG, every snakemake invocation (including dry-runs) now opens all manifest BAMs. Previously a completed run tolerated archived input BAMs. This mirrors the existing parse-time ref/fai checks and is documented in config/README.md.

Testing

• pixi run test-dry (new, runs first in CI): 21-check dry-run matrix — back-compat, 4-column, sentinel, override-INFO, levio parse coverage, and one error fixture per validator branch, plus rendered-shell assertions covering both excludes branches.
• pixi run test + pixi run test-verify (new in CI): the single-sample regression completes 49/49 rules with golden counts identical to main (1,181-line peaks file, 46,223 fire elements) — the chromosome-source swap is behaviorally invisible.
• pixi run test-multi (new, local): full three-sample run (136/136 rules) with per-sample genomes.txt (hg38/testref/testref2 via sentinel), the empty-excludes branch, and a generated chr20-only BAM proving per-sample chromosome sets.

Notes for review

• snakefmt's shell formatting became active with the current pixi env, so pixi run fmt reformatted shell blocks across the workflow — mechanical whitespace noise beyond the functional changes. Two pre-existing ((cat $(...) subshells in fire-peaks.smk were disambiguated to ( (cat ... (shfmt correctly rejects the former as arithmetic).
• The unused leviosam2 chain-lift (DSA) rules are removed entirely: nothing consumed their outputs, chain mode has been broken since rule fire stopped producing a bam, and a single global chain: cannot coexist with per-sample references.
• CI golden counts were measured locally on macOS; if the ubuntu conda builds differ, test-verify's numbers may need hardening from this PR's first CI run.

[mrvollger]

Second multi-agent review (ordering-focused, with empirical bedtools/bigtools/tabix experiments using chromosome names where header and lexicographic order disagree): no ordering-correctness bugs found. Decisive check: test vs test-rev (same reads, reversed header) outputs are byte-identical modulo row order — 1,180 peak rows and 3.75M pileup rows — and bedtools provably fails loudly, not silently, on order inconsistencies.

Three confirmed findings, fixed in the latest commit: (1) verify-multi.sh now asserts full content equality between test and test-rev, the one assertion that catches a silent ordering regression; (2) the output-order change (bam header order instead of lexicographic) is now documented in config/README.md with migration guidance — anchor downstream bedtools intersect -sorted with -g, and rerun pre-existing results dirs from scratch; (3) a manifest row with a uniformly extra column previously column-shifted silently via pandas index inference — now a loud parse error, with a fixture.

Migration notes (for the release)

• Output bed files follow the bam header order (hg38: chr1, chr2, …), not the lexicographic order (chr1, chr10, …) of earlier FIRE versions.
• Downstream bedtools intersect -sorted between FIRE outputs and lexicographically sorted annotations needs -g (or re-sorted inputs).
• Rerun old results directories from scratch; resuming a pre-0.3 results dir mixes sort orders and fails loudly mid-run.
• Input bams must stay readable for the lifetime of the results (headers are read at every pipeline start).