feat(analysis): issue #21 zero handling + glmGamPoi port, and the KYAML pilot

[arena-ai-coding-agent]

Summary

Two pieces of work, on one branch, each with its own commit:

1. Issue feat(analysis): Advanced zero handling — glmGamPoi dispersion, Bayesian multiplicative replacement, multiplicative replacement with delta  #21 — advanced zero handling and the glmGamPoi dispersion port (ae8323d).
2. The KYAML pilot (cf5102f), the owner's ruling of 2026-09-26 making this repository the
   estate's pilot for the migration, plus a temporary CI probe (1e77a81).

1. Issue #21

Zero replacement, implemented rather than aliased (src/analysis/zero_replacement.jl):

• multiplicative replacement (Martín-Fernández, Barceló-Vidal & Pawlowsky-Glahn 2003), the
  operator of zCompositions::multRepl: zeros get delta × per-part detection limit,
  observed parts scale by 1 − Δ, and the sample total and the ratios among observed
  parts are preserved exactly;
• Bayesian multiplicative replacement (Martín-Fernández, Hron, Templ, Filzmoser &
  Palarea-Albaladejo 2015), the GBM of cmultRepl: posterior mean of a Dirichlet-multinomial
  with the leave-one-out prior mean and concentration 1/gmean(t) unless alpha is supplied,
  with the reference's frac × colmins cap and its adjust switch;
• both refuse by name what they cannot do (δ outside (0,1); an imputed mass that would consume
  the sample, naming the largest admissible δ; all-zero samples; never-observed parts; parts
  seen in fewer than two samples) and both record a full provenance block, including all
  replacement is biased.

Dispersion (src/analysis/dispersion.jl) is a pure-Julia port of glmGamPoi
(Ahlmann-Eltze & Huber 2020): Cox-Reid adjusted NB MLE with the reference's 0.99 factor and
its early returns, the dnorm-weighted local-median trend, the quasi-likelihood conversion,
the inverse-chisquare prior by Nelder-Mead. The natural-spline abundance trend is not
ported: true is refused, unset is refused at ≥ 100 features, false runs the reference's
own non-trended prior and records the deviation. estimation.jl's by-name refusal becomes the
real two-pass path (theta = 1/alpha, stats::glm(poisson()) where α is 0).

Proofs (proofs/agda/, Agda 2.7.0.1 + agda-stdlib 2.1.1, --safe, no postulates):
totals and ratios preserved; imputed values positive and below their limit; no
data-determined rule can be faithful (the fibre behind "everything is biased"); the shrinkage
lies between prior and estimate. proofs/agda/README.md records what is not proved and
what would falsify each file.

Surface: Julia config + Nickel contract + JSON schema + frontend types/expander (δ slider
with a replacement preview, α field, trend selector, estimated-runtime note), DEED echo,
DANGER banner at ≥3 δ trials, unit + integration tests, 100/1000/10000-taxa benchmark.

2. KYAML pilot

just use-kyaml / just use-yaml / just check-kyaml (scripts/kyaml/KYAML.jl), the drift
list (config/kyaml/drift.txt), and the operating manual (docs/pilots/kyaml-pilot.md).
Comments keep their association, --check is canonical-form checking (idempotent by
construction), refusals name file and line and write nothing, and the doc records what the
tools make the owner do: nothing, beyond one just use-kyaml.

What is verified, and what is not

Honest status: none of the Julia or R code in this PR has been executed. The authoring
sandbox has no Julia and no R (all distribution hosts are blocked), so:

• every Python code path was checked by reading, the Julia by tree-sitter parse only;
• the numeric expectations come from test/fixtures/issue21/golden.json, whose generator
  block states plainly that they are a pinned transcription of the published methods rather
  than a live call, and that test/reference/issue21_reference.jl (next, in Julia) is what
  turns them into a reproduced result;
• the Agda proofs were checked (all three modules, clean) on a hand-assembled Agda
  toolchain; the CI lane for them is written but needs the image built (below).

The first CI run is the debugger. Expect it to find something; the fixtures, the tests and
the bench are written so that what it finds is named rather than mysterious.

Blockers that are yours, not the branch's

1. CI cannot start in this repository at all, for anybody. Two repository settings:
   • Allowed actions: julia-actions/setup-julia, julia-actions/cache,
     julia-actions/julia-processcoverage are not allowed ("all actions must be from a
     repository owned by hyperpolymath, created by GitHub, verified in the Marketplace, or
     match arena-ai-coding-agent[bot]"). Your own push to main fails on this.
   • Allowed actors: "Actor is not allowed to trigger Actions workflows" for
     arena-ai-coding-agent[bot] — proven independent of the allow-list by
     .github/workflows/ci-probe.yml, a workflow with no uses: at all, which still
     fails startup.
     Both are in Settings → Actions → General. Until then this PR cannot be verified: the only
     Julia/R runners available are CI's.
2. The proofs lane needs the image: proofs job is gated on vars.STAPELN_AGDA_IMAGE;
   build Containerfile (per stapeln.toml) and set the variable. While it is unset the
   hygiene job prints the actionable line on every run, so the skip is visible rather than
   silent. The Containerfile has never been built (no container runtime here) — that is
   stated in the file itself.
3. just use-kyaml has not been run (it needs Julia). The conversion is one command; the
   CI gate step lands in the same commit as the conversion. Until then just check-kyaml
   fails as designed on unconverted files, and it is deliberately not in CI yet.

Testing

Gate	State
scripts/check-spdx.sh	OK (296 files)
scripts/check-format.sh	OK (358 files)
scripts/check-blob-hygiene.sh	ok (0 violations)
npx js-yaml .github/workflows/ci.yml	parses; jobs repo-hygiene, test, cicd-squabbler, proofs
tree-sitter Julia parse, all changed .jl	no errors
just prove-agda	clean locally on Agda 2.7.0.1 (all three modules, --safe); CI lane pending the image
just test	not run — no Julia in the authoring sandbox

Follow-ups (recorded, not hidden)

• Run just use-kyaml, commit the result, add the CI gate step.
• Extract the shell from .github/workflows/ci.yml into scripts/ci/*.sh (owner-approved
  style), then convert the workflows to KYAML.
• Delete .github/workflows/ci-probe.yml once the actor policy is relaxed and the probe has
  been run.
• test/reference/issue21_reference.jl — a Julia re-derivation of the fixture's numbers,
  deliberately by a different algorithm so agreement is evidence rather than a tautology.
• Port test/unit/test_exact_summaries.jl's python3 oracle to Julia (LANGUAGE-POLICY §3).

[coderabbitai]

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: c6de99d8-46fb-4eef-827d-fa446bb39631

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}