Skip to content
CI

fix(test): probe R once so the guard answers what the consumer asks #94

Sign in to view logs

fix(test): probe R once so the guard answers what the consumer asks

fix(test): probe R once so the guard answers what the consumer asks #94

Workflow file for this run

.github/workflows/ci.yml at b739f35
# SPDX-License-Identifier: AGPL-3.0-only
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
# A push to `main` QUEUES; a pull_request head update still cancels.
#
# Measured 2026-09-21: the Julia job takes ~31 min, and `main` was merged to at
# 14:25, 14:28 and 14:43. Every run was cancelled 17-18 min in by the next one,
# so three consecutive commits to `main` produced NO test verdict at all --
# not a pass, not a failure, nothing. Cancelling is right for a PR, where a
# verdict on a superseded head is worthless; it is wrong for `main`, where each
# commit is a thing we actually want a recorded answer about.
#
# Trade-off, stated plainly: pushes to `main` now run serially, so a burst of
# N merges takes N x ~31 min to drain. That is the cost of getting an answer.
# This does NOT rescue an upstream PR whose head keeps moving (e.g.
# JoshuaJewell#6, whose head IS this fork's `main`) -- only letting `main`
# settle for ~31 min does that.
cancel-in-progress: ${{ github.event_name != 'push' }}
jobs:
# Estate hygiene gates (standards/RSR alignment): cheap, run alongside the
# heavyweight Julia/frontend matrix. Each gate has a documented local
# equivalent — scripts/check-*.sh (see CONTRIBUTING.md).
#
# ENFORCING on this fork, ADVISORY upstream. The original blanket
# `continue-on-error: true` (67f2faaf) was correct about upstream and wrong
# about here: it left the fork with a check that literally could not fail,
# so its green carried no information. The expression below keeps the
# upstream guarantee — JoshuaJewell/MetaManifold-WebUI does not use
# conventional-commit / SPDX / format / lint as a merge gate, and a
# `pull_request` run against that base evaluates `github.repository` as the
# BASE repo, so the checks stay non-blocking for the owner's PR — while
# restoring real teeth on pushes and PRs to this fork.
# Local hooks remain available via core.hooksPath=.githooks.
repo-hygiene:
name: Repo hygiene (licence · format · lint · commit)
runs-on: ubuntu-24.04
continue-on-error: ${{ github.repository != 'hyperpolymath/MetaManifold-WebUI' }}
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
# Full history, deliberately: the commit convention check below grades
# the whole base..head range a PR proposes (#37), and a shallow clone
# would make that range unresolvable. The check treats an unresolvable
# range as a hard failure rather than grading zero commits, so the
# depth it depends on is pinned here instead of left to a default.
fetch-depth: 0
- name: Set up Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version-file: .bun-version
- name: Install frontend dependencies
working-directory: frontend
run: bun install --frozen-lockfile --ignore-scripts
- name: Licence header check
continue-on-error: ${{ github.repository != 'hyperpolymath/MetaManifold-WebUI' }}
run: scripts/check-spdx.sh
- name: Formatting check
continue-on-error: ${{ github.repository != 'hyperpolymath/MetaManifold-WebUI' }}
run: scripts/check-format.sh
- name: Lint check
continue-on-error: ${{ github.repository != 'hyperpolymath/MetaManifold-WebUI' }}
run: scripts/check-lint.sh
# Blob-hygiene mirror of .githooks/pre-commit. Both callers exec the SAME
# script, so the local gate and this one cannot disagree about what is
# allowed -- a CI check that re-implements a hook drifts from it, and the
# drift is invisible because both keep passing.
#
# --tree, not a commit range: a whole-tree check also catches anything
# that landed before the guard existed.
- name: Blob hygiene check
continue-on-error: ${{ github.repository != 'hyperpolymath/MetaManifold-WebUI' }}
run: scripts/check-blob-hygiene.sh --tree HEAD
# Commit convention mirror of .githooks/commit-msg (available locally via
# the hooksPath setting documented in CONTRIBUTING.md). Advisory only when
# this runs under upstream, whose commit subjects need not match.
#
# Grades EVERY commit a change proposes, not the tip (#37). The tip can be
# a merge commit that GitHub's own "Update branch" button created; grading
# it fails a conforming PR, while grading ONLY it lets a branch of nine
# non-conforming commits pass because the tenth is clean. Hence:
# - pull_request: the base..head range; push: the before..head range
# - merge commits excluded (--no-merges): nobody typed their subjects,
# and they cannot be rewritten without a force-push
# - every offender named by its sha
# - an unresolvable range, or one with zero non-merge commits, is a
# hard failure: a pass must mean the gate examined something, never
# that it declined to run
- name: Commit convention check
continue-on-error: ${{ github.repository != 'hyperpolymath/MetaManifold-WebUI' }}
env:
EVENT: ${{ github.event_name }}
HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
BASE_SHA: ${{ github.event.pull_request.base.sha }}
BEFORE_SHA: ${{ github.event.before }}
run: |
set -euo pipefail
pattern='^(feat|fix|docs|style|refactor|perf|test|build|ci|chore|revert)(\([a-zA-Z0-9_/-]+\))?!?: .{1,72}$'
if [ "$EVENT" = pull_request ]; then
range="$BASE_SHA..$HEAD_SHA"
elif [ -n "$BEFORE_SHA" ] && ! printf '%s' "$BEFORE_SHA" | grep -qE '^0+$'; then
range="$BEFORE_SHA..$HEAD_SHA"
else
# A new ref has no recorded before-sha: grade the tip alone.
range="$HEAD_SHA^..$HEAD_SHA"
fi
# Resolve both ends explicitly. If either is absent from this clone
# the gate cannot see the commits it was asked to grade, and must say
# so loudly rather than print a vacuous pass.
for sha in "${range%..*}" "${range#*..}"; do
if ! git rev-parse --verify --quiet "$sha^{commit}" >/dev/null; then
echo "::error::commit convention check: cannot resolve $sha in range $range; refusing to grade zero commits"
exit 1
fi
done
count="$(git rev-list --no-merges --count "$range")"
if [ "$count" -eq 0 ]; then
echo "::error::commit convention check: range $range contains no non-merge commits; refusing a vacuous pass"
exit 1
fi
commits="$(git log --no-merges --pretty=format:'%H%x09%s' "$range")"
fail=0
while IFS= read -r line; do
[ -n "$line" ] || continue
sha="${line:0:40}"
subject="${line:41}"
if ! printf '%s' "$subject" | grep -qE "$pattern"; then
echo "::error::commit $sha fails the conventional pattern: $subject"
fail=1
fi
done <<< "$commits"
if [ "$fail" -ne 0 ]; then
exit 1
fi
echo "commit convention ok: $count non-merge commit(s) graded in $range"
test:
# The name is a fixed string, it carries no version, and the job has NO matrix.
# Both are required, and the first without the second is a trap that has already
# been sprung here once.
#
# A branch ruleset matches a required status check by the DISPLAY NAME GitHub posts
# for the job. A renamed check does not fail -- it is simply absent, and an absent
# required check can never be satisfied, so every pull request deadlocks until an
# admin bypasses the rule, triggered by nothing more alarming than editing a version
# number.
#
# MEASURED on PR #39 (2026-09-21): this job, named exactly `Julia tests` but still
# carrying a 1x1 `strategy.matrix`, posted `Julia tests (1.12.5, ubuntu-24.04)`.
# GitHub appends the matrix combination whenever the name does not already reference
# the matrix, so a 1x1 matrix is still a matrix and the pins were still embedded in
# the check name. The matrix was therefore removed rather than merely renamed around:
# it selected exactly one combination and bought nothing.
#
# The two values it held are literals below, each keeping its own reasoning.
# `test/unit/test_install_pins.jl` asserts, for EVERY job in this file, that the name
# interpolates nothing AND that the job has no matrix -- so this cannot regress
# silently. If a matrix is ever genuinely wanted here, the required context must move
# to a matrix-free aggregator job first.
name: Julia tests
# Explicit, not ubuntu-latest: the R pin below names an apt package revision built
# for 24.04, and a runner that silently rolled to the next LTS would take that pin
# with it. `runs-on` cannot read `env`, so this is a literal.
runs-on: ubuntu-24.04
# The repository's root .Rprofile sources renv/activate.R, so R started from
# the checkout auto-activates renv and rewrites the library paths. renv is a
# local-development convenience; CI installs into the system library and
# reaches it through R_LIBS_SITE, so the autoloader is switched off here.
# This variable covers the unprivileged R that Julia embeds through RCall.
# It cannot cover the installs themselves, which run under sudo: sudo resets
# the environment, so those commands pass --no-init-file instead.
env:
RENV_CONFIG_AUTOLOADER_ENABLED: "FALSE"
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- name: Set up Julia
uses: julia-actions/setup-julia@4c0cb0fce8556fdb04a90347310e5db8b1f98fb9 # v2
with:
# Pinned rather than tracking latest stable, because a pinned Manifest.toml
# resolved by an unpinned Julia is not a reproducible build. This must equal
# julia_version in Manifest.toml; it cannot be read from the pin file, since
# nothing can be read before Julia exists. test/unit/test_install_pins.jl
# fails if the three ever disagree.
version: "1.12.5"
- name: Cache Julia packages
uses: julia-actions/cache@d10a6fd8f31b12404a54613ebad242900567f2b9 # v2
# Static source lint, deliberately dependency-free (`--project=no`) so it can
# run before instantiation and fail in seconds rather than after the ~26
# minute build-and-test pipeline. Every check corresponds to a defect class
# that previously reached CI and cost a full run to diagnose:
# - module/struct name collision (the `using MetaManifold.AnalysisConfig`
# shadowing that killed 80+ qualified references)
# - `\$var` inside interpolating strings (silently degraded 50 diagnostics
# and broke tests matching on the interpolated value)
# - two adjacent docstrings ("cannot document the following expression")
# - packages used in src/ but missing from Project.toml
# - modules referenced by tests but never imported (UndefVarError)
# - files that do not parse
- name: Source lint (fail fast, no dependencies)
run: julia --project=no --startup-file=no config/ci/lint_source.jl
# Every external version CI installs is read from the committed pin file, so CI
# and a developer's machine cannot drift apart. A temporary environment is used
# because the project environment cannot be instantiated until R is present:
# RCall's build step requires it.
- name: Read pinned versions
run: |
julia -e '
using Pkg
Pkg.activate(temp = true)
Pkg.add("YAML")
using YAML
pins = YAML.load_file("config/defaults/tool_versions.yml")
open(ENV["GITHUB_ENV"], "a") do io
println(io, "R_APT_VERSION=", pins["runtimes"]["r"]["apt_version"])
println(io, "BUN_VERSION=", pins["toolchain"]["bun"]["version"])
println(io, "CUTADAPT_SPEC=cutadapt==", pins["tools"]["cutadapt"]["version"])
println(io, "MULTIQC_SPEC=multiqc==", pins["tools"]["multiqc"]["version"])
# FastQC ships one platform-neutral zip, so its archive hangs off the
# "any" key rather than a per-platform one, and it cannot join the loop
# below. test_install_pins.jl asserts that shape, so a pin file that
# grew a linux-x86_64 fastqc entry would fail there before it failed here.
fastqc = pins["tools"]["fastqc"]["archives"]["any"]
println(io, "FASTQC_VERSION=", pins["tools"]["fastqc"]["version"])
println(io, "FASTQC_URL=", fastqc["url"])
println(io, "FASTQC_SHA256=", fastqc["sha256"])
for tool in ("vsearch", "swarm")
archive = pins["tools"][tool]["archives"]["linux-x86_64"]
println(io, uppercase(tool), "_VERSION=", pins["tools"][tool]["version"])
println(io, uppercase(tool), "_URL=", archive["url"])
println(io, uppercase(tool), "_SHA256=", archive["sha256"])
end
end'
# R + required packages (needed by RCall)
# Install R directly via apt to avoid r-lib/actions/setup-r mangling
# R_HOME and stripping default packages (utils, methods, stats, etc.)
# The version is pinned to the apt revision named in the pin file, which tracks
# R.Version in renv.lock; CRAN's Ubuntu repository retains older revisions, so
# this resolves rather than merely requesting the newest.
- name: Set up R
run: |
wget --https-only -qO- https://cloud.r-project.org/bin/linux/ubuntu/marutter_pubkey.asc \
| sudo gpg --dearmor -o /usr/share/keyrings/r-project.gpg
echo "deb [signed-by=/usr/share/keyrings/r-project.gpg] https://cloud.r-project.org/bin/linux/ubuntu $(lsb_release -cs)-cran40/" \
| sudo tee /etc/apt/sources.list.d/r-project.list
sudo apt-get update -qq
sudo apt-get install -y \
r-base-core="$R_APT_VERSION" \
r-base-dev="$R_APT_VERSION" \
r-base-html="$R_APT_VERSION" \
r-recommended="$R_APT_VERSION" \
r-base="$R_APT_VERSION"
Rscript -e 'cat(R.version.string, "\n")'
- name: Install R system dependencies
run: sudo apt-get install -y libcurl4-openssl-dev libssl-dev libxml2-dev libfontconfig1-dev
# The R packages are restored from renv.lock, the same pin a developer restores
# from, rather than resolved against whatever the repositories serve that day.
# Asking for a bare package name pins nothing: a Bioconductor release is frozen,
# so dada2 and its kin held still, but CRAN publishes only its newest revision,
# so vegan silently moved off the pinned 2.7-3 and test_provenance caught the
# drift. Restoring into .Library keeps the system library that the RCall steps
# and R_LIBS_SITE below expect, and renv is invoked explicitly here rather than
# left to activate itself. --no-init-file stops R sourcing the repository's
# .Rprofile, and with it the renv autoloader, which would otherwise rebind
# .Library to its own sandbox under root's cache; the flag is used rather than
# the environment variable above because sudo resets the environment first.
# renv keeps a content-addressed cache: a package already in it is linked into
# the library instead of being downloaded and compiled again. Persisting that
# cache across runs is what makes a failed restore RESUMABLE. Without it all 79
# packages are rebuilt from source on every run -- measured at 10m08s and 13m21s
# on two consecutive runs, the largest single cost in this workflow -- and a
# restore that dies at package 60 throws away all 60.
#
# Reordering the packages is not an alternative: renv derives install order from
# the dependency graph, so a package cannot be pulled to the front of the queue
# ahead of the packages it links against.
#
# The cache path is pinned rather than left at the default ~/.cache/R/renv,
# because the restore runs under sudo where ~ is root's home, not the runner's.
- name: Restore the renv cache
id: renv-cache
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
with:
path: /opt/renv-cache
# The run id keeps every key unique so each attempt writes a NEW entry and
# progress accumulates; restore-keys then picks the most recent entry that
# matches the prefix. A changed renv.lock still falls through to the second
# key and re-uses every package whose version did not move.
key: renv-${{ runner.os }}-R${{ env.R_APT_VERSION }}-${{ hashFiles('renv.lock') }}-${{ github.run_id }}
restore-keys: |
renv-${{ runner.os }}-R${{ env.R_APT_VERSION }}-${{ hashFiles('renv.lock') }}-
renv-${{ runner.os }}-R${{ env.R_APT_VERSION }}-
- name: Install R packages
env:
RENV_PATHS_CACHE: /opt/renv-cache
# renv draws its download counter by hiding the cursor (ESC[?25l), rewriting
# the line in place, then showing it again (ESC[?25h) -- the "25l25h" residue
# that litters the log. A captured CI log is not a terminal, so the rewrite
# never lands and the counter appears frozen at (0/79) for the whole ten
# minutes while the download is in fact progressing. Disabling cli's dynamic
# output makes each update print on its own line, so the log shows real
# progress and a genuine hang becomes distinguishable from a working step.
R_CLI_DYNAMIC: "false"
TERM: dumb
run: |
sudo mkdir -p "$RENV_PATHS_CACHE"
sudo chmod 777 "$RENV_PATHS_CACHE"
sudo env R_CLI_DYNAMIC=false TERM=dumb Rscript --no-init-file -e 'install.packages(c("renv", "BiocManager"), repos="https://cloud.r-project.org", lib=.Library)'
# `sudo env VAR=...` rather than `sudo -E` or a bare VAR=value prefix: it sets
# the variable for Rscript directly and so does not depend on the runner's
# sudoers env_reset policy, which the comment above already notes resets it.
sudo env RENV_PATHS_CACHE="$RENV_PATHS_CACHE" R_CLI_DYNAMIC=false TERM=dumb Rscript --no-init-file -e 'renv::restore(project=".", library=.Library, prompt=FALSE)'
Rscript --no-init-file -e 'for (pkg in c("dada2","Biostrings","ShortRead","vegan","dplyr")) if (!require(pkg,character.only=TRUE,quietly=TRUE)) stop(pkg, " failed to install")'
# Both of the next two steps run on failure ON PURPOSE. actions/cache saves only
# when the job succeeds, which would discard exactly the partial progress this
# cache exists to preserve: the packages that DID build before the restore died
# are the ones the next attempt must not build again. The cache is written by
# root, so it is made readable before it is packed.
- name: Make the renv cache readable
if: always()
run: sudo chmod -R a+rX /opt/renv-cache || true
- name: Save the renv cache
if: always()
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
with:
path: /opt/renv-cache
key: renv-${{ runner.os }}-R${{ env.R_APT_VERSION }}-${{ hashFiles('renv.lock') }}-${{ github.run_id }}
- name: Install cutadapt
run: pip install "$CUTADAPT_SPEC"
# The one tool CI does not pin. Ubuntu ships no versioned cd-hit revision worth
# naming, and 24.04 is frozen, so its cd-hit does not move under us; the version
# that arrives is recorded at preflight, where a discrepancy is visible.
- name: Install cd-hit
run: sudo apt-get install -y cd-hit
# Downloaded against the pinned URL and refused unless it hashes to the pinned
# SHA256, so CI runs the same bytes install.jl puts on a developer's machine.
- name: Install vsearch
run: |
curl -sSL -o vsearch.tar.gz "$VSEARCH_URL"
echo "$VSEARCH_SHA256 vsearch.tar.gz" | sha256sum -c -
tar xzf vsearch.tar.gz --warning=no-unknown-keyword
sudo mv "vsearch-$VSEARCH_VERSION-linux-x86_64/bin/vsearch" /usr/local/bin/vsearch
rm -rf vsearch.tar.gz "vsearch-$VSEARCH_VERSION-linux-x86_64"
vsearch --version
- name: Install swarm
run: |
curl -sSL -o swarm.tar.gz "$SWARM_URL"
echo "$SWARM_SHA256 swarm.tar.gz" | sha256sum -c -
tar xzf swarm.tar.gz --warning=no-unknown-keyword
sudo mv "swarm-$SWARM_VERSION-linux-x86_64/bin/swarm" /usr/local/bin/swarm
rm -rf swarm.tar.gz "swarm-$SWARM_VERSION-linux-x86_64"
swarm --version
- name: Install multiqc
run: pip install "$MULTIQC_SPEC"
# FastQC is a Perl launcher round a Java jar, so it is installed as a directory
# rather than a single binary: the launcher resolves its jars relative to its own
# location, and moving it out of FastQC/ breaks it. Unpacked to /opt and reached
# through a symlink, which is what config/ci/tools.yml means by `path: "fastqc"`.
# The launcher runs `java` off PATH, so `fastqc --version` proves both that the
# install landed and that a JRE is present on the runner.
- name: Install fastqc
run: |
curl -sSL -o fastqc.zip "$FASTQC_URL"
echo "$FASTQC_SHA256 fastqc.zip" | sha256sum -c -
sudo unzip -q -d /opt fastqc.zip
rm -f fastqc.zip
sudo chmod +x /opt/FastQC/fastqc
sudo ln -sf /opt/FastQC/fastqc /usr/local/bin/fastqc
fastqc --version
- name: Instantiate Julia project
run: julia --project=. -e 'import Pkg; Pkg.instantiate()'
# Smoke test: precompile and load the package before committing to the full
# suite. An undeclared dependency or a load-time lowering error used to
# surface only partway through the ~9 minute test run; this catches it in
# about a minute, and proves the precompile cache CI just built is usable.
- name: Precompile and load smoke test
run: julia --project=. -e 'using MetaManifold; @info "MetaManifold precompiled and loaded" version=string(pkgversion(MetaManifold))'
- name: Set up Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
with:
bun-version: ${{ env.BUN_VERSION }}
- name: Install frontend dependencies
working-directory: frontend
run: bun install --frozen-lockfile --ignore-scripts
# Explicit strict-typecheck gate (strict foundation; see
# docs/types/strict-mode-status.md). Runs before the (heavier) build so
# type regressions fail fast instead of surfacing inside vite's bundling.
- name: Typecheck frontend
working-directory: frontend
run: bun run typecheck
# Scaffold smoke battery (bun:test) — machine-readable results + coverage
# profile, both shipped as artifacts. No coverage gate yet by design
# (docs/testing/infrastructure.md).
- name: Test frontend
working-directory: frontend
run: bun test --coverage --coverage-reporter=lcov --coverage-dir tests/coverage --reporter=junit --reporter-outfile tests/results/junit.xml
# Benchmark harness (proven-tests-and-benches discipline): medians vs committed baseline; JSON result ships as an artifact.
# Milestone 2: now includes table_loading, epistemic_parsing, duckdb_aggregation, permanova_nmds, tree_rendering workloads
- name: Benchmark frontend
working-directory: frontend
run: bun run bench -- --json bench/results/results.json
# Benchmark deltas vs the committed baseline are REPORTED, never gated.
# The harness workloads (frontend/bench/index.ts) are frozen and
# self-contained — they import no app code — so a delta cannot be caused
# by PR contents; it measures the runner, not the commit. Evidence on
# hosted ubuntu-24.04: two consecutive runs of identical benchmark code
# produced per-workload deltas between -16% and +52%, flapping in both
# directions (median-of-5 samples of 2-8 ms on shared vCPUs ride
# co-tenant throttling). A hard 10% gate below that noise floor can only
# block at random — which the harness header anticipates ("Baseline
# comparison is INFORMATIONAL only — there is no regression gate"). The
# real signals remain: checksums (hard-fail), the freeze policy
# (workload edits must re-cut the baseline and are visible in review),
# and the deltas + machine factor printed here and shipped as artifacts
# for human review. A same-runner A/B gate can revisit once the harness
# exists on the base branch (measure the base in-job, compare like-for-like).
- name: Report frontend benchmark deltas (informational)
working-directory: frontend
run: |
node -e '
const fs = require("fs");
const baselinePath = "bench/baseline.json";
const resultsPath = "bench/results/results.json";
if (!fs.existsSync(baselinePath) || !fs.existsSync(resultsPath)) {
console.log("No baseline or results — nothing to report (first run)");
process.exit(0);
}
const baseline = JSON.parse(fs.readFileSync(baselinePath, "utf8"));
const results = JSON.parse(fs.readFileSync(resultsPath, "utf8"));
const deltas = [];
for (const r of results.results) {
const b = baseline.results.find(x => x.name === r.name);
if (!b) { console.log(`NEW ${r.name}: no baseline entry`); continue; }
const delta = (r.median_ns - b.median_ns) / b.median_ns * 100;
deltas.push({ name: r.name, delta, base: b.median_ns, cur: r.median_ns, checksum: r.checksum });
}
if (!deltas.length) process.exit(0);
// Machine factor: median delta across workloads — the systematic
// speed offset of this runner vs the one that cut the baseline.
const sorted = [...deltas].sort((a, b) => a.delta - b.delta);
const machine = sorted[Math.floor(sorted.length / 2)].delta;
console.log(`machine factor (median delta): ${machine >= 0 ? "+" : ""}${machine.toFixed(1)}%`);
for (const d of deltas) {
const rel = d.delta - machine;
console.log(`${d.name}: ${d.delta >= 0 ? "+" : ""}${d.delta.toFixed(1)}% vs baseline ${d.base} ns (current ${d.cur} ns)`);
if (!d.checksum) {
console.error(`::error::checksum failure in workload ${d.name} — workload output changed`);
process.exitCode = 1;
}
if (rel > 25) {
console.error(`::warning::${d.name} runs ${rel.toFixed(1)}pp above machine factor — expected wobble on shared runners; investigate only if bench/index.ts changed`);
}
}
'
- name: Upload frontend test & benchmark artifacts
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: frontend-tests-benchmarks
path: |
frontend/tests/results/junit.xml
frontend/tests/coverage/lcov.info
frontend/bench/results/results.json
frontend/bench/baseline.json
if-no-files-found: warn
- name: Build frontend
working-directory: frontend
run: bun run build
- name: Download PR2 databases
run: julia --project=. config/ci/download_databases.jl
- name: Rebuild RCall against installed R
run: julia --project=. -e 'import Pkg; Pkg.build("RCall")'
- name: Verify R packages visible from RCall
run: |
julia --project=. -e '
using RCall
R"cat(.libPaths(), sep=\"\n\")"
for pkg in ["dada2", "dplyr", "vegan", "tibble"]
R"if (!require($pkg, character.only=TRUE, quietly=TRUE)) stop($pkg, \" not found\")"
end
'
- name: Run tests
env:
CI_SKIP_TAXONOMY: "1"
R_LIBS_SITE: /usr/local/lib/R/site-library:/usr/lib/R/site-library:/usr/lib/R/library
run: julia --project=. -t 2 --code-coverage=user --compiled-modules=no test/runtests.jl --integration --server
- name: Process coverage
uses: julia-actions/julia-processcoverage@03114f09f119417c3242a9fb6e0b722676aedf38 # v1
- name: Upload coverage artifact (local, Codecov removed per Milestone 2)
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: julia-coverage-lcov
path: lcov.info
if-no-files-found: warn
# Comprehensive benchmarks (Milestone 2) — table loading, epistemic parsing, DuckDB aggregation, PERMANOVA/NMDS, tree rendering
- name: Benchmark Julia comprehensive
run: |
julia --project=. bench/table_loading/benchmark.jl
julia --project=. bench/epistemic_parsing/benchmark.jl
julia --project=. bench/duckdb_aggregation/benchmark.jl
julia --project=. bench/permanova_nmds/benchmark.jl
julia --project=. bench/tree_rendering/benchmark.jl
# bench/analysis_config existed but was never run here, which is why its
# 13 broken qualified references survived until the unit tests tripped
# over the same shadowing. Running it means the next AnalysisConfig API
# change breaks this step immediately instead of at test time.
julia --project=. bench/analysis_config/benchmark.jl
julia --project=. bench/comprehensive_benchmark.jl
- name: Summarise Julia benchmark deltas (informational)
run: |
echo "Julia benchmark deltas are informational (same cross-host noise argument as the frontend step above; the bench scripts no longer exit non-zero on >10%)."
# Record that baseline.json files exist for each category
for cat in table_loading epistemic_parsing duckdb_aggregation permanova_nmds tree_rendering; do
if [ ! -f bench/$cat/baseline.json ]; then
echo "::warning::No baseline.json for $cat — first run will create it"
else
echo "Found baseline for $cat: $(cat bench/$cat/baseline.json | head -c 200)"
fi
done
if [ -f bench/results/comprehensive_results.json ]; then
echo "Comprehensive results: $(cat bench/results/comprehensive_results.json | head -c 500)"
fi
- name: Upload Julia benchmark artifacts
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: julia-benchmarks-comprehensive
path: |
bench/*/baseline.json
bench/results/comprehensive_results.json
bench/**/baseline.json
if-no-files-found: warn
# Milestone 2 category. Deliberately NOT wrapped in `if [ -f … ]`: that
# shape reports success when the file is absent, so it cannot distinguish
# "passed" from "never ran". If the file is deleted, `include` raises and
# this step goes red, which is the intended behaviour.
#
# The companion `test_clade_cumulus.jl` step was removed: the file has
# never existed on main (only src/analysis/clade_cumulus.jl), so the step
# was a permanent no-op reporting success. Restore it alongside the file.
- name: Test analysis-config category
run: |
julia --project=. -e 'using Test; using MetaManifold; include("test/unit/test_analysis_config.jl")'
# ─────────────────────────────────────────────────────────────────────────
# cicd-squabbler — gate-deadlock triage
# https://github.com/hyperpolymath/cicd-squabbler (MPL-2.0)
#
# WHAT THIS DOES AND DOES NOT DO.
#
# squabbler will NOT turn a red test suite green, and that is deliberate: its
# charter puts red→green *code* fixes out of scope for v0.1 because "fixing" a
# failing test by weakening it would violate the squabble ≠ bypass invariant
# (proved in SPARK: the only transition into Green is a required check that
# actually ran and passed). Do not read a green triage job as a green build.
#
# What it does cover is the *gate* layer, which is a distinct failure mode from
# a failing test: a required check whose name drifted from what the workflow
# emits, an `on.*.paths` filter that strands a required check so it never runs,
# a reusable workflow pinned to a stale SHA, and modify/delete or rebase
# conflicts. Those deadlock a PR that is otherwise fine, and they are invisible
# from inside the test job.
#
# This runs in propose mode. `--apply` is intentionally NOT used: it only
# enacts the path-filter self-win by editing a workflow file, and never
# commits, pushes or re-runs CI, so on a runner it would edit a checkout that
# is then discarded. Landing a move stays a human decision.
#
# ubuntu-latest ships a Rust toolchain, so no third-party setup action is
# needed and the SHA-pinning convention of this file is preserved.
# ─────────────────────────────────────────────────────────────────────────
cicd-squabbler:
name: Gate triage (cicd-squabbler)
runs-on: ubuntu-latest
needs: [test]
# Gate triage needs a PR: every substantive step below takes <owner>/<repo>
# <pr-number>. On a push it could only build squabbler and run a bundled
# fixture, then report 'Gate triage: success' having triaged nothing.
# !cancelled() stays so triage still runs when the test job FAILS — that
# is the case it exists for.
if: ${{ !cancelled() && github.event_name == 'pull_request' }}
permissions:
contents: read
actions: read
checks: read
pull-requests: read
env:
GH_TOKEN: ${{ github.token }}
# Pinned, matching this file's convention for every other external ref.
# FLOOR: this pin must be at or after hyperpolymath/cicd-squabbler#99
# (merged 2026-09-21T18:55Z as 9846169c), which is what introduced the
# distinct exit 3 the Fetch step below branches on. At any earlier
# revision "no gate" is exit 2, falls into the `*` arm, and hard-fails --
# i.e. moving this pin backwards silently reverts the fix below without
# touching it. Bump it forwards freely; never behind 9846169c.
SQUABBLER_SHA: 9846169c1dc8e549edd72fa20efc6680d324d484
SQUABBLE: /tmp/squabbler/target/release/squabble
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
- name: Build squabble at the pinned revision
run: |
git clone --quiet https://github.com/hyperpolymath/cicd-squabbler.git /tmp/squabbler
git -C /tmp/squabbler checkout --quiet "$SQUABBLER_SHA"
echo "building cicd-squabbler at $(git -C /tmp/squabbler rev-parse HEAD)"
cargo build --release --locked --quiet --manifest-path /tmp/squabbler/Cargo.toml -p squabble-cli
"$SQUABBLE" --version
# Proves the engine itself works before we trust its verdict on this repo.
# Same fixture the upstream `just demo` recipe uses.
- name: Engine self-check (bundled gate-deadlock fixture)
run: |
"$SQUABBLE" diagnose /tmp/squabbler/examples/gate-deadlock.json
# `squabble fetch` and `squabble fight` both take <owner>/<repo>
# <pr-number>; the job-level gate above guarantees a PR is present.
#
# Exit codes, fixed by SHA pin so their meaning cannot drift underneath:
# 0 a gate exists and was fetched -> triage it
# 3 the base branch has no `required_status_checks` ruleset rule
# -> there is nothing to triage. A finding, not a breakage.
# * anything else is a real failure and still fails this job.
#
# Before hyperpolymath/cicd-squabbler#99 every one of those was exit 2, so
# this step could not tell "nothing to triage" from "squabble is broken".
# It went red on a legitimate non-finding, and the only alternative was to
# swallow rc=2 -- which would have muted genuine breakage along with it.
# Discriminating on the message, or inferring the state from the error
# code, would both be guesses; the producer answers it instead.
- name: Fetch the live gate for this PR
id: fetch
run: |
set +e
"$SQUABBLE" fetch "${{ github.repository }}" \
"${{ github.event.pull_request.number }}" > gate.json 2> fetch.err
rc=$?
set -e
cat fetch.err >&2
case "$rc" in
0)
echo "has_gate=true" >> "$GITHUB_OUTPUT"
echo "--- gate.json ---"; cat gate.json
;;
3)
echo "has_gate=false" >> "$GITHUB_OUTPUT"
rm -f gate.json
{
echo "## Gate triage: no gate to triage"
echo
echo "\`squabble fetch\` exited 3. Base branch \`${{ github.event.pull_request.base.ref }}\`"
echo "of \`${{ github.repository }}\` carries no \`required_status_checks\` ruleset rule,"
echo "so there is no gate to squabble over and triage was skipped."
echo
echo "**This job is green because nothing was triaged, not because a gate passed.**"
echo "Merges into that branch are gated by no required status check."
echo
echo "Classic branch protection is a separate API and is not visible to this query,"
echo "so this says nothing about it."
} | tee triage-outcome.md >> "$GITHUB_STEP_SUMMARY"
echo "::warning title=No gate to triage::base branch has no required_status_checks ruleset rule -- triage skipped, nothing was verified"
;;
*)
# The redirect leaves a 0-byte gate.json even when the fetch
# failed; uploading it would look like an empty gate was fetched.
rm -f gate.json
echo "::error title=squabble fetch failed::exit $rc -- this is a real failure, not a missing gate"
exit "$rc"
;;
esac
- name: Diagnose the gate
if: steps.fetch.outputs.has_gate == 'true'
run: |
"$SQUABBLE" diagnose gate.json | tee squabble-diagnose.txt
# Propose only. `|| true` because fight exits non-zero when it has work it
# cannot legitimately land — that is a finding to report, not a build break,
# and failing here would mask the very deadlock we are trying to surface.
- name: Fight (propose only — never commits, pushes or re-runs CI)
if: steps.fetch.outputs.has_gate == 'true'
run: |
"$SQUABBLE" fight "${{ github.repository }}" "${{ github.event.pull_request.number }}" \
--repo-root . --json > squabble-fight.json || true
echo "--- fight report ---"; cat squabble-fight.json || true
- name: Upload triage report
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: cicd-squabbler-report
path: |
gate.json
squabble-diagnose.txt
squabble-fight.json
triage-outcome.md
if-no-files-found: ignore