ci: track coverage and block coverage regressions on PRs

[BabyKoan]

What

Track test coverage in CI and block any PR that decreases it.

Why

Coverage was measured ad-hoc locally (npx vitest run --coverage) but never enforced. Without a guard, untested code can land and silently erode coverage over time. This adds a CI gate so a PR that drops coverage cannot merge.

How

A dedicated coverage job runs on every PR (.github/workflows/ci.yml):

1. Checks out the PR base branch and the PR head into separate dirs.
2. Runs V8 coverage on both with the same src/**/*.ts scope (apples-to-apples).
3. compare-coverage.mjs computes weighted statements and branches coverage for each side and fails if head < base.

Enforcement is by direct base-vs-head comparison, not a static threshold — there is no number to chase, coverage may only go up. Functions are intentionally excluded because the v8 provider reports unreliable per-function hit counts; statements and branches are stable.

Supporting changes:

• vitest.config.ts — v8 coverage over the whole src tree, text + json reporters (json feeds the comparator).
• package.json — npm run coverage script.
• .github/scripts/compare-coverage.mjs — aggregates coverage-final.json, emits ::error:: annotations on regression.
• .gitignore — ignore coverage/.
• CONTRIBUTING.md — document the script and the CI guard.

Testing

• npm run typecheck && npm test && npm run lint — all green (682 tests).
• npm run coverage — exits 0; statements 79.13%, branches 68.59%.
• compare-coverage.mjs against identical reports → exit 0 ("No coverage regression").
• compare-coverage.mjs against a report with 300 statements zeroed → exit 1 with ::error::statements decreased 4.16%, as designed.
• CI YAML validated.

The existing check job (typecheck/test/build on Node 20 + 22) is unchanged.

---

Quality Report

Changes: 6 files changed, 138 insertions(+), 1 deletion(-)

Code scan: 4 issue(s) found

• .github/scripts/compare-coverage.mjs:60 — console.log statement
• .github/scripts/compare-coverage.mjs:61 — console.log statement
• .github/scripts/compare-coverage.mjs:62 — console.log statement
• .github/scripts/compare-coverage.mjs:81 — console.log statement

Tests: passed (0 test)

Branch hygiene: clean

Generated by Kōan

[BabyKoan]

Previous review — superseded by a newer review below.

[BabyKoan]

Rebase with requested adjustments

Branch baby.koan/coverage-ci-guard was rebased onto main and review feedback was applied.

Changes applied

• Changes made:
• Verified @vitest/coverage-v8 already present in package.json and package-lock.json (fixed in prior commit cf72974).
• Added trailing newline to .github/scripts/compare-coverage.mjs.
• Added trailing newline to .github/workflows/ci.yml.
• vitest.config.ts already had trailing newline; no change needed.

Stats

7 files changed, 331 insertions(+), 32 deletions(-)

Actions performed

• Already-solved check: negative (confidence=high, reasoning=No existing commit or merged PR on main implements CI coverage tracking or a coverage regression gat)
• Rebased baby.koan/coverage-ci-guard onto upstream/main
• Applied review feedback
• Pre-push CI check: previous run passed
• Force-pushed baby.koan/coverage-ci-guard to origin
• Private review gate skipped: disabled by config
• CI check enqueued in ## CI (async)

CI status

CI will be checked asynchronously.

---

Automated by Kōan

[DenizOkcu]

Really nice work on this — a coverage gate that can only go up is a genuinely good idea, and the implementation is solid. 👏

Deferring it for now rather than merging, for a couple of reasons specific to this moment rather than the quality of the PR:

• It changes the merge requirements for every future PR, so it is more of a project/process decision than a code fix. We are keeping the bar simple while the harness settles.
• Landing it now would force a rebase + re-run of the other open branches against the new gate.

This is a "not yet," not a "no." Will revisit when we are ready to adopt a coverage policy. Thanks for the thorough implementation and the documentation in CONTRIBUTING!

[BabyKoan]

Rebase with requested adjustments

Branch baby.koan/coverage-ci-guard was rebased onto main and review feedback was applied.

Changes applied

• Added trailing newline to vitest.config.ts.
• Other review feedback already resolved in branch:
• @vitest/coverage-v8 present in package.json devDependencies.
• Trailing newlines already present in .github/scripts/compare-coverage.mjs and .github/workflows/ci.yml.

Stats

7 files changed, 330 insertions(+), 31 deletions(-)

Actions performed

• Already-solved check: negative (confidence=high, reasoning=No coverage job, compare script, coverage config, or related documentation exists on main; recent co)
• Rebased baby.koan/coverage-ci-guard onto upstream/main
• Applied review feedback
• Pre-push CI check: previous run passed
• Force-pushed baby.koan/coverage-ci-guard to origin
• Private review gate skipped: disabled by config
• CI check enqueued in ## CI (async)

CI status

CI will be checked asynchronously.

---

Automated by Kōan

[atoomic]

@BabyKoan review

[BabyKoan]

PR Review — ci: track coverage and block coverage regressions on PRs

Clean coverage gate with minor CI hygiene issues; not quite merge-ready.

The PR does what it promises: V8 coverage over src/**/*.ts, a base-vs-head comparator, and a CI job that blocks regressions on statements and branches. The aggregation math is correct, the epsilon absorbs float noise, and the separate base/head checkouts keep the comparison apples-to-apples. Documentation in CONTRIBUTING.md is accurate and concise.

What needs attention:

• The coverage job should enable npm caching to avoid slow, uncached installs on every PR.
• package-lock.json contains unrelated libc removal noise that should be cleaned up.
• The comparator lacks tests and could guard malformed branch data more defensively.
• The branch is behind the current main tip and will need a rebase before merge.

---

🟡 Important

1. Coverage job missing npm cache
.github/workflows/ci.yml:29-31

The check job uses cache: npm, but the coverage job does not. Every coverage run will therefore re-download all dependencies from the registry instead of using the Actions cache.

Impact: slower PR checks, unnecessary network dependence, and extra load on npm. The coverage job already runs a full npm ci on both base and head, so this is the most expensive job in the workflow.

Fix: add cache: npm under the setup-node with: block in the coverage job.

      - uses: actions/setup-node@v4
        with:
          node-version: 22

2. Lockfile contains unrelated format churn

The lockfile diff removes many libc arrays from optional native dependencies (e.g. @rollup/rollup-*, @rspack/binding-*). That format change is not caused by adding @vitest/coverage-v8; it appears to come from a different npm version than the one used to generate the lockfile on main.

Impact: the diff is much larger than the actual dependency addition, which makes it harder to audit and increases the chance of introducing unintended platform-resolution changes.

Fix: regenerate package-lock.json with the same npm version used on main, or use npm install --package-lock-only @vitest/coverage-v8 to keep the change minimal.

🟢 Suggestions

1. Branch coverage shape is not validated
.github/scripts/compare-coverage.mjs:31-35

The loop assumes d.b[k] is always an array. If a malformed coverage-final.json ever has a non-array value there, the script will throw an unhelpful runtime error instead of a clean ::error:: annotation.

Impact: a corrupted coverage artifact produces a stack trace in CI rather than a clear failure message, which is harder to diagnose.

Fix: add a small guard before iterating, for example:

if (!Array.isArray(d.b[k])) {
  throw new Error(`Expected branch data for ${file}.${k} to be an array`);
}

    for (const k of Object.keys(d.b ?? {})) {
      for (const v of d.b[k]) {
        bt++;
        if (v > 0) bc++;
      }
    }

2. No tests for the coverage comparator

The comparator script is small but it is the merge gate. It currently has no automated tests for the identical-report, regression, and invalid-argument paths.

Impact: future edits to thresholds, metrics, or formatting risk silently changing the gate behavior.

Fix: add a small Node test file (e.g. tests/scripts/compare-coverage.test.mjs) that runs the script against fixture coverage-final.json files and asserts the exit codes/output. This can reuse the same fixtures already described in the PR description.

---

Checklist

• No hardcoded secrets
• Input validation at boundaries
• Error handling for coverage parsing
• Dependencies declared
• Production readiness (CI config, lockfile hygiene) — warning ＃1, warning ＃2

---

To rebase specific severity levels, mention me: @BabyKoan rebase critical (fixes 🔴 only), @BabyKoan rebase important (fixes 🔴 + 🟡), or just @BabyKoan rebase for all.

---

Automated review by Kōan (Claude) HEAD=f7881d0 15 min 58s

[BabyKoan]

Rebase with requested adjustments

Branch baby.koan/coverage-ci-guard was rebased onto main and review feedback was applied.

Changes applied

• Added cache: npm to the coverage job's actions/setup-node step per reviewer request.
• Regenerated package-lock.json from main and re-added only @vitest/coverage-v8; removed the unrelated libc array churn per reviewer request.
• Hardened .github/scripts/compare-coverage.mjs: it now validates branch coverage data is an array and exits 2 with a clear error on malformed input.
• Added tests/scripts/compare-coverage.test.ts with fixtures in tests/fixtures/coverage-final/ covering identical reports, statement regression, branch regression, missing arguments, missing files, and malformed branch data.
• Verified npm ci --dry-run succeeds and the comparator returns expected exit codes for all fixture cases.

Stats

12 files changed, 468 insertions(+), 1 deletion(-)

Actions performed

• Already-solved check: negative (confidence=high, reasoning=Main branch lacks coverage job in CI, coverage script, vitest coverage config, @vitest/coverage-v8 d)
• Rebased baby.koan/coverage-ci-guard onto upstream/main
• Applied review feedback
• Pre-push CI check: previous run passed
• Force-pushed baby.koan/coverage-ci-guard to origin
• Private review gate skipped: disabled by config
• CI check enqueued in ## CI (async)

CI status

CI will be checked asynchronously.

---

Automated by Kōan