Skip to content

feat(coverage): render coverage summaries as a Markdown table - #14

Open
dchaudhari7177 wants to merge 1 commit into
esherialabs:mainfrom
dchaudhari7177:feat/5-coverage-summary-markdown
Open

dchaudhari7177 wants to merge 1 commit into
esherialabs:mainfrom
dchaudhari7177:feat/5-coverage-summary-markdown

Conversation

@dchaudhari7177

Copy link
Copy Markdown

Closes #5.

npm run coverage:summary turns the two Vitest coverage-summary.json files into a deterministic Markdown table.

# Coverage summary

## All source

Source: `coverage/all-source/coverage-summary.json`

| Metric | Covered | Total | Percent |
| --- | ---: | ---: | ---: |
| Statements | 120 | 400 | 30.00% |
| Branches | 40 | 200 | 20.00% |
| Functions | 15 | 60 | 25.00% |
| Lines | 118 | 395 | 29.87% |

## Critical safety and privacy
...

Acceptance criteria

  • Reads both the all-source and critical coverage summary files
  • Emits statements, branches, functions and lines with covered/total counts and percentages
  • Focused unit tests for missing files and malformed input — 15 cases
  • Does not weaken the 15% global or 80% critical gates — see below
  • Documented in docs/open-source/build-and-test.md — new "Human-readable summary" subsection under Coverage gates

It cannot weaken the gates

The generator is read-only and asserts no threshold — it reports numbers the existing gates already produced. config/release/coverage-policy.v1.json and all three vitest configs are untouched.

The only change outside new files is one added npm script:

package.json | 1 +

Structure

Split the way the other scripts here are — pure functions in scripts/lib/saferide-coverage-summary.mjs, a thin CLI in scripts/saferide-coverage-summary.mjs that only owns stdout and the exit code, mirroring saferide-claim-register-check.mjs. buildCoverageReport({ rootDir, sources }) is injectable, so the tests never need a real coverage run.

Deterministic on purpose: fixed metric order and two-decimal percentages, so re-rendering the same input is byte-identical and evidence diffs stay readable. One test asserts exactly that by building twice and comparing.

Failure modes name the file

A stack trace is not useful in CI, so every bad input raises CoverageSummaryError with the offending path:

coverage/critical-safety-privacy/coverage-summary.json: not found -- run npm run coverage:check:public first

exit code 1.

Covered: missing file, malformed JSON, a JSON array, a missing total block, each of the four metrics missing individually, and a non-numeric pct.

Verification

  • node --test scripts/__tests__/saferide-coverage-summary.test.mjs15 pass, 0 fail
  • Ran npm run coverage:summary end-to-end against fixture summaries (output above) and confirmed the missing-file path exits 1 with the message shown.

Add `npm run coverage:summary`, converting the two Vitest coverage-summary
JSON files into a deterministic Markdown table: statements, branches,
functions and lines with covered/total counts and percentages, for both the
all-source and critical safety/privacy suites.

Split the way the other scripts here are: pure functions in
scripts/lib/saferide-coverage-summary.mjs, a thin CLI in
scripts/saferide-coverage-summary.mjs that only handles stdout and the exit
code. buildCoverageReport takes rootDir and sources, so the tests never need a
real coverage run.

Deterministic on purpose -- fixed metric order and two-decimal percentages --
so re-rendering the same input produces a byte-identical file and evidence
diffs stay readable.

Read-only. It asserts no threshold, so the 15% global and 80% critical gates
in coverage-policy.v1.json are untouched; the only change outside new files is
one added npm script.

Bad input fails with the offending path rather than a stack trace: a missing
summary names the file and the command that produces it, and malformed JSON, a
non-object, a missing total block, a missing metric or a non-numeric field all
raise CoverageSummaryError.

15 node --test cases cover those paths plus table formatting, percentage
rounding, byte-identical repeat runs, and both suites being required.

Documented under Coverage gates in docs/open-source/build-and-test.md.

Closes esherialabs#5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a human-readable coverage summary generator

1 participant