Skip to content

chore(format): stop prettier rewriting the generated eval baseline - #123

Merged
mrsibe merged 1 commit into
mainfrom
chore/prettierignore-eval-baseline
Sep 25, 2026
Merged

mrsibe merged 1 commit into
mainfrom
chore/prettierignore-eval-baseline

Conversation

@mrsibe

@mrsibe mrsibe commented Sep 25, 2026

Copy link
Copy Markdown
Owner

Fixes the recurring uncommitted docs/eval noise.

The committed docs/eval/baseline-v1.4.json is the eval harness output verbatim — src/main/eval/run.ts writes it with JSON.stringify(report, null, 2) — which is deliberately not prettier's formatting. Prettier collapses the short matchesByRank arrays onto one line, turning a 12 KB file into an 8 KB one without changing a value.

So prettier was a second writer for a generated file. npm run format reformatted both files and left a dirty worktree behind: a ~450-line diff that said nothing, which then had to be manually excluded from every unrelated PR (it showed up in this session's work on #118, #120 and #122).

Both files are now ignored. docs/eval/baseline-* rather than the directory, so a hand-written document landing in docs/eval/ later is still formatted.

Why ignoring is the right fix rather than reformatting

Regenerating these is a documented command (npm run eval, which "rewrites docs/eval/" per CONTRIBUTING), and the baseline's entire value is that it is byte-comparable to what the harness produces — verify.yml diffs a fresh run against the committed file to prove the committed numbers still reproduce. Reformatting it once would have hidden the churn for exactly one commit and left the next npm run eval dirty again, because the harness would keep writing its own format.

Verification

  • npx prettier --check . — no issues repo-wide (it previously flagged both baseline files)
  • npx prettier --check --ignore-path /dev/null docs/eval/baseline-v1.4.json — still flagged, which is the point: the file is untouched, and the churn stops because it is left alone rather than reformatted once
  • npm test 254 pass · npm run typecheck · npm run check:design · npm run lint 0 errors (107 warning baseline unchanged)
  • The committed baseline bytes are unchanged in this PR, so the verify.yml determinism diff is unaffected

One file, 8 added lines.

The committed `docs/eval/baseline-v1.4.json` is the eval harness output verbatim -
`src/main/eval/run.ts` writes it with `JSON.stringify(report, null, 2)` - and that
is deliberately *not* prettier's formatting: prettier collapses the short
`matchesByRank` arrays onto one line, which turns a 12 KB file into an 8 KB one
without changing a value.

So prettier was a second writer for a generated file. `npm run format` (and any
editor running prettier on save in that directory) reformatted both files and left
a dirty worktree behind, which is where the recurring uncommitted `docs/eval` noise
came from: a diff of ~450 lines that said nothing, and which had to be manually
excluded from every unrelated PR.

Both files are now ignored, since regenerating them is a documented command
(`npm run eval`, which rewrites `docs/eval/`) and the baseline's value is that it is
byte-comparable to what the harness produces. `docs/eval/baseline-*` is ignored
rather than the directory, so a hand-written document that later lands in `docs/eval/`
is still formatted.

Verified: `npx prettier --check .` reports no issues repo-wide, and with the ignore
bypassed (`--ignore-path /dev/null`) prettier still flags `baseline-v1.4.json` -
so the churn stops because the generated file is left alone, not because it was
reformatted once.
@github-actions github-actions Bot added the skip-changelog Exclude from generated release notes label Sep 25, 2026
@mrsibe
mrsibe merged commit 94dd0ff into main Sep 25, 2026
4 checks passed
@mrsibe
mrsibe deleted the chore/prettierignore-eval-baseline branch September 25, 2026 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog Exclude from generated release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant