Repository navigation
chore(format): stop prettier rewriting the generated eval baseline - #123
Merged
Merged
Conversation
The committed `docs/eval/baseline-v1.4.json` is the eval harness output verbatim - `src/main/eval/run.ts` writes it with `JSON.stringify(report, null, 2)` - and that is deliberately *not* prettier's formatting: prettier collapses the short `matchesByRank` arrays onto one line, which turns a 12 KB file into an 8 KB one without changing a value. So prettier was a second writer for a generated file. `npm run format` (and any editor running prettier on save in that directory) reformatted both files and left a dirty worktree behind, which is where the recurring uncommitted `docs/eval` noise came from: a diff of ~450 lines that said nothing, and which had to be manually excluded from every unrelated PR. Both files are now ignored, since regenerating them is a documented command (`npm run eval`, which rewrites `docs/eval/`) and the baseline's value is that it is byte-comparable to what the harness produces. `docs/eval/baseline-*` is ignored rather than the directory, so a hand-written document that later lands in `docs/eval/` is still formatted. Verified: `npx prettier --check .` reports no issues repo-wide, and with the ignore bypassed (`--ignore-path /dev/null`) prettier still flags `baseline-v1.4.json` - so the churn stops because the generated file is left alone, not because it was reformatted once.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the recurring uncommitted
docs/evalnoise.The committed
docs/eval/baseline-v1.4.jsonis the eval harness output verbatim —src/main/eval/run.tswrites it withJSON.stringify(report, null, 2)— which is deliberately not prettier's formatting. Prettier collapses the shortmatchesByRankarrays onto one line, turning a 12 KB file into an 8 KB one without changing a value.So prettier was a second writer for a generated file.
npm run formatreformatted both files and left a dirty worktree behind: a ~450-line diff that said nothing, which then had to be manually excluded from every unrelated PR (it showed up in this session's work on #118, #120 and #122).Both files are now ignored.
docs/eval/baseline-*rather than the directory, so a hand-written document landing indocs/eval/later is still formatted.Why ignoring is the right fix rather than reformatting
Regenerating these is a documented command (
npm run eval, which "rewritesdocs/eval/" per CONTRIBUTING), and the baseline's entire value is that it is byte-comparable to what the harness produces —verify.ymldiffs a fresh run against the committed file to prove the committed numbers still reproduce. Reformatting it once would have hidden the churn for exactly one commit and left the nextnpm run evaldirty again, because the harness would keep writing its own format.Verification
npx prettier --check .— no issues repo-wide (it previously flagged both baseline files)npx prettier --check --ignore-path /dev/null docs/eval/baseline-v1.4.json— still flagged, which is the point: the file is untouched, and the churn stops because it is left alone rather than reformatted oncenpm test254 pass ·npm run typecheck·npm run check:design·npm run lint0 errors (107 warning baseline unchanged)verify.ymldeterminism diff is unaffectedOne file, 8 added lines.