Skip to content

build(version): fail the release when the tag and package.json disagree - #120

Merged
mrsibe merged 4 commits into
mainfrom
build/version-consistency
Sep 25, 2026
Merged

mrsibe merged 4 commits into
mainfrom
build/version-consistency

Conversation

@mrsibe

@mrsibe mrsibe commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

Closes #61.

package.json's version is what electron-builder stamps into every artifact
name and into the updater feed, and what app.getVersion() reports — so it is
what the About panel shows. The git tag has to agree, and nothing in CI noticed
when it didn't: package.json sat at 1.2.2 while the newest release was v1.3.x,
and only a human reading both numbers could tell.

What the current state actually is

Worth stating up front, because the issue's Context describes a drift that no
longer exists and an About panel that no longer hardcodes anything:

State
package.json 1.3.0
newest tag v1.3.0
latest release v1.3.0 (Latest)
AboutSettings already reads window.api.getAppVersion() → app.getVersion(); no hardcoded string anywhere
scripts/check-version.mjs did not exist
verify.yml / release.yml neither checked anything

So the genuine gap was the guard plus its wiring, not a wrong number. Two
acceptance items are therefore already satisfied and one needed a decision — see
the table at the bottom.

The two modes, and why they differ

This is the one design decision worth reviewing, because the acceptance criterion
("verify.yml fails when the tag and package.json disagree") cannot be
implemented literally: on a pull request there is no tag, so that condition
only exists at release time. Splitting it is not a workaround, it is the correct
reading:

  • tag mode (release.yml, GITHUB_REF=refs/tags/…) — the tag must equal
    package.json exactly. This is the release blocker.
  • compare mode (verify.yml, a PR) — package.json must not be behind
    the newest tag.

Compare mode deliberately does not require equality. npm version is merged
before the tag is pushed, so during that window main is legitimately ahead of
the newest tag; requiring equality would fail every unrelated PR opened in
between, which would make the gate something people learn to ignore.

Failure modes it now catches, with their messages

$ node scripts/check-version.mjs --tag=v1.4.0        # package.json is 1.3.0
Tag v1.4.0 does not match package.json ("version": "1.3.0").
  electron-builder takes every artifact name and the updater feed from package.json,
  so releasing this tag would publish a build whose version does not match its tag.
  Pick one:
    - you are releasing 1.4.0:  run `npm version 1.4.0` on main, merge it, then re-tag
    - you are releasing 1.3.0:  delete this tag and tag v1.3.0 instead
$ node scripts/check-version.mjs --tag=1.3.0         # right version, no v prefix
Tag 1.3.0 has the right version but the wrong name: it must be v1.3.0.
  release.yml triggers on `tags: v*.*.*`, so `1.3.0` would push nothing and
  publish no artifacts. Delete it and tag v1.3.0 instead.
$ node scripts/check-version.mjs                     # package.json behind the newest tag
package.json is at 1.3.0 but the newest release is v1.3.0. … Run `npm version 1.3.0`
$ node scripts/check-version.mjs                     # pass
check:version — package.json 1.3.0 matches the newest tag v1.3.0

The missing-v case gets its own message rather than a bare inequality, because
it is fatal rather than cosmetic: release.yml triggers on tags: v*.*.*, so
such a tag pushes nothing at all. A v-less version tag has never been pushed here,
but the release titles of v1.1.0 and v1.0.7 are inconsistent, which is how
close that class of mistake has come.

Wiring

  • release.yml gains a guard job, and create-draft gains needs: guard. A
    mismatched tag therefore fails before the draft is created — no platform
    build, and no orphan draft release left behind for a tag that was wrong. That
    matters here: the file already carries a long comment about the v1.2.2 incident
    where two half-populated drafts were created for one tag.
  • verify.yml runs it before npm ci (it reads only package.json and the
    tags, so a drifted version fails in seconds) and now uses fetch-depth: 0.
    See the section below for why fetch-tags: true, which was the obvious
    spelling, does not work.
  • package.json gains check:version; CONTRIBUTING.md documents the flow.

Caught by CI on the first run: the guard was silently doing nothing

Status: fixed and re-verified. CI found three real problems across two runs —
the silent pass below, a Windows-only test failure, and then an environment
leak in my own regression test. All three are fixed; see the commits.

Worth its own section, because it is the most useful thing this PR found and the
first version of it shipped the bug it was written to prevent.

verify.yml was written with fetch-tags: true, which does not do what it sounds
like. The CI log from the first run:

check:version — package.json 1.3.0; no version tag reachable, nothing to compare
(not a git clone, or a shallow checkout without tags)

…exit code 0. The step went green while comparing nothing, which is the one
failure mode a guard must not have — a gate that reports "no violations" without
running is worse than no gate, because everything downstream trusts it.

actions/checkout only fetches every tag when it fetches all history. In the
shallow path it builds the refspec with fetchTags, and that adds just the
checked-out ref's own tag — a pull request ref (refs/pull/N/merge) has no tag
at all, so nothing was fetched. fetch-depth: 0 is the fix, and at 184 commits
the full fetch is not worth optimising.

Two changes came out of it:

  1. fetch-depth: 0 in verify.yml, with a comment recording why fetch-tags is
    not the answer, so nobody "optimises" it back.
  2. The silent pass is closed at the source: outside CI a missing tag list is still
    tolerated (running the script in a plain directory should not fail), but under
    GITHUB_ACTIONS it is now a failure that names the fix. That is the same
    call as allowMissingDependencies: false in electron-builder.yml — a check
    that cannot run must not look like one that ran. A regression test runs the
    script in a directory with no git repository to pin it.

The same run also caught a Windows-only test failure of mine:
spawnSync('npm', ...) returns status: null there, because npm is an
npm.cmd shim that spawn cannot execute without shell: true. The entry point
is now asserted by reading package.json instead, which is platform-independent,
and CI runs the real npm run check:version step anyway. (The repo's own
designGuard.test.ts invokes node directly rather than npm — that turns out to
be the reason.)

Review fix: pre-release precedence was not SemVer

Found in review, and a real correctness bug rather than a style nit.

compareVersions compared pre-release strings with <, so it reported

1.4.0-beta.10 < 1.4.0-beta.2      // wrong

because 'beta.10' < 'beta.2' is true lexicographically. That feeds straight into
newestVersionTag, so the guard could select the wrong tag as the newest release
and then validate package.json against it. This project has shipped
v1.0.5-beta.1, so the shape is real, and the original tests only covered
alpha < beta and pre-release < final — which is exactly why they missed it.

comparePrerelease now implements SemVer 11.4 (identifiers left to right, numeric
compared numerically, numeric ranks below alphanumeric, shorter set loses on a tie)
with no new dependency. The six cases that motivated it are pinned as tests, and
the end-to-end behaviour is verified in a throwaway git repository:

package.json is at 1.3.0 but the newest version tag is v1.4.0-beta.10

v1.4.0-beta.2 is what the old comparison would have selected.

Bare version tags were tightened in the same commit, since they produced a
confusing state. newestVersionTag used to accept 1.2.0 as a release tag even
though CONTRIBUTING requires v<version> and release.yml only listens for
v*.*.*. So a pushed 2.0.0 published nothing, and verify.yml still counted
it as the newest release — which made every pull request fail with
"package.json is behind 2.0.0" over a version that never shipped. Only
v-prefixed tags count as release tags now, and a bare version tag is reported
rather than silently ignored:

Unusable version tag: 2.0.0
  release.yml triggers on `tags: v*.*.*`, so this tag would publish no artifacts.
  It is not counted as a release here, or every pull request would fail against a
  version that was never published.
  Delete it, or rename to v<version>.

This is slightly stricter than the minimum asked for (excluding bare tags would
have been enough): the tag is failed on rather than ignored, because a tag that
silently produces no release is the same class of drift this guard exists to
surface. Easy to soften if you would rather it only exclude.

Finally, "the newest release" is now "the newest version tag" — the check reads
git tag --list, not GitHub Releases.

Acceptance

Criterion Status
The check exits non-zero with a clear message on a mismatch Covered — test/versionGuard.test.ts asserts the real exit code via spawnSync, and the messages are shown above
verify.yml fails on disagreement; release.yml refuses to build Covered, split by necessity — release.yml refuses on any tag/package.json disagreement; verify.yml fails when package.json is behind the newest tag (no tag exists on a PR)
CONTRIBUTING.md documents the versioning flow Covered
The About panel shows the version that actually shipped Already true, no code change — AboutSettings reads app.getVersion() over IPC. Verified by reading the call chain, not by a test

On the last row: I deliberately did not add a smoke-test assertion for it. The
smoke test could assert that the packaged app.getVersion() equals the repo's
package.json, but both come from the same file via electron-builder, so it would
test the packager rather than the acceptance criterion — it would not verify that
the About panel displays it. Adding it would have made the PR look more verified
without making it more verified. Flagging that as a judgement call.

Scope notes on things I deliberately did not do

  • Not pinning GitHub Actions to commit SHAs. Every uses: in this repo uses a
    mutable tag, and my new job follows that. Tightening it is a repo-wide security
    change touching all three workflows and does not belong in a versioning PR, but
    it is a real finding if you want it as its own issue.
  • Not changing docs/eval. Unrelated; covered separately.
  • CONTRIBUTING.md's command table got re-aligned by prettier because the new row
    is wider. Unavoidable in a prettier-managed aligned table, and the file was
    prettier-clean before, so no unrelated content moved.

Verification

  • npm run check:version in every mode, including a real throwaway git repository with a bare 2.0.0 tag, including the behind-the-newest-tag
    path induced with a temporary local tag (removed afterwards) and a no-git
    directory for the "nothing to compare" branch
  • npm test — 254 pass / 0 fail (17 new), and again with GITHUB_ACTIONS=true set, which is the condition that broke it earlier
  • npm run typecheck, npm run build, npm run check:design
  • npm run lint — 0 errors, the pre-existing warning baseline unchanged
  • npx prettier --check on every touched file
  • Both workflow files parse, and the job graph is guard → create-draft → release → publish

Not verified

release.yml's guard job only runs on a tag push, so it is not exercised
here: verify.yml is proven on this PR, including that the version step really
compares rather than skipping, but the needs: guard wiring and the tag-mode
branch are only proven the first time a tag is pushed. The tag-mode logic itself
is covered by unit tests, and release.yml runs the same script as the PR path.

Confirmed on the final run (c2c21b8, all three platforms green):

check:version — package.json 1.3.0 matches the newest tag v1.3.0

which is the line that read "no version tag reachable, nothing to compare" before
fetch-depth: 0, so the fix is verified rather than assumed.

`package.json`'s version is what electron-builder stamps into every artifact name
and into the updater feed, and what `app.getVersion()` reports - so it is what the
About panel shows. The git tag has to agree with it, and nothing in CI noticed
when it did not: `package.json` sat at 1.2.2 while the newest release was v1.3.x,
and only a human reading both numbers could tell.

- `scripts/check-version.mjs` is the guard, with two modes because the two
  situations have genuinely different invariants. A tag build (`release.yml`)
  requires the tag to equal `package.json` exactly - that is the release blocker.
  A pull request (`verify.yml`) requires `package.json` only to not be *behind*
  the newest tag: equality cannot be required there, because a version bump is
  merged before the tag is pushed and `main` is legitimately ahead during that
  window. Requiring it would fail every unrelated PR opened in between.
- It runs in a new `guard` job that `create-draft` needs, so a mismatched tag
  fails before any of it: no platform build, and no orphan draft release left
  behind for a tag that was wrong.
- `verify.yml` runs it before `npm ci` (it reads only `package.json` and the
  tags, so a drifted version fails in seconds) and now fetches tags: the default
  depth-1 checkout fetches none, so the comparison would otherwise have had
  nothing to compare and silently passed.
- A tag carrying the right version but without the `v` prefix gets its own
  message, because that is fatal rather than cosmetic: `release.yml` triggers on
  `tags: v*.*.*`, so such a tag would push nothing and publish no artifacts.
- `CONTRIBUTING.md` documents the flow, including why the two modes differ. It
  recommends `npm version` (one command bumps, commits and tags, so the two
  cannot drift) while noting that a hand cut release, which is how earlier
  releases here were actually tagged, is fine as long as the invariant holds.
- The About panel already read the real version (`app.getVersion()` over IPC,
  with no hardcoded string), so that acceptance item needed no code change.

Verification: `npm run check:version` in every mode (matching tag, mismatched
tag, unprefixed tag, and `package.json` behind the newest tag, the last induced
with a temporary local tag); `test/versionGuard.test.ts` (11 new tests: version
parsing, numeric vs lexicographic ordering, pre-release ordering, tag selection,
and the script's real exit codes through `spawnSync`); `npm run typecheck`;
`npm test` (248 pass); `npm run build`; `npm run check:design`; `npm run lint`
(0 errors, the pre-existing warning baseline unchanged);
`npx prettier --check` on every touched file.

Not verified: the workflow changes cannot be executed locally. `verify.yml` runs
on this PR, but `release.yml`'s guard job only runs on a tag push, so that path
stays unexercised until the next release. It runs the same script as the PR path,
and the tag-mode branch is covered by the unit tests.

Closes #61
@github-actions github-actions Bot added the build Build, packaging and CI label Sep 25, 2026
`verify.yml` used `fetch-tags: true`, which does not do what it sounds like: the
step reported "no version tag reachable, nothing to compare" and exited 0. The
guard went green while comparing nothing, which is the one failure mode a guard
must not have - and it is exactly the trap this PR was written to close.

actions/checkout only fetches every tag when it fetches all history. In the
shallow path it builds the refspec with `fetchTags`, which adds just the
checked-out ref's own tag, and a pull request ref (`refs/pull/N/merge`) has no
tag. `fetch-depth: 0` fetches all branches and tags; the repository is 184 commits
and 15 tags, so the full fetch is not worth optimising away.

The silent pass is closed too, so it cannot regress unnoticed: outside CI a
missing tag list is still tolerated, but under GITHUB_ACTIONS it now fails and
names the fix. Same call as `allowMissingDependencies: false` in
electron-builder.yml - a check that cannot run must not look like one that ran.
Covered by a regression test that runs the script in a directory with no git
repository at all.

Also fixes a Windows-only test failure found by CI: `spawnSync('npm', ...)`
returns `status: null` there, because npm is an `npm.cmd` shim that spawn cannot
execute without `shell: true`. The entry point is now asserted by reading
`package.json`, which is platform-independent, and CI runs the real
`npm run check:version` step regardless.
The regression test for the silent-pass hole asserted 'outside CI a missing tag
list is tolerated' while passing `cwd` but not `env`. CI sets GITHUB_ACTIONS, so
the child inherited it, took the CI branch and exited 1 - the test failed on all
three platforms by asserting the opposite branch from the one it ran.

The local case now runs with GITHUB_ACTIONS removed from the environment, and the
whole file passes both normally and with GITHUB_ACTIONS=true set, which is the
condition that broke it.
`compareVersions` compared prereleases with `<` on the raw strings, which is not
SemVer. `'beta.10' < 'beta.2'` is true lexicographically, so the guard reported

    1.4.0-beta.10 < 1.4.0-beta.2

when the opposite is correct. That flows straight into `newestVersionTag`, so the
guard could select the wrong tag as the newest release and then validate
`package.json` against it. This project has shipped `v1.0.5-beta.1`, so the shape
is real rather than theoretical, and the old tests only covered `alpha < beta` and
pre-release < final, which is why they did not catch it.

`comparePrerelease` now implements SemVer 11.4: identifiers are compared left to
right, numeric identifiers numerically, numeric identifiers always rank below
alphanumeric ones, and a shorter set of identifiers loses once every shared one is
equal. No dependency is added for it.

Also tightens bare version tags, which were accepted as releases while
CONTRIBUTING requires `v<version>` and release.yml only listens for `v*.*.*`. A
pushed `2.0.0` therefore published nothing, and `verify.yml` nevertheless counted
it as the newest release - so every pull request failed with "package.json is
behind 2.0.0" over a version that never shipped. Only `v`-prefixed tags count as
release tags now, and a bare version tag is reported instead of ignored, since a
tag that silently produces no release is exactly the drift this guard exists to
surface. Tag mode also says so when a bare tag is both misnamed and mismatched.

The "newest release" wording becomes "newest version tag": the check reads
`git tag --list`, not GitHub Releases.

Tests: the six precedence cases that motivated this (`beta.2 < beta.10`,
`alpha < beta`, `1 < alpha`, `beta < beta.1`, `beta.1 < beta.2`,
`beta.2 < beta.10`) plus the deeper comparisons, `newestVersionTag` picking
`v1.4.0-beta.10` over `v1.4.0-beta.2`, `bareVersionTags`, and an end-to-end case
that builds a real throwaway git repository with a bare `2.0.0` tag and asserts it
is reported rather than compared against.
@mrsibe
mrsibe merged commit 8e8716b into main Sep 25, 2026
4 checks passed
@mrsibe
mrsibe deleted the build/version-consistency branch September 25, 2026 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build Build, packaging and CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Build] Enforce version consistency: package.json ↔ git tag ↔ release ↔ app version

1 participant