test(ci): prove the cooldown rejects a freshly published release - #598
test(ci): prove the cooldown rejects a freshly published release#598guyofeck wants to merge 2 commits into
Conversation
Implements the interim policy Dima Ryskin set out for OSS repos, so GitHub
Actions can be re-enabled for base44/cli at the org level:
1. enforce lock-files on all node builds, so the build never overrides one
2. enforce a package manager that supports a minimal-age directive
3. use embargo in non-publish workflow tasks
Bun satisfies point 2 natively, so no package manager migration is needed:
bun install --minimum-release-age=<val>
Only install packages published at least N seconds ago (security feature)
bunfig.toml sets `minimumReleaseAge = 604800` (7 days; Bun takes SECONDS, unlike
npm's min-release-age, which takes days). Point 1 was already satisfied by
`bun install --frozen-lockfile`, which is what every workflow runs.
That choice matters operationally, not just aesthetically: Wix-managed machines
block registry.npmjs.org at the network-extension layer, so no developer here can
regenerate an npm lockfile without routing through the embargo gateway or an
internal mirror. A Bun-native cooldown keeps bun.lock as the single source of
truth and needs no lockfile regeneration at all.
Point 3 — the publish workflows no longer run the embargo gateway:
- Drops the `sudo sed -i /etc/hosts` unpin hack from both. There is no pin to
strip now, so the window where the gateway was bypassed is gone entirely.
- Safe because those jobs resolve nothing: `--frozen-lockfile` installs bun.lock
verbatim, so dropping the gateway does not widen what they can pull.
- check_wix_proxy_steps.py enforces the split bidirectionally: non-publish jobs
must run the proxy, publish jobs must not. Exemptions are keyed on exact
filename and printed on success, so the list cannot quietly grow.
Also closes the remaining unprotected registry fetches in the publish path:
- Authenticate via npm trusted publishing (OIDC). Drops NODE_AUTH_TOKEN /
secrets.NPM_TOKEN from preview-publish; manual-publish never had a credential
wired at all, so it could not have published regardless.
- Replace `bunx json-bump` with `npm version` and `npm pkg set`. json-bump is
declared in no manifest, so bunx fetched it at run time — outside bun.lock and
outside the cooldown, since bunx accepts --minimum-release-age without
enforcing it (oven-sh/bun#30748).
- Remove `npm install -g npm@latest` from both publish workflows.
- Least-privilege per-job permissions replace the workflow-level blocks; drops
manual-publish's unused packages: write and pull-requests: read.
Known Bun gaps, documented in bunfig.toml and docs/AGENTS.md rather than left to
be discovered: bunx does not enforce the flag (#30748), `bun update --latest`
skips transitive deps (#25305), and there is no bypass for vulnerability fixes
(#26065) short of minimumReleaseAgeExcludes.
Verified: check_wix_proxy_steps.py reports 12 of 12 non-publish jobs across 11
workflows with both publish jobs confirmed to abstain; 12 of 12 unit tests pass
(4 new, covering both directions of the rule).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Dima Ryskin asked for a test rather than a code review of #597: "Can we stack another PR on top of that and check if installing a fresh-package is rejected? I used npmjs.com/package/electron-nightly for always 'fresh' versions". Adds .github/workflows/cooldown-check.yml plus check_cooldown.sh, which probes the real bunfig.toml setting in a temp project. Three cases, because an exit code alone does not distinguish a working guardrail from a broken probe: control `bun add lodash` must succeed, so a red result means the cooldown fired rather than the environment being broken. floating `bun add electron-nightly` must not land a version inside the cooldown — either refused, or resolved to an older one. Asserted on the resolved version's publish time from the registry, so silently installing a one-day-old version cannot pass. exact pin `bun add electron-nightly@<newest>` must be refused. This is the bypass that matters: a PR pinning an exact fresh version. The cooldown is read out of bunfig.toml rather than hardcoded, so the test cannot drift from the setting it verifies. Two constraints shaped the workflow: - It must NOT run the embargo gateway. Gatewayed, embargo would reject the fresh release itself and the run would prove nothing about Bun. check_wix_proxy_steps gains a third exemption, and now records why each one exists: the frozenset becomes a dict of path -> reason, printed per exemption on success, so an exemption cannot be added without stating its justification. - No third-party actions. The org now sets github_owned_allowed with an empty patterns_allowed and requires SHA pinning, so oven-sh/setup-bun is not usable here; Bun is installed from a run step and actions/checkout is pinned to 3d3c42e5 (v7.0.1). Verified: bash -n clean; bunfig parsing smoke-tested (604800 -> 7 days); check_wix_proxy_steps reports 12 of 12 gatewayed jobs with all 3 exempt jobs confirmed to abstain; 13 of 13 unit tests pass (1 new, asserting each exemption prints its reason). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
@dimar this is the test you asked for, and it passed on this PR. Run: https://github.com/base44/cli/actions/runs/31491517277 Reading the numbers: Bun skipped the Aug 10 nightly (52,760s ≈ 15 hours old) and resolved back to the Aug 3 one (679,956s ≈ 7.9 days). When pinned explicitly to the fresh version it refused outright — so both the floating resolve and the exact-pin bypass are closed. The control case matters for trust in the result: The run was ungatewayed on purpose — with embargo in front, the gateway would have rejected the fresh version itself and this would have proven nothing about Bun. Separate problem this run surfaced. Every other workflow on this PR came back
The repo's policy is now This workflow only ran because it was written to need neither: Bun installs from a Which would you prefer:
Either way we'll do the SHA-pinning pass across all workflows; SHA-pinning alone won't help while |
Stacked on #597 — base that PR, not
main. This is the test @dimar asked for in place of a code review:Adds
cooldown-check.yml+check_cooldown.sh, which probes the realbunfig.tomlsetting in a temp project. Three cases, because an exit code alone can't tell a working guardrail from a broken probe:bun add lodashbun add electron-nightlybun add electron-nightly@<newest>The cooldown is read out of
bunfig.tomlrather than hardcoded, so the test can't drift from the setting it verifies.No gateway in this workflow, deliberately. Gatewayed, embargo would reject the fresh release itself and the run would prove nothing about Bun — which is the guardrail #597 relies on.
check_wix_proxy_steps.pygains a third exemption and now records why each exists: the frozenset becomes apath -> reasondict, printed per exemption on success, so no exemption can be added without stating its justification.No third-party actions, per the policy that landed with Actions being re-enabled:
github_owned_allowedwith emptypatterns_allowed, andsha_pinning_required: true. Sooven-sh/setup-bunisn't usable — Bun installs from arunstep, andactions/checkoutis pinned to3d3c42e5(v7.0.1).Status
bash -nclean; bunfig parsing smoke-tested (604800→ 7 days)check_wix_proxy_steps.py: 12/12 gatewayed jobs, all 3 exempt jobs verified to abstain; 13/13 unit testspull_requestpath filterHeads-up for @dimar
The rest of the repo can't run under the new policy yet:
lint,test,typecheck,knipand both publish workflows all depend onoven-sh/setup-bun, which the emptypatterns_allowedblocks, and 15 of 16uses:refs are still tag-pinned. This workflow was written to need neither, so it can run today — but the others need either allowlist entries or the samecurl-install treatment. Which do you prefer?🤖 Generated with Claude Code