Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 92 additions & 2 deletions .devflow/features/release-pipeline/KNOWLEDGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,7 +188,7 @@ Steps (in order):
1. Verify publish credentials (npm `whoami` + cargo token non-empty + PyPI OIDC mint-token exchange).
2. Assert synchronized versions, no `file:` refs.
3. Assert no hazardous codepoints in tracked source.
4. Run `npm run test:gates` — all four spec files, 211 tests including pin-shape specs (S16), per-leg cache key spec (S20), and Alpine load test job spec (S21).
4. Run `npm run test:gates` — all four spec files, 212 tests including pin-shape specs (S16), per-leg cache key spec (S20), Alpine load test job spec (S21), and the cargo-zigbuild musl-leg spec (S22).
5. Assert tagged SHA has green CI history (step-skipped on `pull_request`).

Because `npm run test:gates` runs inside `version-gate`, a malformed pin (e.g. a commit SHA
Expand Down Expand Up @@ -248,6 +248,54 @@ success). `npm publish` calls in `publish-npm` have NO such guard — a partial
unrecoverable by re-run. This makes `publish-crates` the single irreversible point of no
return; all correctness gates run before it.

### build-napi — musl legs with cargo-zigbuild (#339)

Both musl matrix entries (`x86_64-unknown-linux-musl`, `aarch64-unknown-linux-musl`) use
`napi build --platform --release --target <triple> --no-js -x`. The `-x` flag instructs
`@napi-rs/cli` 3.8.6 to run `cargo zigbuild` instead of `cargo build` for non-Windows targets;
it cannot be combined with `--use-napi-cross`.

**cargo-zigbuild 0.23.0** is pinned via `taiki-e/install-action` at SHA
`6c6fd71fe4fb72c3697d269963d0e15df8adedad` (= v2.85.10, composite action; SHA pin is correct
per PF-040) with `tool: cargo-zigbuild@0.23.0`, `fallback: none`, placed BEFORE
`Swatinem/rust-cache`. Step ordering in `build-napi` (musl legs):

| Index | Step | Purpose |
|---|---|---|
| 3 | `Install cargo-zigbuild (pinned)` | SHA-pinned taiki-e/install-action, `fallback: none` |
| 4 | `Swatinem/rust-cache` | `key: matrix.settings.target` (PF-041, S20) |
| 5 | `Install zig` | SHA-pinned mlugg/setup-zig, version 0.16.0 |
| 6 | `Assert cargo-zigbuild is the pinned version` | `cargo help zigbuild` + `cargo-zigbuild --version` + 0.23.0 check |
| 9 | `Verify no lingering musl linker export or wrapper (no-op detector)` | reads MUSL_LINKER env vars in `[ -z ]` guards; asserts the wrapper cache absent |
| 10 | `Build addon` | `npx napi build … -x` |
| 11 | `Assert cargo-zigbuild 0.23.0 wrappers were generated` | finds `zigcc-*` under the wrapper cache and re-asserts `cargo-zigbuild --version` |
| 12 | `Assert musl artifact links musl, not glibc` | readelf + ALLOWED_NEEDED allowlist |

cargo-zigbuild 0.23.0 owns the full linker-arg filter (`--fix-cortex-a53-843419`,
`--no-undefined-version`, `-lgcc_s` to `-lunwind`, self-contained musl CRT skip, response
files) — all of it in `src/zig.rs` (0.23.0 has no `src/zig/` directory; the filter lives
around `run_filter_one`). Its CI tests zig 0.16.0; nothing in 0.23.2–0.23.4 touches
x86_64/aarch64 musl. Bumping zig OR cargo-zigbuild is a deliberate, paired decision.

Wrapper cache layout (verified against 0.23.0's `cache_dir()` in `src/zig.rs`):
`${XDG_CACHE_HOME:-$HOME/.cache}/cargo-zigbuild/0.23.0/wrappers/<exe-hash>/zigcc-<target>-<hash>.sh`
(plus `zigcxx-…`, `zigranlib.sh`, and `ar`/`lib` symlinks). Both the pre-build absence
check and the post-build presence check resolve the root with `${XDG_CACHE_HOME:-$HOME/.cache}`
so they name the directory cargo-zigbuild actually writes; the post-build `find` is
recursive because of the `wrappers/<exe-hash>/` level. Spec S22 pins the XDG-aware form
(a hard-coded `$HOME/.cache` path is rejected, with a planted control).

The readelf gate uses `ALLOWED_NEEDED='libc\.so|libgcc_s\.so\.1'` with a planted
`libunwind.so.1` positive control (spec S22, PF-013, PF-038). Spec S22 asserts the
ALLOWED_NEEDED assignment literal (not a comment) contains both patterns.
The expected DT_NEEDED set for a dynamic musl cdylib is just `libc.so`; `libgcc_s.so.1`
is tolerated because `node:22-alpine` ships it. `libunwind.so.1` is the control precisely
because cargo-zigbuild rewrites `-lgcc_s` to `-lunwind` and zig links its own libunwind
into the artifact — a `libunwind.so.1` DT_NEEDED would mean a shared object Alpine does
not ship, invisible to readelf and fatal at dlopen. The allowlist has not yet been observed
against a real cargo-zigbuild-linked artifact; the first dry run is where it earns its keep,
and the Alpine load tests (#340) are the acceptance instrument for any linkage delta.

### GitHub expression preprocessor trap

GitHub's expression preprocessor scans `run:` block text **including shell comments** without
Expand Down Expand Up @@ -312,9 +360,45 @@ such a file. Never write `${{` in comments; describe it in words.
beside `index.js`) and short-circuits the fixture, making the test vacuous.
- **Testing the musl addon through `@mdscript/mds`**: its WASM fallback makes the test
vacuous — a successful load does not prove the native addon was reached.
- **Installing a pinned cargo binary (e.g. cargo-zigbuild) AFTER `Swatinem/rust-cache`**:
rust-cache deletes every pre-existing `~/.cargo/bin` binary before saving; on a warm cache
hit it restores nothing to `~/.cargo/bin`, so a binary installed after the step is wiped.
Place `taiki-e/install-action` with `fallback: none` BEFORE `Swatinem/rust-cache`.
- **Setting any musl linker export (`CARGO_TARGET_*_MUSL_LINKER`)**: cargo-zigbuild's
`add_env_if_missing` yields to a pre-set `CARGO_TARGET_*_LINKER` value; a leftover export
from an old wrapper step silently reverts the migration with every gate green. The no-op
detector step reads both musl linker vars inside `[ -z "${...:-}" ]` guards before the build.
- **Caching `~/.cache/cargo-zigbuild`**: the wrapper cache is version-keyed
(`~/.cache/cargo-zigbuild/<version>/…`); presence after the build proves cargo-zigbuild
0.23.0 ran in THAT job. Caching it would allow a prior version's wrappers to persist through
an upgrade, defeating the version assertion.

## Gotchas

- **napi's cargo-zigbuild detector is presence-only**: `@napi-rs/cli` checks for zigbuild with
`cargo help zigbuild`; if absent it runs an UNPINNED `cargo install cargo-zigbuild` mid-build,
silently replacing the pinned version. Pre-install with `taiki-e/install-action` and `fallback:
none` (never `fallback: cargo-binstall`), then assert with `cargo-zigbuild --version` AFTER
rust-cache and AGAIN after the build. `cargo help zigbuild` is itself non-vacuous — it exits
101 when the binary is absent and 0 when it is present (verified locally).
- **`cargo zigbuild --version` is NOT a version probe — it exits 2**: cargo-zigbuild's clap enum
(`src/bin/cargo-zigbuild.rs`) puts `version` on the top-level command and never sets
`propagate_version`, and neither `cargo_options::Build` nor `CommonOptions` defines a
`--version` arg, so `cargo zigbuild --version` (and `-V`) fail with `error: unexpected
argument '--version' found`. Under `set -euo pipefail` that aborts the asserting step and
fails BOTH musl legs on every tag push, dispatch and release-surface PR. Always probe the
binary: `cargo-zigbuild --version` prints exactly `cargo-zigbuild 0.23.0`. S22 asserts the
binary form and rejects the substring `cargo zigbuild --version` anywhere in `build-napi`.
- **`-x` cannot be combined with `--use-napi-cross`**: they are mutually exclusive flags in
`@napi-rs/cli`; combining them is a build error.
- **`add_env_if_missing` in cargo-zigbuild yields to a pre-set `CARGO_TARGET_*_LINKER`**:
any lingering musl linker export (from a prior step or a cached `$GITHUB_ENV` restore)
silently overrides zigbuild's own linker selection. The no-op detector asserts both musl
linker vars are unset before the build, and S22 rejects any SET form anywhere in the file.
- **Version-keyed wrapper cache (`~/.cache/cargo-zigbuild/0.23.0/…`) is fresh per upgrade**:
the path includes the version; an upgrade to 0.23.1 gets a new empty directory. The post-build
step checks `~/.cache/cargo-zigbuild/0.23.0/` to prove the CURRENT version ran (not a cached
older one). Never include `~/.cache/cargo-zigbuild` in `Swatinem/rust-cache`'s `cache-directories`.
- **RELEASE_SURFACE set-equality is spec-enforced**: `RELEASE_SURFACE` in
`verify-pr-checks.mjs` and `on.pull_request.paths:` in `release.yml` must be identical sets.
Spec S10 in `release-auth-probe.spec.mjs` asserts this. If you add a path to one, add it
Expand Down Expand Up @@ -374,6 +458,10 @@ such a file. Never write `${{` in comments; describe it in words.
from the fixture directory — the shape any real non-root cwd has. `musl-load-probe.cjs`
asserts `process.cwd() === '/w'` so a dropped flag fails loudly rather than silently
returning a spurious "file not found" error. S21 pins `-w /w` in the needle list.
- **`DEBUG` env on `Build addon` was deliberately NOT added**: there is no verified `@napi-rs/cli`
debug namespace to enable, and the wrapper-cache presence check (wrappers found under
`~/.cache/cargo-zigbuild/0.23.0/wrappers/` by the post-build step) is the run-proof that
cargo-zigbuild 0.23.0 actually executed in the job.

## Key Files

Expand All @@ -389,7 +477,9 @@ such a file. Never write `${{` in comments; describe it in words.
- `scripts/__test__/release-auth-probe.spec.mjs` — specs for release.yml structure: pin shape
(S16), set equality S10, guard detection, no `${{ }}` literal (S19), `uses:` count
(S14), per-leg cache key (S20), cargo token -z guard (S3 extension), Alpine load test
job structure and wiring (S21).
job structure and wiring (S21), cargo-zigbuild musl build: `-x` flag, install-before-cache
ordering, `fallback: none`, no musl linker exports, no-op detector, post-build wrapper
assert, readelf ALLOWED_NEEDED literal, setup-zig SHA pin (S22).
- `scripts/__test__/fixtures/protection-main.json` — 6-context branch protection (historical,
2026-08 baseline; kept byte-identical).
- `scripts/__test__/fixtures/protection-main-2026-09.json` — 15-context branch protection
Expand Down
Loading
Loading