Conversation
…at install time An install of this package is a download of one file for the machine doing the installing. The root package carries the loader and no binary, each of the eight platforms has its own package holding exactly one addon, and npm picks the one that fits out of optionalDependencies by os, cpu and libc. No postinstall, no node-gyp, no compiler and no fetch from anywhere but the registry. The platform list is the tier 1 rows of platforms.toml in tamnd/zu plus Windows on arm64, which is tier 2 there and cheap here because the runner exists. It lives once, in tools/platforms.mjs, and the release workflow builds it, the test suite holds package.json and the npm directories to it, and tools/packages.mjs checks that every row ended up with a binary in it before anything is published. The glibc rows build inside manylinux_2_28 rather than on the runner, because a binary linked against the runner's own glibc loads where it was built and dies on the user's. The musl rows build inside Alpine through docker rather than as a job container, because GitHub runs the JavaScript half of an action inside the job's container and has no Alpine build of it for arm64. The optional dependencies are not checked in. A package that has not been published yet resolves to nothing, so npm install records the range and no lock entry, and every npm ci after it fails on all eight. napi pre-publish writes the block from napi.targets at the version being released, and there is a test that keeps it out of the manifest for the reason it has to stay out. tools/install.mjs is the check that reads nothing: it packs the two tarballs npm would fetch, installs them into an empty project outside the checkout, and runs a statement through what came out.
Rust's aarch64-unknown-linux-musl target asks for aarch64-linux-musl-gcc, which is the name a cross toolchain gives its compiler. Inside Alpine on an arm64 machine there is no cross toolchain and the compiler is called gcc, so the build got as far as linking the first proc macro and stopped. The x86_64 musl target asks for cc and found it, which is why seven of the eight rows were green.
The flag went into RUSTFLAGS, which cargo stops applying to host artifacts as soon as a target is named, and napi names one. The host artifacts were the proc macro build scripts, so the build died in the same place with the same message. A per target linker applies to every unit built for that triple, host ones included.
33 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
npm i zudbshould be a download of one file for the machine doing the installing, and nothing else. This is the packaging that makes that true.The root package carries the loader and no binary. Each of the eight platforms has its own package holding exactly one addon, and npm picks the one that fits out of
optionalDependenciesbyos,cpuandlibc. There is nopostinstall, nonode-gyp, no compiler and no fetch from anywhere but the registry, which is what makes the package installable behind a proxy, inside a locked-down CI image, and on a machine with no toolchain on it.The eight
The tier 1 rows of
platforms.tomlin tamnd/zu, plus Windows on arm64, which is tier 2 there and cheap here because the runner exists. The list lives once, intools/platforms.mjs. The release workflow builds it, the test suite holdspackage.jsonand thenpm/directories to it, andtools/packages.mjschecks that every row ended up with a binary in it before anything is published.The glibc rows build inside
manylinux_2_28rather than on the runner, because a binary linked against the runner's own glibc loads where it was built and dies on the user's. The musl rows build inside Alpine through docker rather than as a job container, because GitHub runs the JavaScript half of an action inside the job's container and has no Alpine build of it for arm64, so a job container of Alpine is a checkout that fails on the arm runner and passes on the other. macOS x64 is cross compiled from the arm runner, which on macOS is one flag.The optional dependencies are not checked in
Naming the eight packages in
package.jsonis the obvious way to say what an install resolves. It is also a lock file npm cannot complete: a package that has not been published yet resolves to nothing,npm installrecords the range and no entry, and everynpm ciafter it fails with "missing from lock file" on all eight. That is not a guess, it is what happened here first.napi pre-publishwrites the block fromnapi.targetsat the version being released, and there is a test that keeps it out of the manifest for the reason it has to stay out.What is checked
Five tests over the manifests: no install-time script and no gyp-shaped dependency,
filesis four files and no binary, the targets and the directories and the loader's requires all agree, every platform package says which machine it is for and holds one file, and one built addon sits beside the loader.Two tools for the release:
tools/binary.mjsfails the build row that produced a file named after a different target, andtools/packages.mjschecks the whole set at once, because a release that ships seven of eight installs on the eighth platform and fails at the require.One tool that reads nothing at all:
tools/install.mjspacks the two tarballs npm would fetch, installs them into an empty project outside the checkout, and runs a statement through what came out. It is also the only thing that covers the case where seven of the eight optional dependencies cannot be fetched, which is every install today and any install of a platform whose package is missing.Verified here
The whole publish path was run locally against a staged copy: eight packages with binaries in them,
napi pre-publishwriting the eight optional dependencies at0.0.1,tools/packages.mjsgreen, the root tarball 14.3 kB with no binary in it and the platform tarball carrying exactly one, then both installed into an empty project whereconnect,exec,queryandclosedid what they say. 40 tests pass.This pull request touches the release workflow and
package.json, so the workflow runs on it, which is the point: the eight-row build matrix gets exercised here rather than on the tag, which is the one moment a release cannot be rebuilt quietly.Part of DX3, tamnd/zu#169.