Read the archives we ship without an external tool, and give every deployment gate a binary it can start

[MelbourneDeveloper]

TLDR

Five deployment gates could not run on Windows — three because they shelled out to unzip, which Windows does not ship, and two because they staged shell scripts where a binary had to be started. All ten now pass there, the archive format we ship is read in-process, and the Windows CI job runs the gates that guard a Windows release for the first time.

Details

Three verifiers could not open the artifact they verify. verify-jetbrains-package.mjs, verify-vsix-package.mjs and assert-vsix-schema-doc.mjs each shelled out to Info-ZIP's unzip. Windows ships neither zip nor unzip, so on the platform a win32-x64 VSIX is built for — and where a maintainer is most likely to want to check one before publishing — they aborted on a missing program before checking anything. That is not a weaker answer; it is indistinguishable from having no gate.

scripts/lib/zip.mjs reads and writes the format we actually ship. A VSIX and a JetBrains plugin are both zip archives, and the format is small enough to read directly — which buys two things a subprocess cannot: every entry's checksum is verified as it is decompressed, so a corrupt archive is a named failure rather than a confusing one, and an archive written here is byte-for-byte reproducible, so a fixture cannot differ between two runs. Only what our archives contain is supported: stored and deflated entries in a single-part archive, with Zip64 and any other compression method refused by name rather than guessed at.

The proofs staged something Windows cannot start. The verifier proof suite wrote #!/bin/sh scripts as stand-ins for shipped binaries. Windows has no shebang and CreateProcess will not start a text file, so every proof spawned nothing, got a null status and empty output, and compared that against an expectation which was also empty. Those comparisons passed. This is the more dangerous of the two failures, because it is green.

scripts/lib/fake-binary.rs is compiled once per run — rustc is the one compiler this repository is guaranteed to have, and the CI job that runs these gates already installs the toolchain — and copied per fixture, each copy carrying its own two answers appended after a marker. The answers ride inside the image rather than in a file beside it because a second file staged next to a binary is an undeclared package entry, which the verifiers are right to reject.

One rule about file names, written four times and left out of a fifth. A component's file is its binaryName plus .exe on Windows and nothing elsewhere. test-action-diff-gate.mjs did not carry that rule and looked for target/release/deslop, so it reported the CLI missing on the one platform that has it. executableName in scripts/release/vsix-platforms.mjs is now the only implementation, read by all three verifiers, the fixtures and the action gate; currentPlatform there replaces three more copies of the host-platform chain, including the extension's own platform.mjs. New spec section [DEPLOY-BINARY-FILE-NAME].

Three more gates reached for things Windows resolves differently. bash by name, which finds WSL's copy first and sees a different filesystem than the checkout; a POSIX PATH assembled by hand around a drive letter, which the shell reads as two entries and finds neither; and rm, which is not a Windows program. Each now uses the resolved shell, a shell-spelled path passed as a positional argument, or Node's own rmSync.

publish-vsixes.mjs could not start npx on Windows. An npm command there is a .cmd shim, and Node refuses to spawn a batch file without a shell at all — so the publish-contract suite could not distinguish a broken publisher from an unreachable stub. The script now reaches an installed command through the same POSIX shell the workflow step runs under (spawnableCommand), with every argument positional and nothing interpolated into a command line. It is the identity off Windows, so the Linux release path is unchanged.

Duplication. Measured with the CLI over the same tree, with the **/scripts/** exclusion lifted so the numbers mean something (that exclusion is why the repo-wide figure says nothing about this tree): 42 clusters / mass 2154 before, 32 / 1600 after — a 26% reduction — and none of the seven clusters that touched the changed files survive. The zip header writers now share one field-writing routine, the twenty-two proof bodies share one runner per verifier, the two package builders share their staging root and their bundled-LSP staging, expectFail/expectSuccess share one runner, and the stage-a-file idiom that appeared in three files is writeFileAt/copyFileAt. The stamper proof's four copies of its spawn-and-check are one runStamper, and its two copies of the Cargo assertions are one assertCargoVersion. What is left in a changed file is two clusters of mandatory idiom at the noise floor — making a temp directory and naming two children of it, in two unrelated files, and a two-line "nothing leaked, clean up" tail whose copies assert different outcomes with different messages. A helper for either would read worse than the lines it replaced. test-verifiers.mjs was over the 500-line limit before this change and is now 328, with its fixtures in verifier-fixtures.mjs.

New spec sections [DEPLOY-BINARY-FILE-NAME] and [DEPLOY-GATE-PORTABILITY] in docs/specs/deployment.md, cross-referenced from every file that implements them.

Breaking changes

None. No CLI flag, report field, or existing spec ID changed. publish-vsixes.mjs spawns through a shell on Windows only; on the Linux runners the release actually uses, the argv is byte-identical to before.

How Do The Automated Tests Prove It Works?

scripts/deployment/zip.test.mjs is new and exists to defeat the obvious way this change could be wrong — a reader and a writer that agree with each other and with nothing else. It holds the reader against clients/jetbrains/gradle/wrapper/gradle-wrapper.jar, an archive this repository did not produce: the entries it must carry, a text entry that must start Manifest-Version:, and a deflated, checksummed .class entry whose first four bytes must be CA FE BA BE — which exercises inflate and the CRC check against bytes nothing here wrote. Only then is the writer held against the reader: a staged tree round-trips by name and content, entry names carry no host separator even though the host joined those paths with one, extraction restores the mode the staged file had (stated against the source, not against 0o755, because NTFS cannot record an execute bit and a fixed expectation would be asserting the host), two archives of one tree are byte-identical, and the archive is smaller than its contents. Refusals are asserted too: a non-archive and a truncated archive are both refused naming the missing end-of-central-directory record.

test-verifiers.mjs goes from 13 passing of 34 on Windows to 34, with no assertion relaxed — the fixtures changed, not the expectations. binariesNameTheFileEachTargetPlatformSpells is new and asserts the full expected path for all five published platforms from any host, so deslop.exe is pinned on Linux and deslop on Windows; nothing runs, which is what makes that possible. It replaces a hard-coded darwin-arm64 that left the .exe branch unexercised on every machine that has ever run this suite. Every proof that does execute a binary now runs against the host platform, so binariesAcceptValidContract and vsixAcceptsValidPackage exercise the win32 naming and staging path positively for the first time.

test-release-publish-contract.mjs goes from 3 passing of 8 to 8, including the five that assert each platform is attempted exactly once when one registry call fails. test-action-diff-gate.mjs goes from 0 of 5 (it died on the missing binary before any check) to 5, and it runs the action's own Run deslop step body verbatim against the branch-built CLI.

The reader was also run against a real artifact, not only fixtures. verify-vsix-package.mjs was pointed at a 14 MB vsce-produced deslop-live-win32-x64.vsix: it read all 16 entries, checked them against the shipping allow-list, extracted and executed all three bundled .exe binaries for their version contract, and scanned the packaged assets for stub-provider strings — exit 0. assert-vsix-schema-doc.mjs matched the packaged schema_doc.md byte-for-byte against docs/specs/REPORTING-CONTEXT.md from the same archive. Neither had ever run on Windows before.

The reader also refuses rather than guesses: an entry whose 32-bit size or offset carries the Zip64 sentinel is rejected by name, proved by stamping that sentinel onto a real catalogue entry and asserting the read fails. Handing back a wrong length there is the one corruption a checksum cannot catch, because the checksum would be computed over the wrong bytes.

Full local Windows run of the deployment sequence — all ten gates in make deployment-verify order — exits 0. Every repository contract test passes: 87 across the eleven node --test suites, plus the four contract runners. make lint (clippy -D warnings plus every Node gate, now including the zip gate), make fmt CHECK=1, and make dup-gate all exit 0.

CI could not previously catch any of this. The Windows job ran Rust only. It now runs the host-shell gate, the zip gate, the PATH scrub, the installer snippet, the verifier proofs and the publish contract, ahead of the instrumented build so they fail fast.

For AI

openArchive(path) reads the central directory once and returns {names, read, readText, extract}; writeArchive(archivePath, sourceRoot, entryRoot) walks sorted and emits directory records the way zip -rq did, so the VSIX allow-list sees the same entry shape it saw before. External attributes carry the Unix type bits plus 0o777 in the high half — note that OR-ing into a shifted value yields a signed int32 in JS and must be made unsigned on the outside, not the inside, which is the one bug that bit during development. describeEntry writes the five fields both headers share; the central directory's copy sits exactly two bytes later because it carries a versionMadeBy the local header does not, which is why the two writers were a detected clone.

fake-binary.rs finds its payload with rposition, not position: the marker is also a string constant in the compiled image, so the first match is the program's own rodata and the appended copy is always last. Built -O -C strip=symbols because each fixture is copied into a zip and deflated several times.

spawnableCommand(name, args) returns [file, argv] — the bare name and its arguments off win32, and on it the resolved POSIX shell running exec "$@" with the command and its arguments supplied positionally. $0 takes the name so $@ is the command plus its arguments; nothing is interpolated into script text. The same positional-argument technique prepends the CLI's directory to PATH inside the action step's shell, because a Windows directory handed in through env carries the character PATH separates on.

clients/vscode/scripts/* now imports ../../../scripts/lib/zip.mjs and ../../../scripts/release/vsix-platforms.mjs. Safe: .vscodeignore excludes scripts/**, so none of it is packaged, and both are dependency-free ESM run by node directly.

gh #500. No unwrap/expect/panic! added, no suppressions, no file over 500 lines, no function over 20.

🤖 Generated with Claude Code