Repository navigation
deps(better-sqlite3): move to the N-API 13.x line and stop ABI rebuilds - #187
Merged
Merged
Conversation
better-sqlite3 12.11.1 -> 13.0.3. Why this is not just a version bump: 12.x ships one ABI-specific binary per Electron version and downloads it with prebuild-install. Its published prebuilds stop at electron-v146 (Electron 42). Electron 44 is ABI 149, so `electron-builder install-app-deps` could not find a prebuild and fell back to `node-gyp rebuild --build-from-source`, which needs Python + MSVC. CI runners and Linux have that toolchain, so every CI check passed; a plain Windows dev machine did not, and `npm install` failed with "Could not find any Python installation to use". 13.x is N-API and ships a prebuild per platform/arch inside the npm package itself (prebuilds/win32-x64.node, darwin-arm64.node, ...), with no install script. N-API binaries are stable across Node and Electron versions, so there is nothing left to rebuild. Changes: - `better-sqlite3` -> `^13.0.3`. - Removed the `postinstall` hook. `electron-builder install-app-deps` ignores `npmRebuild` and always rebuilds; with every native dependency now N-API that rebuild can only replace a correct prebuild with a locally compiled one. - `npmRebuild: false` in electron-builder.yml, with the reasoning recorded there. This also means packaging no longer needs a C++ toolchain. - release.yml asserted that `better_sqlite3.node` was unpacked. That filename no longer exists (13.x uses `prebuilds/<platform>-<arch>.node`), so the check would have failed the first macOS release. It now asserts a `better-sqlite3/prebuilds/*.node` file instead. - CONTRIBUTING: the native-toolchain requirement is gone. - docs/dependency-audit.md: better-sqlite3 is no longer "rebuilt against the Electron ABI and cannot load in plain Node"; SQLite is now 3.53.4, not 3.53.2. Verified on Linux x64 (WSL2): - `npm install` - better-sqlite3 13.0.3 with prebuilds, and no `build/` directory, i.e. nothing was compiled. - `npm run typecheck` - clean; `npm test` - 472/472 pass. - `npm run build:unpack` - "skipped dependencies rebuild reason=npmRebuild is set to false"; the packaged app carries every platform's prebuild. - `npm run smoke:packaged` - 29 checks pass, better-sqlite3 opens the database from the N-API prebuild, FTS5 + bm25() rank, sqlite-vec v0.1.9 loads. Not verified locally, left to CI: macOS arm64 and Windows packaging. The one thing to watch is cross-arch. Rebuilding could never fetch sharp's or @napi-rs/canvas's target-arch prebuilds anyway, so `npmRebuild: false` does not make that worse; better-sqlite3 now carries every platform/arch itself, and the release matrix builds each artifact on its own architecture. Separately noticed while measuring: the packaged onnxruntime-node is ~416 MB, of which ~260 MB is `libonnxruntime_providers_cuda.so`. That predates this change (it is what `npm ci` installs) and is the "ships binaries for the wrong platform" follow-up already recorded in the audit, only larger.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
better-sqlite312.11.1 → 13.0.3, plus the packaging change that makes the upgrade actually pay off.Why this is not just a version bump
better-sqlite312.x ships one ABI-specific binary per Node/Electron version and downloads it withprebuild-install. Its published prebuilds stop atelectron-v146:12.11.1prebuild?(
v12.12.0would have had an ABI 148 asset, but it was never published to npm —npm view better-sqlite3@12.12.0is a 404.)So on Electron 44
electron-builder install-app-depsfound no prebuild and fell back tonode-gyp rebuild --build-from-source, which needs Python + MSVC:CI runners and Linux have that toolchain, which is exactly why every CI check was green while a plain Windows dev machine could not
npm install.The fix
13.x is N-API and ships a prebuild per platform/arch inside the npm package (
prebuilds/win32-x64.node,darwin-arm64.node, …), with no install script. N-API binaries are stable across Node and Electron, so nothing needs rebuilding:better-sqlite3→^13.0.3postinstallhook.electron-builder install-app-depsignoresnpmRebuildand always rebuilds (verified inapp-builder-lib/out/util/yarn.js,installOrRebuild→rebuild), so the flag alone would not have unblockednpm install.npmRebuild: falseinelectron-builder.yml, with the reasoning recorded there. Every native dependency here is now N-API:better-sqlite3(package prebuilds),onnxruntime-node(bin/napi-v6/<platform>/<arch>,napi_versions: [6]),sharp(@img/sharp-*),@napi-rs/canvas. Rebuilding can now only replace a correct prebuild with a locally compiled one — and it was the thing demanding a toolchain.npm installno longer compiles anything, on any platform.A trap this would have hit on the first release
release.ymlasserted the unpacked artifact containsbetter_sqlite3.node. That filename no longer exists (13.x usesprebuilds/<platform>-<arch>.node), so the macOS release check would have failed the first tagged release with "better_sqlite3.node was not unpacked". It now asserts abetter-sqlite3/prebuilds/*.nodefile. Verified the oldbuild/Release/better_sqlite3.nodereally is gone.Docs corrected where the N-API change made them false:
CONTRIBUTING(no toolchain needed),docs/dependency-audit.md("rebuilt against the Electron ABI / cannot load in plain Node", and SQLite 3.53.2 → 3.53.4), thesmokeTest.tscomment, and the entitlements comment.Verified (Linux x64, WSL2)
npm installbuild/directory → nothing compilednpm run typechecknpm testnpm run build:unpackskipped dependencies rebuild reason=npmRebuild is set to falsenpm run smoke:packagedFrom the packaged run:
Left to CI, and the one thing to watch
macOS arm64 and Windows packaging. Cross-arch is the risk:
npmRebuild: falsemeans electron-builder does not rebuild for a target arch different from the host. It does not make cross-arch worse than before —@electron/rebuildnever fetchedsharp's or@napi-rs/canvas's target-arch prebuilds either — andbetter-sqlite3now carries every platform/arch itself whileonnxruntime-nodeships all of them in one package. The release matrix builds each artifact on its own architecture. If a cross-arch build is ever added, this is the setting to revisit.Separately noticed (not fixed here)
While measuring sizes: packaged
onnxruntime-nodeis now ~416 MB, of which ~260 MB islibonnxruntime_providers_cuda.so. That predates this change (it is whatnpm ciinstalls on main) and is the "ships binaries for the wrong platform" follow-up already recorded in the audit — only much larger now. Recorded in the audit table so the stale 155 MB figure does not mislead.Rollback
Revert the branch. No schema or storage-format change;
better-sqlite313 still bundles SQLite 3.53.4 and the same on-disk format, and the packaged app re-reads existing databases unchanged.