Skip to content

deps(better-sqlite3): move to the N-API 13.x line and stop ABI rebuilds - #187

Merged
mrsibe merged 1 commit into
mainfrom
deps/better-sqlite3-13
Sep 30, 2026
Merged

mrsibe merged 1 commit into
mainfrom
deps/better-sqlite3-13

Conversation

@mrsibe

@mrsibe mrsibe commented Sep 30, 2026

Copy link
Copy Markdown
Owner

What

better-sqlite3 12.11.1 → 13.0.3, plus the packaging change that makes the upgrade actually pay off.

Why this is not just a version bump

better-sqlite3 12.x ships one ABI-specific binary per Node/Electron version and downloads it with prebuild-install. Its published prebuilds stop at electron-v146:

Electron ABI 12.11.1 prebuild?
42 146 ✅ last one published
43 148 ❌
44 149 ❌

(v12.12.0 would have had an ABI 148 asset, but it was never published to npm — npm view better-sqlite3@12.12.0 is a 404.)

So on Electron 44 electron-builder install-app-deps found no prebuild and fell back to node-gyp rebuild --build-from-source, which needs Python + MSVC:

Error: Could not find any Python installation to use
  at PythonFinder.fail (.../node-gyp/lib/find-python.js:300:11)
  ⨯ node-gyp failed to rebuild 'node_modules/better-sqlite3'

CI runners and Linux have that toolchain, which is exactly why every CI check was green while a plain Windows dev machine could not npm install.

The fix

13.x is N-API and ships a prebuild per platform/arch inside the npm package (prebuilds/win32-x64.node, darwin-arm64.node, …), with no install script. N-API binaries are stable across Node and Electron, so nothing needs rebuilding:

  • better-sqlite3 → ^13.0.3
  • Removed the postinstall hook. electron-builder install-app-deps ignores npmRebuild and always rebuilds (verified in app-builder-lib/out/util/yarn.js, installOrRebuild → rebuild), so the flag alone would not have unblocked npm install.
  • npmRebuild: false in electron-builder.yml, with the reasoning recorded there. Every native dependency here is now N-API: better-sqlite3 (package prebuilds), onnxruntime-node (bin/napi-v6/<platform>/<arch>, napi_versions: [6]), sharp (@img/sharp-*), @napi-rs/canvas. Rebuilding can now only replace a correct prebuild with a locally compiled one — and it was the thing demanding a toolchain.
  • Net effect: npm install no longer compiles anything, on any platform.

A trap this would have hit on the first release

release.yml asserted the unpacked artifact contains better_sqlite3.node. That filename no longer exists (13.x uses prebuilds/<platform>-<arch>.node), so the macOS release check would have failed the first tagged release with "better_sqlite3.node was not unpacked". It now asserts a better-sqlite3/prebuilds/*.node file. Verified the old build/Release/better_sqlite3.node really is gone.

Docs corrected where the N-API change made them false: CONTRIBUTING (no toolchain needed), docs/dependency-audit.md ("rebuilt against the Electron ABI / cannot load in plain Node", and SQLite 3.53.2 → 3.53.4), the smokeTest.ts comment, and the entitlements comment.

Verified (Linux x64, WSL2)

Check Result
npm install 13.0.3 with prebuilds, no build/ directory → nothing compiled
npm run typecheck clean
npm test 472/472 pass
npm run build:unpack skipped dependencies rebuild reason=npmRebuild is set to false
npm run smoke:packaged 29 checks pass

From the packaged run:

[SmokeTest] ok   better-sqlite3 opened the database and loaded the sqlite-vec extension
[SmokeTest] ok   FTS5 is available and ranks: sqlite 3.53.4, ENABLE_FTS5 flag present
[SmokeTest] ok   sqlite-vec reports version v0.1.9
[SmokeTest] PASS - 29 checks passed

Left to CI, and the one thing to watch

macOS arm64 and Windows packaging. Cross-arch is the risk: npmRebuild: false means electron-builder does not rebuild for a target arch different from the host. It does not make cross-arch worse than before — @electron/rebuild never fetched sharp's or @napi-rs/canvas's target-arch prebuilds either — and better-sqlite3 now carries every platform/arch itself while onnxruntime-node ships all of them in one package. The release matrix builds each artifact on its own architecture. If a cross-arch build is ever added, this is the setting to revisit.

Separately noticed (not fixed here)

While measuring sizes: packaged onnxruntime-node is now ~416 MB, of which ~260 MB is libonnxruntime_providers_cuda.so. That predates this change (it is what npm ci installs on main) and is the "ships binaries for the wrong platform" follow-up already recorded in the audit — only much larger now. Recorded in the audit table so the stale 155 MB figure does not mislead.

Rollback

Revert the branch. No schema or storage-format change; better-sqlite3 13 still bundles SQLite 3.53.4 and the same on-disk format, and the packaged app re-reads existing databases unchanged.

better-sqlite3 12.11.1 -> 13.0.3.

Why this is not just a version bump: 12.x ships one ABI-specific binary per
Electron version and downloads it with prebuild-install. Its published prebuilds
stop at electron-v146 (Electron 42). Electron 44 is ABI 149, so
`electron-builder install-app-deps` could not find a prebuild and fell back to
`node-gyp rebuild --build-from-source`, which needs Python + MSVC. CI runners and
Linux have that toolchain, so every CI check passed; a plain Windows dev machine
did not, and `npm install` failed with "Could not find any Python installation to
use".

13.x is N-API and ships a prebuild per platform/arch inside the npm package
itself (prebuilds/win32-x64.node, darwin-arm64.node, ...), with no install script.
N-API binaries are stable across Node and Electron versions, so there is nothing
left to rebuild.

Changes:

- `better-sqlite3` -> `^13.0.3`.
- Removed the `postinstall` hook. `electron-builder install-app-deps` ignores
  `npmRebuild` and always rebuilds; with every native dependency now N-API that
  rebuild can only replace a correct prebuild with a locally compiled one.
- `npmRebuild: false` in electron-builder.yml, with the reasoning recorded there.
  This also means packaging no longer needs a C++ toolchain.
- release.yml asserted that `better_sqlite3.node` was unpacked. That filename no
  longer exists (13.x uses `prebuilds/<platform>-<arch>.node`), so the check
  would have failed the first macOS release. It now asserts a
  `better-sqlite3/prebuilds/*.node` file instead.
- CONTRIBUTING: the native-toolchain requirement is gone.
- docs/dependency-audit.md: better-sqlite3 is no longer "rebuilt against the
  Electron ABI and cannot load in plain Node"; SQLite is now 3.53.4, not 3.53.2.

Verified on Linux x64 (WSL2):

- `npm install` - better-sqlite3 13.0.3 with prebuilds, and no `build/`
  directory, i.e. nothing was compiled.
- `npm run typecheck` - clean; `npm test` - 472/472 pass.
- `npm run build:unpack` - "skipped dependencies rebuild reason=npmRebuild is set
  to false"; the packaged app carries every platform's prebuild.
- `npm run smoke:packaged` - 29 checks pass, better-sqlite3 opens the database
  from the N-API prebuild, FTS5 + bm25() rank, sqlite-vec v0.1.9 loads.

Not verified locally, left to CI: macOS arm64 and Windows packaging. The one
thing to watch is cross-arch. Rebuilding could never fetch sharp's or
@napi-rs/canvas's target-arch prebuilds anyway, so `npmRebuild: false` does not
make that worse; better-sqlite3 now carries every platform/arch itself, and the
release matrix builds each artifact on its own architecture.

Separately noticed while measuring: the packaged onnxruntime-node is ~416 MB, of
which ~260 MB is `libonnxruntime_providers_cuda.so`. That predates this change
(it is what `npm ci` installs) and is the "ships binaries for the wrong platform"
follow-up already recorded in the audit, only larger.
@github-actions github-actions Bot added the dependencies Dependency updates label Sep 30, 2026
@mrsibe
mrsibe merged commit 19f76e3 into main Sep 30, 2026
4 checks passed
@mrsibe
mrsibe deleted the deps/better-sqlite3-13 branch September 30, 2026 06:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Dependency updates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant