Skip to content

feat(APP-204): pre-cache DuckDB extensions, remove runtime download#22046

Merged
insomnious merged 5 commits intomasterfrom
halgari/duckdb-extension-caching
Apr 1, 2026
Merged

feat(APP-204): pre-cache DuckDB extensions, remove runtime download#22046
insomnious merged 5 commits intomasterfrom
halgari/duckdb-extension-caching

Conversation

@halgari
Copy link
Copy Markdown
Contributor

@halgari halgari commented Mar 31, 2026

Closes APP-204.

Summary

  • Download script (scripts/download-duckdb-extensions.ts) fetches .duckdb_extension.gz files at install/build time and decompresses them into src/main/build/duckdb-extensions/{version}/{platform}/
  • DuckDBSingleton: removed FORCE INSTALL ... FROM 'https://...'; sets extension_directory so LOAD level_pivot resolves from the local cache — no network call at startup
  • LevelPersist: passes getVortexPath("base")/duckdb-extensions as the extension directory
  • InstallAssets.json: copies pre-downloaded extensions into out/ and dist/
  • postinstall in src/main/package.json: downloads extensions automatically after pnpm install
  • build:assets / dist:assets: download step prepended so CI and packaging always have fresh extensions

Test plan

  • pnpm install triggers postinstall and downloads extensions
  • pnpm run build:assets skips download if already cached, copies to out/
  • Vortex starts without network activity for DuckDB extension loading
  • All download script unit tests pass

@halgari halgari requested a review from a team as a code owner March 31, 2026 16:21
…ownload

- Download script: fix CJS compat (import.meta.dirname → __dirname), add
  gzip decompression, correct @duckdb/node-api package path
- DuckDBSingleton: drop FORCE INSTALL over HTTPS; use extension_directory
  config so LOAD level_pivot resolves from the pre-downloaded cache
- LevelPersist: pass getVortexPath("base")/duckdb-extensions as extensionDir
- InstallAssets.json: copy pre-downloaded extensions into out/ and dist/
- package.json: run download script in build:assets and dist:assets pipelines
- src/main/package.json: add postinstall to download extensions after pnpm install
@halgari halgari force-pushed the halgari/duckdb-extension-caching branch from e2fd139 to 3e296d2 Compare March 31, 2026 16:35
halgari added 2 commits March 31, 2026 11:37
Download now runs explicitly via build:assets and dist:assets in the root package.json.
…cess

DuckDB's C++ extension loader uses OS file I/O and cannot read from inside
an Electron ASAR archive. Add duckdb-extensions to asarUnpack so they land
in app.asar.unpacked/, and introduce getVortexPath("base_unpacked") so
LevelPersist always points DuckDB at the real filesystem path.
@insomnious insomnious merged commit e3a8839 into master Apr 1, 2026
5 checks passed
@insomnious insomnious deleted the halgari/duckdb-extension-caching branch April 1, 2026 06:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants