Repository navigation
feat(embedding): built-in local backend with on-demand model download - #54
Merged
Merged
Conversation
Embedding is infrastructure, not a configuration step: uploading a PDF must not require the user to understand protocol / base URL / API key / model id / dimensions. With no remote embedding connection configured, KnowNote now uses a built-in local model, so the path collapses to "configure chat -> upload a PDF". Local inference (no bundled weights): - LocalEmbeddingBackend runs multilingual-e5-small (q8, pinned revision, 384-dim) through ONNX runtime; weights are downloaded on first RAG use and cached under userData/models. Inference sets env.allowRemoteModels=false, so transformers.js never touches the network after install. - E5 requires distinct query/document prefixes; indexing passes 'document' and search passes 'query'. Previously both used the same input. Index identity: - EmbeddingSpace (model + revision + dtype + pooling + normalize + prefixes + dimensions) is persisted per notebook. Comparing dimensions alone missed same-width model changes, whose vectors are not comparable. Search refuses a mismatched space and asks for a re-index; the indexing path rebuilds/clears the vector table and marks documents pending. Download transport (issue #53): - Checked-in manifest with real sizes and SHA256; downloader verifies every file and only then atomically renames it into place. - HTTP Range resume, per-file retry with backoff, disk-space pre-check, classified, actionable errors (DNS / refused / timeout / HTTP / checksum / disk / cancelled), and reachability probing across huggingface.co and hf-mirror.com. Manual import from a local folder is the offline fallback. - transformers.js v4.3.0 does not forward the pinned revision during its file discovery step, so revision-less hardlink aliases are created for the cache to stay readable with allowRemoteModels=false. Settings: Built-in Local vs Custom provider, with Not installed / Downloading / Installed states plus model path, size and delete. Packaging: @huggingface/transformers, onnxruntime-node and sharp are production dependencies with asarUnpack entries; the packaged smoke test loads them. Tests cover space identity, the manifest, the downloader (resume, checksum, retry, cancel, probing), error classification, model registry/aliases and the local backend prefixes. Closes #52 Closes #53
This was referenced Sep 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Implements the built-in local embedding backend (#52) and its download transport (#53).
With no remote embedding connection configured, KnowNote now indexes and searches
with a bundled runtime and an on-demand-downloaded model, instead of failing
with
No embedding model configured. The first time a user actually uses theknowledge base (upload / create knowledge base / search), the model is downloaded
once to
userData/modelsand then works fully offline. A remote connection, whenconfigured, still takes precedence and keeps the existing AI SDK path.
Why?
The RAG path required the user to understand protocol / base URL / API key /
model id / dimensions before uploading a first PDF. Public issues (#33, #35, #31,
#14, #10, #12) all trace back to this. Embedding should be infrastructure, not a
configuration step:
configure chat -> upload a PDF.Issue #53 covers making the weights arrive reliably: the default downloader in
transformers.js does not verify, resume, or retry, and
huggingface.cois notalways reachable.
Related issue
Fixes #52
Fixes #53
Fixes #33
Fixes #37
#37's report is brief, but upload is what triggers indexing, so the zero-config
local default plus the mirror fallback should remove the failure it describes.
Already closed before this PR, listed as context only (no closing keyword):
#35, #31, #14, #10, #12. The dimension-mismatch family (#14 / #31) was fixed by
the per-notebook vector table (#47); this PR adds the zero-config local default
and the
EmbeddingSpaceidentity on top of it.What changed?
EmbeddingBackendabstraction:EmbeddingServiceonly says "I want anembedding";
LocalEmbeddingBackend(ONNX) andRemoteEmbeddingBackend(existing
ModelClient) implement it. Remote is used when configured,otherwise local.
Xenova/multilingual-e5-small(q8, pinned revision,384-dim), E5
query:/passage:prefixes applied by purpose,truncationat 512 tokens,
normalize. Inference setsenv.allowRemoteModels = false.EmbeddingSpace(model + revision + dtype + pooling +normalize + prefixes + dimensions) persisted per notebook in a new
notebook_embedding_spacestable. Search rejects vectors from a differentspace; the indexing path rebuilds/clears and marks documents pending.
downloader with SHA256 verification, atomic rename, HTTP Range resume,
per-file retry/backoff, disk-space pre-check, classified actionable errors,
source reachability probing (
huggingface.cothenhf-mirror.com), andmanual import from a local folder. Download progress is broadcast to the UI.
and Not installed / Downloading / Installed states, model path, size, delete,
import and source probing.
@huggingface/transformers,onnxruntime-nodeandsharpareproduction dependencies with
asarUnpackentries; the packaged smoke testloads them.
How was this tested?
npm run typecheck(node + web + test) — passes.npm run lint— 0 errors.npm test— 49 tests pass (22 pre-existing + 27 new).npx electron-vite build— main/preload/renderer bundle builds; the@huggingface/transformersdynamic import stays external as intended.manifest files (140,461,908 bytes, every SHA256 verified), then loaded the
pinned model with
allowRemoteModels = falseand embedded text offline:dimensions === 384, all components finite, L2 norm1.0000.Chinese document vs 0.804/0.670 for others, and an English query scored 0.870
against the Eiffel Tower document — confirming the query/document prefixes and
cross-lingual behavior.
checksum mismatch cleanup, retry, cancel, probing, error classification),
model registry/aliases and local-backend prefixes.
Not run:
npm run smoke:packaged/build:unpack(needs the full electron-buildermatrix locally). The packaged smoke check was extended, but should be exercised in CI.
Notes / deferred
revisionduring its filediscovery step, which breaks offline (
allowRemoteModels=false) loading.This PR works around it by hardlinking the manifest files at the revision-less
cache path; the revision directory stays canonical so a revision bump still
forces a fresh download.
huggingface.co->hf-mirror.comprobing plus manual import is shipped. Itis a small follow-up if enterprise/internal mirrors are needed.
Screenshots / recordings
UI change: the Embedding section in Settings now offers Built-in Local vs Custom
provider with install state. No screenshot captured in this environment.
Checklist
npm run typecheckpasses.npm run buildpasses (electron-vite build).Desktop / build changes
npm run build:unpackpasses.npm run smoke:packagedpasses.