Summary
When RTFM runs inside a container, the embedding model cache lands in a directory that does not survive a container recreate. The model (~85 MB) is silently re-downloaded on every recreate, and if a recreate happens between the download and the embed jobs, every embed job fails with a low-level ONNX error that doesn't say what's actually wrong.
Current behaviour
rtfm/core/embeddings.py:92:
os.environ.get("FASTEMBED_CACHE_PATH") or (Path.home() / ".cache" / "fastembed")The fallback is sensible on a workstation. In a container, $HOME is on the ephemeral writable layer, so the cache is wiped by docker compose up -d / docker run after a rebuild — while the index itself (bind-mounted .rtfm/) persists. The two halves of the same state have different lifetimes.
When the model is absent at embed time, the failure surfaces as:
NoSuchFile: [ONNXRuntimeError] : 3 : NO_SUCHFILE : Load model from
/tmp/fastembed_cache/models--qdrant--paraphrase-multilingual-MiniLM-L12-v2-onnx-Q/snapshots/.../
Nothing in that message points to the cache directory as the cause, and rtfm failed buckets it as other.
Impact
On a long-running indexing host (agent container, CI, K8s), every image rebuild costs a re-download, and any embed job that races a recreate fails. In our case the whole embed queue was failing while ingestion looked healthy — search silently degraded to lexical-only, which is easy to miss because search still returns results.
Suggestions (any one helps)
- Prefer the project's own
.rtfm/ directory when one exists — e.g. .rtfm/fastembed. The model then shares the lifetime of the index it serves, which is what users bind-mount anyway. Falls back to ~/.cache/fastembed when there's no project dir.
- Document
FASTEMBED_CACHE_PATH in the install / deployment notes, with an explicit "if you run RTFM in a container, point this at a persistent path" line.
- Translate the ONNX error: catch
NO_SUCHFILE on model load and raise something like "embedding model not found in cache <path> — it may have been cleared; set FASTEMBED_CACHE_PATH to a persistent directory".
Environment
- rtfm-ai 0.26.6, fastembed 0.8.0, Python 3.11,
node:22-bookworm-slim container
.rtfm/ bind-mounted from the host; $HOME inside the container's writable layer- Workaround applied on our side:
FASTEMBED_CACHE_PATH=<project>/.rtfm/fastembed — embeddings resumed immediately (22 314 chunks embedded on the first pass).
Note on an older version
We reached this while upgrading a container stuck on 0.24.1. Two things worth knowing, both already fixed in 0.26.6 — recording them only in case they're useful for release notes:
- 0.24.1 ignored
honor_gitignore, so a corpus of gitignored PDFs indexed as zero files, with no warning.
- 0.24.1 requested
qdrant/paraphrase-multilingual-MiniLM-L12-v2-onnx-Q, a name current fastembed rejects (ValueError: Model ... is not supported). 0.26.6 uses sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 and works.
Summary
When RTFM runs inside a container, the embedding model cache lands in a directory that does not survive a container recreate. The model (~85 MB) is silently re-downloaded on every recreate, and if a recreate happens between the download and the embed jobs, every embed job fails with a low-level ONNX error that doesn't say what's actually wrong.
Current behaviour
rtfm/core/embeddings.py:92:The fallback is sensible on a workstation. In a container,
$HOMEis on the ephemeral writable layer, so the cache is wiped bydocker compose up -d/docker runafter a rebuild — while the index itself (bind-mounted.rtfm/) persists. The two halves of the same state have different lifetimes.When the model is absent at embed time, the failure surfaces as:
Nothing in that message points to the cache directory as the cause, and
rtfm failedbuckets it asother.Impact
On a long-running indexing host (agent container, CI, K8s), every image rebuild costs a re-download, and any embed job that races a recreate fails. In our case the whole embed queue was failing while ingestion looked healthy — search silently degraded to lexical-only, which is easy to miss because search still returns results.
Suggestions (any one helps)
.rtfm/directory when one exists — e.g..rtfm/fastembed. The model then shares the lifetime of the index it serves, which is what users bind-mount anyway. Falls back to~/.cache/fastembedwhen there's no project dir.FASTEMBED_CACHE_PATHin the install / deployment notes, with an explicit "if you run RTFM in a container, point this at a persistent path" line.NO_SUCHFILEon model load and raise something like "embedding model not found in cache<path>— it may have been cleared; setFASTEMBED_CACHE_PATHto a persistent directory".Environment
node:22-bookworm-slimcontainer.rtfm/bind-mounted from the host;$HOMEinside the container's writable layerFASTEMBED_CACHE_PATH=<project>/.rtfm/fastembed— embeddings resumed immediately (22 314 chunks embedded on the first pass).Note on an older version
We reached this while upgrading a container stuck on 0.24.1. Two things worth knowing, both already fixed in 0.26.6 — recording them only in case they're useful for release notes:
honor_gitignore, so a corpus of gitignored PDFs indexed as zero files, with no warning.qdrant/paraphrase-multilingual-MiniLM-L12-v2-onnx-Q, a name current fastembed rejects (ValueError: Model ... is not supported). 0.26.6 usessentence-transformers/paraphrase-multilingual-MiniLM-L12-v2and works.