Dev

.github/workflows/ci.yml

@@ -66,7 +66,6 @@ jobs:
   cd-equivalence:
     name: Docker cd Equivalence
     runs-on: ubuntu-latest
-    needs: rust
 
     steps:
       - name: Checkout
@@ -92,4 +91,5 @@ jobs:
       - name: Run Docker cd equivalence test
         env:
           CDS_DOCKER_IMAGE: rust:1
+          CDS_DOCKER_RANDOM_CASES: 20
         run: cargo test --test docker_cd_equivalence -- --nocapture

AGENTS.md

@@ -105,10 +105,12 @@ Current user-facing commands:
 
 ```sh
 cds --init
+cds --daemon
 cds --index [PATH...]
 cds --search QUERY...
 cds --dir-type-count
 cds --reset
+cds --restart-daemon
 cds --shell-init [bash|zsh]
 ```
 
@@ -174,27 +176,10 @@ Add tests for:
 - indexing change detection
 - ranking behavior with deterministic fake embeddings
 - shell integration output
-- Docker-backed `cd` equivalence for Bash behavior
 
 Do not make tests depend on downloading or running the real embedding model unless they
 are explicitly marked as slow/integration tests. Use a fake embedder for normal tests.
 
-The Docker equivalence test runs the project inside a Rust container and compares `cds`
-against Bash's built-in `cd` for status, stdout, stderr, `PWD`, `OLDPWD`, and physical path.
-Keep these details in mind when editing it:
-
-- CI currently uses `CDS_DOCKER_IMAGE=rust:1`.
-- The Docker image must include Cargo and C++ standard library/linker support. The
-  FastEmbed/ONNX dependency links against `libstdc++`, so very small images such as
-  `rust:1-slim` can fail with `unable to find library -lstdc++`.
-- The test runner must explicitly add `/usr/local/cargo/bin` to `PATH`; some container
-  invocations otherwise fail with `cargo: command not found`.
-- Bash includes source line numbers in `cd` diagnostics. Normalize only those line numbers
-  before comparing stderr, while keeping the actual diagnostic text exact.
-- The Docker test can skip locally when Docker is unavailable, so use
-  `CDS_DOCKER_IMAGE=rust:1 cargo test --test docker_cd_equivalence -- --nocapture`
-  with Docker access when changing equivalence behavior.
-
 ## Repository Hygiene
 
 - Keep generated databases, model files, caches, and local indexes out of git.

Cargo.toml

@@ -3,10 +3,15 @@ name = "cds"
 version = "0.1.0"
 edition = "2024"
 
+[features]
+default = ["real-embedder"]
+real-embedder = ["dep:fastembed"]
+
 [dependencies]
+chrono = { version = "0.4.38", default-features = false, features = ["clock", "std"] }
 clap = { version = "4.5.40", features = ["cargo"] }
 color-eyre = "0.6.5"
-fastembed = { version = "5.15.0", default-features = false, features = ["hf-hub-rustls-tls", "ort-download-binaries-rustls-tls"] }
+fastembed = { version = "5.15.0", default-features = false, features = ["hf-hub-rustls-tls", "ort-download-binaries-rustls-tls"], optional = true }
 serde = { version = "1.0.219", features = ["derive"] }
 serde_json = "1.0.140"
 sqlx = { version = "0.8.6", default-features = false, features = ["runtime-tokio", "sqlite", "migrate", "macros"] }

README.md

@@ -39,8 +39,9 @@ From the repository root:
 ./install.sh
 ```
 
-The installer runs `cargo install --path .` and adds an idempotent shell integration
-block to your `~/.zshrc` or `~/.bashrc`. After it finishes:
+The installer runs `cargo install --path .`, adds an idempotent shell integration block to
+your `~/.zshrc` or `~/.bashrc`, and starts the indexing daemon in the background. The daemon
+pid and log are written under `~/.cache/cds/` by default. After it finishes:
 
 ```sh
 source ~/.zshrc
@@ -61,19 +62,8 @@ To replace an existing installed binary:
 cargo test --all
 ```
 
-The test suite includes a Docker-backed equivalence test. When Docker is available, it
-mounts this project into a Rust container, builds `cds`, generates a fresh random
-filesystem tree, and compares `cds` against the shell's built-in `cd` for exit status,
-stdout, stderr, `PWD`, `OLDPWD`, and physical path behavior.
-
-Use a specific container image with:
-
-```sh
-CDS_DOCKER_IMAGE=rust:1 cargo test --test docker_cd_equivalence
-```
-
-The Docker image must include Cargo and the C++ standard library/linker support required by
-the embedding runtime. The default image is `rust:1`.
+Default builds include the real local embedding runtime. Use `--no-default-features` with
+`CDS_EMBEDDER=fake` for shell-only checks that do not need `bge-small-en-v1.5`.
 
 ## Indexing
 
@@ -112,18 +102,64 @@ The initial config looks like this:
       "*.xcassets",
       "*.imageset",
       "*.appiconset",
-      "*.colorset"
+      "*.colorset",
+      "*.svg",
+      "*.png",
+      "*.jpg",
+      "*.jpeg",
+      "*.gif",
+      "*.webp",
+      "*.ico",
+      "*.pdf",
+      "*.zip"
     ],
     "max_file_bytes": 65536,
     "max_excerpt_bytes": 4096,
     "max_entries_per_directory": 80,
     "max_depth_per_top_level_directory": 3,
-    "max_chunk_bytes": 4096
+    "max_chunk_bytes": 4096,
+    "generic_terms": [
+      "a",
+      "an",
+      "and",
+      "app",
+      "application",
+      "code",
+      "dir",
+      "directory",
+      "for",
+      "from",
+      "game",
+      "in",
+      "last",
+      "me",
+      "my",
+      "of",
+      "on",
+      "program",
+      "project",
+      "repo",
+      "repository",
+      "search",
+      "site",
+      "that",
+      "the",
+      "thing",
+      "to",
+      "tool",
+      "web",
+      "website",
+      "with",
+      "work"
+    ]
   },
   "detectors": []
 }
 ```
 
+`generic_terms` controls low-signal query words that should not dominate path filtering or
+ranking. You can add or remove words from that list without reindexing.
+
 Run an indexing pass over configured roots:
 
 ```sh
@@ -136,11 +172,35 @@ Or index explicit roots:
 cds --index ~/Projects ~/work
 ```
 
-The current indexer stores directory metadata and text-file content separately in SQLite.
-Directory rows store structured filesystem metadata: name, type, parent path, size in bytes,
-created time, modified time, accessed time, readonly status, and index time. Text files are
-stored as file metadata plus embedded content chunks linked back to their containing
-directories. Directory metadata itself is not embedded.
+Run the indexing daemon in the foreground:
+
+```sh
+cds --daemon
+```
+
+Kill existing cds daemons and start a fresh background daemon:
+
+```sh
+cds --restart-daemon
+```
+
+The daemon keeps the embedding model loaded, polls configured roots for filesystem changes,
+and reindexes changed roots without requiring another manual scan. Generated file chunk
+embeddings are also recorded in an append-only history table keyed by file fingerprint, so
+search can still match earlier content versions after a file changes. On Unix platforms, the
+daemon also opens a local socket under the cds cache directory; normal `cds --search` and
+shell-integration searches use that socket when available so the query can reuse the
+daemon's already-loaded embedding model. If the daemon is not running or the socket cannot be
+created, searches fall back to the normal direct path.
+
+The current indexer stores directory metadata and high-signal text-file content separately
+in SQLite. Directory rows store structured filesystem metadata: name, type, parent path,
+size in bytes, created time, modified time, accessed time, readonly status, and index time.
+High-signal files such as READMEs, package manifests, project config files, and SQL schemas
+are stored as file metadata plus embedded content chunks linked back to their containing
+directories. Low-signal arbitrary text files are skipped by default to keep initial indexing
+fast, and media/asset/archive files such as SVGs, images, PDFs, and ZIPs are ignored.
+Directory metadata itself is not embedded.
 Hidden directories are always skipped and pruned from the local index. Each configured index
 root is treated as a container, and each top-level directory inside it is indexed only through
 `max_depth_per_top_level_directory` levels.
@@ -288,14 +348,21 @@ List detected directory types:
 cds --dir-type-count
 ```
 
+Restart the background daemon:
+
+```sh
+cds --restart-daemon
+```
+
 Delete all indexed data from the SQLite database:
 
 ```sh
 cds --reset
 ```
 
-This prompts before deleting directory metadata, file metadata, content chunks, and directory
-type classifications. The database file and schema are kept in place.
+This prompts before deleting directory metadata, file metadata, current and historical
+content chunks, and directory type classifications. The database file and schema are kept in
+place.
 
 When the shell integration is installed, `cds` also tries semantic search automatically for
 plain directory changes that do not look like local `cd` usage. For example, `cds Projects`