Skip to content

Incremental indexing: skip unchanged files + handle deletes #58

Description

@ryuhmanov-m

Parent issue: #17

Context

Currently reindex_vault re-embeds every file on every run. The files table already stores SHA-256 hashes, but they're not used to skip unchanged files.

Scope

1. Skip unchanged files

  • Before embedding a file, compare its current SHA-256 hash with files.hash in the database
  • If unchanged, skip embedding entirely
  • Only process new or modified files

2. Handle deleted files

  • After scanning the vault, find files in the files table that no longer exist on disk
  • Remove their chunks from the chunks table
  • Remove the file record from files

3. Handle renamed files

  • Detect renames (same hash, different path) — update file_path in chunks instead of re-embedding

Files to look at

  • src-tauri/src/adapters/indexer/ — indexing pipeline
  • src-tauri/src/adapters/db/ — database queries
  • files and chunks table schemas in migrations

Metadata

Metadata

Assignees

No one assigned

    Labels

    backendRust / Tauri backendenhancementNew feature or requestgood first issueGood for newcomersragRetrieval pipeline, embeddings, search

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions