Sync aborts entire run if a file is deleted/renamed mid-sync (unhandled FileNotFoundError)

[ergut]

Summary

logseq-sync aborts the entire sync run with an unhandled FileNotFoundError if any single .md file is deleted, moved, or renamed between the initial directory scan and the point where its contents are read. Because nothing is written on abort, all embedding work for that run is lost and the vector index stays stale until a run happens to complete without a concurrent edit.

Impact

The race window is large in practice: a --once run that embeds many changed pages via Ollama takes minutes, and any concurrent mutation of the graph during that window triggers it — a Logseq autosave that rewrites/renames a page, a git pull/checkout, or a page delete via the HTTP API. A single vanished file poisons the whole run.

Observed in production: the nightly index was ~1 month stale; a manual --once catch-up run crashed at the tail (after the long embedding phase) because an unrelated page was deleted mid-run. --status afterwards still showed the old Last sync timestamp and OUT OF DATE, i.e. zero progress persisted.

Traceback

Traceback (most recent call last):
  File ".../.venv/bin/logseq-sync", line 10, in <module>
    sys.exit(main())
  File ".../src/mcp_logseq/bin/logseq_sync.py", line 227, in main
    _run_sync(config)
  File ".../src/mcp_logseq/bin/logseq_sync.py", line 109, in _run_sync
    result = engine.sync(rebuild=rebuild)
  File ".../src/mcp_logseq/vector/sync.py", line 163, in sync
    file_hash = _hash_file(file_path)
  File ".../src/mcp_logseq/vector/sync.py", line 31, in _hash_file
    h.update(path.read_bytes())
  File "/usr/lib/python3.12/pathlib.py", line 1021, in read_bytes
    with self.open(mode='rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: '.../pages/Setup___Index.md'

Root cause

Engine.sync() snapshots the file list once at the start:

• src/mcp_logseq/vector/sync.py:111 — md_files = _walk_md_files(self._config.graph_path)

It then reads file contents at three later points, none of which tolerate a path that has since disappeared:

• sync.py:123 — _hash_file(file_path) in the change-detection loop
• sync.py:148 — chunk_file(file_path, self._config) in the chunking loop
• sync.py:163 — _hash_file(file_path) in the post-embedding state-update loop (observed crash site)

The crash happened at line 163 specifically because the file still existed during the early scan/hash (line 123) but was deleted during the intervening multi-minute embedding phase (_embed_chunks_batched, line 154).

Suggested fix

Make per-file reads resilient so one missing file can't abort the run:

• Wrap each of the three read sites (or a single shared read helper) to catch FileNotFoundError for an individual path and treat it as a deletion / skip it, then continue — instead of propagating and killing the whole sync.
• A file that vanishes mid-run should be dropped from files_to_process (and, if it was previously indexed, have its chunks deleted like any other deletion) and simply picked up correctly on the next run.
• Optionally, re-check file_path.exists() immediately before hashing/chunking to narrow the window, but the try/except is the load-bearing part since exists() is itself racy.

The invariant to preserve: a concurrent edit to the graph must at worst defer a file to the next sync, never discard a whole run's progress.

Repro

1. Make enough changed pages that a --once run spends a minute or more embedding.
2. While it runs, delete or rename any page (via Logseq, the HTTP API, or the filesystem).
3. logseq-sync --once crashes with the traceback above; --status shows unchanged Last sync and OUT OF DATE.