feat(watch): a watched folder keeps itself current, and never deletes - #173
Merged
Merged
Conversation
#158. A folder import is a snapshot; this is what keeps it current. - documents.source_state + source_mtime_ms (migration 0021) separate the SOURCE file's state from the index state: a deleted file becomes `missing` while its index, notes, excerpts and citations all stay. - folder_watches persists the watch (one row per notebook + folder). - diffFolder() decides added / changed / restored / missing. It is pure, so the decision table is unit-tested without a filesystem or a database — and "changed" and "missing" are determined, not guessed. - FolderWatchService reconciles on launch (so a change made while the app was closed is picked up) and live via fs.watch with an 800ms debounce. New files go through the same batch import as #98; a changed file is re-parsed into the SAME documentId (a plain reindex would reuse the stored content — exactly the thing that changed). - The source list shows the `missing` state; the Add menu gains "Watch folder". Verified: npm run typecheck; npm test (409 pass, incl. 8 diffFolder tests); npm run check:design; npm run build; eval baseline unchanged; electron . --smoke-test PASS (29 checks), incl. a watched folder that imports a newly added file and marks a deleted one missing without dropping its chunks.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Watched folders (#158): mark a folder as a live source. New files are imported, changed files are re-indexed, and deleted files are marked
missing— without deleting anything.Why?
A folder import (#98) is a snapshot: a paper collection that keeps growing falls out of date the moment it changes. This is the "keep it current" counterpart, and it is deliberately a separate code path — the snapshot never runs on a timer, and the watcher never re-implements parsing.
Related issue
Fixes #158
Related to #98, #95, #154
What changed
0021):documents.source_state(available/missing/changed) andsource_mtime_ms; plus afolder_watchestable (unique per notebook + path).source_stateis separate fromstatuson purpose: a deleted file ismissingwhile its index, notes, excerpts and citations all stay.diffFolder()(folderDiff.ts, pure): decidesadded/changed/restored/missing. Being pure, the decision table is unit-tested without a filesystem or a database — and "changed" is an mtime comparison, not a guess. A row with no recorded mtime counts as changed, so the first reconcile after this upgrade does not treat old rows as current.FolderWatchService: reconciles on launch (a change made while the app was closed is not missed) and live viafs.watchwith an 800 ms debounce. New files go through the same batch import as [Feat] Batch and folder import #98; a changed file is re-parsed into the same documentId, because a plain reindex would reuse the stored content — exactly the thing that changed.missingstate; the Add menu gains "Watch folder".How was this tested?
npm run typecheck— passes.npm test— 415 pass, including 8diffFoldercases (added / changed / restored / missing, no-mtime rows, and all four computed together).npm run check:design— no violations.npm run build— passes.electron . --smoke-test— PASS (29 checks), including a real watched folder that imports a newly added file, then marks a deleted filemissingwhile asserting its chunks survive.Not verified
fs.watchpath was not observed firing in a running app; the reconcile it triggers is what the smoke test exercises. If a platform refusesrecursive: true, the watcher falls back to reconcile-on-launch (the catch is explicit inFolderWatchService).Checklist
npm run typecheckpasses.npm run buildpasses.Desktop / build changes
documentscolumns +folder_watches; migration0021).