Skip to content

Live inbound sync for directory libraries (phase 2) - #738

Open
koppor wants to merge 26 commits into
open-directory-libraryfrom
directory-inbound-sync
Open

koppor wants to merge 26 commits into
open-directory-libraryfrom
directory-inbound-sync

Conversation

@koppor

@koppor koppor commented Jul 13, 2026 •

Copy link
Copy Markdown
Member

📚 Directory-as-library stack — bottom → top, each builds on the one below:

  1. Programmatic Hayagriva writer (directory-as-library, phase 0) #736 · Hayagriva YAML writer
  2. Open folder as library (directory as library, phase 1) #737 · Open folder as library
  3. Live inbound sync for directory libraries (phase 2) #738 · Inbound file sync ← this PR
  4. Write-back to Hayagriva sidecars (directory as library, phase 3) #739 · Write-back to sidecars
  5. Groups panel mirrors the directory structure (phase 4, #10930) #740 · Directory-structure groups
  6. Pattern-driven pair renames for directory libraries (phase 5a) #741 · Pattern-driven pair renames
  7. Mirror directory libraries into a .bib with git-sync merge-back #743 · .bib mirror + merge-back
  8. Convert .bib libraries into directory libraries #744 · Convert .bib → directory
  9. Make directory libraries reachable through the REST API #745 · REST API support
  10. Split DirectoryLibrarySynchronizer into collaborators #760 · Split the synchronizer

Related issues and pull requests

Phase 2 of the "directory as library" plan (PLAN.md), stacked on #737 (this PR's base branch is open-directory-library; the diff shows only the sync commits). Related to JabRef#10930; no issue is closed by this PR.

PR Description

🤖 An open directory library now follows external file changes live: creating, editing, renaming, or deleting a sidecar or PDF on disk is reflected in the open library within about a second, while selection, undo history, and group membership survive. Renames are recognised as moves rather than as a delete plus an unrelated add. The library's own writes are ignored, so its changes are not re-imported.

jabref-contrib-policy:4.2:reviewed​:ok

Analogies

Like honey trickling back into the comb, external edits now find their way into the open library without anyone carrying them. Like chocolate that snaps along the same grooves no matter who breaks it, the same entry instances survive renames and edits — the pieces keep their shape. And like the moon steadying the tides, the one-second poll quietly keeps two worlds — disk and table — moving together.

Steps to test

  1. Open a folder as library (File > Open folder as library...) with a few sidecars.
  2. While the tab is open, from a terminal: add a new .yml, edit an existing one, rename one (mv a.yml b.yml), delete one, and add/remove a PDF next to a sidecar.
  3. Within about a second each change appears in the main table; the renamed file's entry stays selected if it was selected.

Row added live after creating a sidecar externally

Guarded by DirectoryLibrarySynchronizerTest (10 tests: create/change/delete, multi-entry diff, rename grace window, self-echo fingerprint, PDF link/stub handling, non-Hayagriva takeover).

AI usage

Claude Code (model claude-fable-5).

AI CHECKLIST.md walkthrough

1. Code self-review

Nullability and control flow
  • No == null / != null in new code except nullable JDK returns (Path.getParent(), map lookups) expressed via Optional.ofNullable where streamable.
  • No Objects.requireNonNull(...) — orElseThrow with message for the constructor contract.
  • New classes annotated with @NullMarked.
  • Optional consumed with ifPresent / map / orElseThrow.
  • [/] StringUtil.isBlank(...) — not needed.
Exceptions
  • Only IOException / NoSuchAlgorithmException caught specifically.
  • No RuntimeException/IllegalStateException thrown (IllegalArgumentException for a violated constructor precondition; AssertionError only for the impossible missing-SHA-256 case).
  • Logged exceptions passed as last logger argument.
Style and idioms
  • BibEntry withers used for new entries.
  • Modern Java: records, List.of(), SequencedMap, streams.
  • [/] Regexes — none added.
  • Long-running work on a dedicated daemon executor; UI marshalling injected (UiTaskExecutor::runInJavaFXThread), no new Thread() in application flow (the executor's thread factory names its worker).
  • No commented-out code, no trivial comments.
  • Markdown Javadoc (///).
User-facing text
  • [/] No new user-facing strings (sync is silent; warnings only in logs).
Security
  • [/] No HTML output.
Tests
  • DirectoryLibrarySynchronizerTest drives the handlers deterministically (injected clock for the grace window, direct-run marshaller); no sleeps, no polling.
  • Plain JUnit asserts, @TempDir, no @DisplayName, exceptions propagate.

2. Verification commands

  • ./gradlew :jablib:check (full suite; only the known environmental RemoteCommunicationTest port-conflict failures on this machine).
  • ./gradlew :jablib:checkstyleMain :jablib:checkstyleTest :jabgui:checkstyleMain.
  • ./gradlew :jablib:modernizer.
  • ./gradlew :rewriteRun left no diff.
  • ./gradlew :jablib:javadoc.
  • ./gradlew traceRequirements (req~directory-library.inbound-sync~1 covered).
  • npx markdownlint-cli2 on changed Markdown.
  • [/] IntelliJ-format docker image — not runnable here; CI format job is the backstop.

3. Documentation

  • CHANGELOG.md entry added (TODO replaced with this PR's link after creation).
  • Requirement added to docs/requirements/directory-library.md.
  • [/] ADR — covered by the existing ADR-0072 (this phase implements its sync section).

4. Pull request

  • PR body from the template, every section filled, HTML comments removed, all items marked.
  • Created from a body file via gh api (cross-repo head, stacked base open-directory-library).
  • CHANGELOG TODO replaced after creation, committed and pushed.

Checklist

  • I own the copyright of the code submitted and I license it under the MIT license
  • If AI tools were used, I disclosed them in the "AI usage" section and reviewed, understood, and take full ownership of all AI-generated code
  • I manually tested my changes in running JabRef (externally created/edited/renamed sidecars while the library tab was open on the X display; see screenshot)
  • I added JUnit tests for changes (if applicable)
  • I added screenshots in the PR description (if change is visible to the user)
  • [/] I added a screenshot in the PR description showing a library with a single entry with me as author and as title the issue number
  • I added one sentence (max 20 words) to CHANGELOG.md describing the change from the user's point of view (if the change is visible to the user)
  • I checked the user documentation for up to dateness and submitted a pull request to our user documentation repository — deferred until the feature stabilizes (write-back phase); tracked in PLAN.md phase 5

🤖 Generated with Claude Code

koppor and others added 10 commits July 14, 2026 00:07
A DirectoryLibrarySynchronizer registers with the Commons-IO
DirectoryMonitor (promoted from jabgui to jablib, per ADR-0030) and
applies external create/change/delete events for .yml/.pdf files to the
open library: changed sidecars update the existing BibEntry instances
field by field, renames are detected via a grace window over the
monitor's delete+create events and keep the affected entries, appearing
PDFs link to their sidecar entry or become stubs, and disappearing PDFs
unlink or remove their stub. All mutations carry
EntriesEventSource.SHARED, and recordWrittenFile lets the upcoming
write-back direction fingerprint its own writes so they are not
re-imported.

Late-joining observers are baselined with a silent first poll,
otherwise the running monitor reports every existing file as created
and duplicates the scanned entries.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DirectoryMonitor and DirectoryLibrarySynchronizer expose commons-io
monitor types in exported-package signatures (flagged by
checkModuleDirectivesScope).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adapts the synchronizer to the new PdfEntryFactory: externally appearing
PDFs without a sidecar now get metadata extracted from the PDF instead
of a filename-only stub, same as during the initial scan.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Externally appearing PDFs also get a generated citation key after
insertion, matching the scan behavior.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The enrichment task reuses the factory instance that also serves the
synchronizer; externally dropped PDFs keep the synchronous extraction
on the directory-sync thread.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
koppor and others added 2 commits July 17, 2026 01:45
External creation, modification, deletion, and renaming of .md files
with Hayagriva frontmatter now flows into the open directory library
like the plain .yml sidecars; the notes body maps to the entry's
comment fields.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q5Y5GTihE9rZxKrvPqnHV4
koppor and others added 8 commits September 3, 2026 03:19
Resync layer 3 of the stack (no conflicts).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DDcHNMt9fPWnpYaHheFvry
A deletion undone within the grace window no longer removes live entries, renames of sidecars with a paired PDF keep the entry instances, an editor polled mid-write only stages the entries instead of dropping them, and events arriving during shutdown are discarded instead of failing the shared monitor thread. Sidecar precedence is deterministic; the baseline snapshot runs off the caller's thread.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vr3E1Gg5DRU4LQDDVnhPys
# Conflicts:
#	jabgui/src/main/java/org/jabref/gui/importer/actions/OpenDirectoryLibraryAction.java
#	jablib/src/main/java/org/jabref/model/database/BibDatabaseContext.java
Fixes the CI format check.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DDcHNMt9fPWnpYaHheFvry
Resync with upstream (no conflicts).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DDcHNMt9fPWnpYaHheFvry
Upstream added a 0071 ADR after this stack claimed the number, so the
MADR duplicate-ID check failed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant