Skip to content

Write-back to Hayagriva sidecars (directory as library, phase 3) - #739

Open
koppor wants to merge 22 commits into
directory-inbound-syncfrom
directory-write-back
Open

koppor wants to merge 22 commits into
directory-inbound-syncfrom
directory-write-back

Conversation

@koppor

@koppor koppor commented Jul 14, 2026 •

Copy link
Copy Markdown
Member

📚 Directory-as-library stack — bottom → top, each builds on the one below:

  1. Programmatic Hayagriva writer (directory-as-library, phase 0) #736 · Hayagriva YAML writer
  2. Open folder as library (directory as library, phase 1) #737 · Open folder as library
  3. Live inbound sync for directory libraries (phase 2) #738 · Inbound file sync
  4. Write-back to Hayagriva sidecars (directory as library, phase 3) #739 · Write-back to sidecars ← this PR
  5. Groups panel mirrors the directory structure (phase 4, #10930) #740 · Directory-structure groups
  6. Pattern-driven pair renames for directory libraries (phase 5a) #741 · Pattern-driven pair renames
  7. Mirror directory libraries into a .bib with git-sync merge-back #743 · .bib mirror + merge-back
  8. Convert .bib libraries into directory libraries #744 · Convert .bib → directory
  9. Make directory libraries reachable through the REST API #745 · REST API support
  10. Split DirectoryLibrarySynchronizer into collaborators #760 · Split the synchronizer

Related issues and pull requests

Phase 3 of the "directory as library" plan (PLAN.md), stacked on #738 (base branch directory-inbound-sync; the diff also contains the programmatic Hayagriva writer commits of #736, which this phase is the first consumer of). No issue is closed by this PR.

PR Description

🤖 A directory library now saves straight into its Hayagriva sidecar files — Ctrl+S never writes a .bib. The first edit of an entry creates a sidecar beside its PDF or named after its citation key; deleting an entry removes it from its file, leaving the PDF untouched. Hand-written content JabRef does not understand survives every write, and directory tabs no longer prompt on close because nothing is left unsaved.

jabref-contrib-policy:4.2:reviewed​:ok

Analogies

Like honey sealed back into its own comb, every change flows into the cell it came from — the sidecar next to the paper. Like a chocolate mold that shapes only the squares you pour, the read-modify-write merge touches only the fields JabRef owns and leaves the rest of the bar untouched. And like the moon completing its cycle, the round trip is closed: what the importer reads, the writer returns to the very same file.

Steps to test

  1. Open a folder as library, create an entry (toolbar +): a sidecar .yml appears immediately.

  2. Type a title and a citation key: within half a second the file contains them, the YAML key follows the citation key:

    writeback2026:
      type: article
      title: Write-back works end to end
  3. Ctrl+S shows "Library saved" — no .bib file is created anywhere.

  4. Delete the entry: the sidecar disappears, PDFs stay.

New entry in a directory library, persisted to its sidecar

Guarded by seven new DirectoryLibrarySynchronizerTest cases (rewrite preserving unknown YAML, sidecar creation for stubs and new entries, key rename, disposal semantics, multi-entry rewrite, self-echo suppression) plus HayagrivaEntryWriter.mergeIntoDocument.

AI usage

Claude Code (model claude-fable-5).

AI CHECKLIST.md walkthrough

1. Code self-review

  • No == null/!= null beyond nullable JDK/Jackson returns; new members @NullMarked via their classes; Optional consumed idiomatically; no Objects.requireNonNull.
  • Only specific exceptions caught (IOException, JacksonException); none of RuntimeException/IllegalStateException thrown; exceptions last in log calls.
  • Withers for new BibEntrys in tests; records, SequencedMap, List.of(); no new threads (existing per-library executor reused); no commented-out code; Markdown Javadoc.
  • User-facing text: reuses the existing Library saved key; no new strings.
  • [/] Security: no HTML output.
  • Tests: plain JUnit asserts, @TempDir, deterministic (direct handler calls + flush(), injected clock), no sleeps.

2. Verification commands

  • ./gradlew :jablib:check — full suite; only the known environmental Remote*Test port-conflict failures on this machine (a running JabRef occupies the port; green on CI).
  • ./gradlew :jablib:checkstyleMain :jablib:checkstyleTest :jabgui:checkstyleMain.
  • ./gradlew :jablib:modernizer and ./gradlew traceRequirements (new req~directory-library.write-back~1 covered).
  • ./gradlew --no-configuration-cache :rewriteDryRun — no changes.
  • npx markdownlint-cli2 on changed Markdown.
  • [/] IntelliJ-format docker image — not runnable on this machine; CI format job is the backstop.

3. Documentation

  • CHANGELOG.md entry (TODO replaced with this PR's link after creation).
  • Requirement req~directory-library.write-back~1 added with impl markers.
  • [/] ADR — ADR-0072 already specifies this phase's design (write-back section).

4. Pull request

  • Body from the template, all sections filled, HTML comments removed, created from a body file via gh api (stacked cross-repo).
  • CHANGELOG TODO replaced after creation, committed and pushed.

Checklist

  • I own the copyright of the code submitted and I license it under the MIT license
  • If AI tools were used, I disclosed them in the "AI usage" section and reviewed, understood, and take full ownership of all AI-generated code
  • I manually tested my changes in running JabRef (created/edited/renamed/deleted entries in a directory library on the X display and inspected the sidecar files after each step; Ctrl+S produced no .bib)
  • I added JUnit tests for changes (if applicable)
  • I added screenshots in the PR description (if change is visible to the user)
  • [/] I added a screenshot in the PR description showing a library with a single entry with me as author and as title the issue number
  • I added one sentence (max 20 words) to CHANGELOG.md describing the change from the user's point of view (if the change is visible to the user)
  • I checked the user documentation for up to dateness and submitted a pull request to our user documentation repository — deferred until the feature stabilizes; tracked in PLAN.md phase 5

🤖 Generated with Claude Code

koppor and others added 4 commits July 14, 2026 04:39
The outbound half of DirectoryLibrarySynchronizer subscribes to entry
events through a CoarseChangeFilter (mirroring the shared-SQL seam) and
persists user changes read-modify-write into the Hayagriva sidecars,
debounced per file: the first edit of an entry without a sidecar
creates one (next to its PDF, sharing the base name, or named after the
citation key), a citation-key edit renames the YAML map key, and
deleting an entry removes it from its file — the file is trashed or
deleted once its last entry is gone, the paired PDF is never touched.
Own writes are fingerprinted so the inbound watcher ignores them.

Ctrl+S now flushes the pending sidecar writes instead of asking for a
.bib path; "Save as" remains the explicit .bib snapshot. Directory tabs
no longer prompt on close and drop the modification marker, and
generated citation keys are set with a non-local event source so
system-initiated changes never materialize sidecars.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
koppor and others added 3 commits July 17, 2026 01:50
# Conflicts:
#	jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizer.java
New sidecars are now created as Markdown files: the Hayagriva data as
YAML frontmatter, the comment fields as a markdownlint-clean notes body
(# Notes intro, one ## comment-<name> section per per-user comment);
body sections under foreign headings survive rewrites. Plain .yml
sidecars keep being written as pure Hayagriva.

The write debounce is now trailing-edge and re-armed by every change
event, including the keystroke events the CoarseChangeFilter marks as
filtered. Previously only the first keystroke after a field switch
scheduled a write, so the write captured a mid-typing value and the
rest of the burst was never persisted (and Ctrl+S had nothing left to
flush).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q5Y5GTihE9rZxKrvPqnHV4
koppor and others added 8 commits September 3, 2026 03:22
Resync layer 4. Conflict: upstream changed SaveDatabaseAction.save() to
return SaveResult instead of boolean — kept the directory-library
flush-instead-of-.bib early return, now returning SaveResult.SUCCESS.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DDcHNMt9fPWnpYaHheFvry
… write-back

Failed sidecar writes stay pending and are reported on save and close instead of being lost; metadata extraction for a dropped PDF no longer runs under the synchronizer's monitor; deleting an entry keeps its file until the debounced write, so an undo lands it back in the same file; an external edit that arrives before a pending write is merged field-wise instead of overwritten; writes are debounced per file, use AtomicFileOutputStream, and are drained before the final flush on shutdown.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vr3E1Gg5DRU4LQDDVnhPys
# Conflicts:
#	jablib/src/main/java/org/jabref/logic/exporter/HayagrivaEntryWriter.java
Fixes the CI format check.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DDcHNMt9fPWnpYaHheFvry
Adapt to upstream: save() gained a mayAutoCommit parameter (directory
early-return unaffected), the change-event filtered flag was renamed to
setFiltered, and the unwritable-sidecar test is disabled on Windows
(setWritable(false) is a no-op there for the owner).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DDcHNMt9fPWnpYaHheFvry
Upstream added a 0071 ADR after this stack claimed the number, so the
MADR duplicate-ID check failed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant