From 2676f6f34878b2f3752d7837438e123abe782c53 Mon Sep 17 00:00:00 2001 From: Oliver Kopp Date: Tue, 14 Jul 2026 05:00:58 +0200 Subject: [PATCH 1/8] Write user changes back into the directory library's sidecar files MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The outbound half of DirectoryLibrarySynchronizer subscribes to entry events through a CoarseChangeFilter (mirroring the shared-SQL seam) and persists user changes read-modify-write into the Hayagriva sidecars, debounced per file: the first edit of an entry without a sidecar creates one (next to its PDF, sharing the base name, or named after the citation key), a citation-key edit renames the YAML map key, and deleting an entry removes it from its file — the file is trashed or deleted once its last entry is gone, the paired PDF is never touched. Own writes are fingerprinted so the inbound watcher ignores them. Ctrl+S now flushes the pending sidecar writes instead of asking for a .bib path; "Save as" remains the explicit .bib snapshot. Directory tabs no longer prompt on close and drop the modification marker, and generated citation keys are set with a non-local event source so system-initiated changes never materialize sidecars. Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 1 + docs/requirements/directory-library.md | 15 ++ .../main/java/org/jabref/gui/LibraryTab.java | 13 +- .../gui/exporter/SaveDatabaseAction.java | 12 ++ .../actions/OpenDirectoryLibraryAction.java | 24 ++- .../DirectoryLibraryCatalog.java | 21 +++ .../DirectoryLibrarySynchronizer.java | 175 +++++++++++++++++- .../directorylibrary/PdfEntryFactory.java | 10 +- .../logic/exporter/HayagrivaEntryWriter.java | 36 ++++ .../model/database/BibDatabaseContext.java | 19 ++ .../DirectoryLibrarySynchronizerTest.java | 124 ++++++++++++- 11 files changed, 437 insertions(+), 13 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8afbebf1a8b4..0eab69bd23c9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,7 @@ Note that this project **does not** adhere to [Semantic Versioning](https://semv ### Added +- Directory libraries now save into their Hayagriva sidecar files: edits are written back automatically (debounced; Ctrl+S forces the write and no longer creates a `.bib`), the first edit of a PDF-only entry creates its sidecar, renaming a citation key renames the YAML key, and deleting an entry removes it from its file (the file is trashed once empty, the PDF stays). Hand-written YAML content that JabRef does not understand survives rewrites. [TODO] - Directory libraries now stay in sync with external file changes: creating, editing, deleting, or renaming `.yml`/`.pdf` files in the opened folder updates the open library live, and renames keep the affected entries (selection and undo history survive). [#738](https://github.com/JabRef/jabref-koppor/pull/738) - We added "Open folder as library" (File menu): JabRef opens a directory as a library, filling the main table from the Hayagriva `.yml` files and PDFs found in the folder tree. A PDF next to a `.yml` sidecar of the same name is linked to the sidecar's entry; PDFs without a sidecar appear as stub entries titled after the file. Edits are not yet written back to the files. [#737](https://github.com/JabRef/jabref-koppor/pull/737) - We added "Open folder as library" (File menu): JabRef opens a directory as a library, filling the main table from the Hayagriva `.yml` files and PDFs found in the folder tree. A PDF next to a `.yml` sidecar of the same name is linked to the sidecar's entry; PDFs without a sidecar appear as entries with metadata extracted from the PDF itself (falling back to the file name). Edits are not yet written back to the files. [#737](https://github.com/JabRef/jabref-koppor/pull/737) diff --git a/docs/requirements/directory-library.md b/docs/requirements/directory-library.md index f00dddd5f700..73a80deaeae9 100644 --- a/docs/requirements/directory-library.md +++ b/docs/requirements/directory-library.md @@ -28,6 +28,21 @@ the last-opened list and routed back through the directory-library opener. Needs: impl +## User changes are written back into the sidecar files +`req~directory-library.write-back~1` + +A directory library persists into its Hayagriva sidecar files: user edits rewrite the entry's +file read-modify-write (content JabRef does not understand survives), the first user edit of an +entry without a sidecar creates one (next to its PDF, sharing the base name, or named after the +citation key), a citation-key edit renames the YAML map key, and deleting an entry removes it +from its file — the file itself is trashed/deleted once its last entry is gone, the paired PDF +is never touched. Writes are debounced per file; Save (Ctrl+S) flushes them and must never +write a `.bib` file ("Save as" remains the explicit `.bib` snapshot). Closing needs no save +prompt. System-initiated changes (background enrichment, generated citation keys, inbound +synchronization) do not create or rewrite sidecars. + +Needs: impl + ## External file changes appear live in an open directory library `req~directory-library.inbound-sync~1` diff --git a/jabgui/src/main/java/org/jabref/gui/LibraryTab.java b/jabgui/src/main/java/org/jabref/gui/LibraryTab.java index 66ee3c6ffb6d..2900b59676cc 100644 --- a/jabgui/src/main/java/org/jabref/gui/LibraryTab.java +++ b/jabgui/src/main/java/org/jabref/gui/LibraryTab.java @@ -473,9 +473,7 @@ public void updateTabTitle(boolean isChanged) { tabTitle.append(Localization.lang("untitled")); } } else if (databaseLocation == DatabaseLocation.DIRECTORY) { - if (isChanged) { - tabTitle.append('*'); - } + // No modification marker: changes are written back to the sidecars continuously bibDatabaseContext.getDirectoryLibraryRoot().ifPresent(root -> { tabTitle.append(root.getFileName().toString()); toolTipText.append(root.toAbsolutePath()); @@ -485,8 +483,7 @@ public void updateTabTitle(boolean isChanged) { addSharedDbInformation(toolTipText, bibDatabaseContext); } addModeInfo(toolTipText, bibDatabaseContext); - if ((databaseLocation == DatabaseLocation.LOCAL || databaseLocation == DatabaseLocation.DIRECTORY) - && bibDatabaseContext.getDatabase().hasEntries()) { + if ((databaseLocation == DatabaseLocation.LOCAL) && bibDatabaseContext.getDatabase().hasEntries()) { addChangedInformation(toolTipText); } } @@ -656,9 +653,9 @@ private boolean showDeleteConfirmationDialog(int numberOfEntries) { } public boolean requestClose() { - // DIRECTORY prompts as well: until file write-back exists, edits are in-memory only - if (bibDatabaseContext.getLocation() == DatabaseLocation.LOCAL - || bibDatabaseContext.getLocation() == DatabaseLocation.DIRECTORY) { + // DIRECTORY needs no prompt: edits are persisted into the sidecar files; pending + // debounced writes are flushed by the synchronizer teardown on close + if (bibDatabaseContext.getLocation() == DatabaseLocation.LOCAL) { if (isModified()) { return confirmClose(); } diff --git a/jabgui/src/main/java/org/jabref/gui/exporter/SaveDatabaseAction.java b/jabgui/src/main/java/org/jabref/gui/exporter/SaveDatabaseAction.java index 246a66fc11d3..f1014dcf16cb 100644 --- a/jabgui/src/main/java/org/jabref/gui/exporter/SaveDatabaseAction.java +++ b/jabgui/src/main/java/org/jabref/gui/exporter/SaveDatabaseAction.java @@ -27,6 +27,7 @@ import org.jabref.gui.maintable.columns.MainTableColumn; import org.jabref.gui.preferences.GuiPreferences; import org.jabref.gui.util.FileDialogConfiguration; +import org.jabref.logic.directorylibrary.DirectoryLibrarySynchronizer; import org.jabref.logic.exporter.AtomicFileWriter; import org.jabref.logic.exporter.BibDatabaseWriter; import org.jabref.logic.exporter.BibWriter; @@ -199,6 +200,17 @@ private Optional askForSavePath() { } private boolean save(BibDatabaseContext bibDatabaseContext, SaveDatabaseMode mode) { + if (bibDatabaseContext.getLocation() == DatabaseLocation.DIRECTORY) { + // A directory library persists into its sidecar files; saving means flushing the + // debounced writes, never writing a .bib ("Save as" remains the explicit snapshot) + // [impl->req~directory-library.write-back~1] + DirectoryLibrarySynchronizer synchronizer = bibDatabaseContext.getDirectorySynchronizer(); + if (synchronizer != null) { + synchronizer.flush(); + } + dialogService.notify(Localization.lang("Library saved")); + return true; + } Optional databasePath = bibDatabaseContext.getDatabasePath(); if (databasePath.isEmpty()) { Optional savePath = askForSavePath(); diff --git a/jabgui/src/main/java/org/jabref/gui/importer/actions/OpenDirectoryLibraryAction.java b/jabgui/src/main/java/org/jabref/gui/importer/actions/OpenDirectoryLibraryAction.java index bc0523b8ea8b..17522ea49a51 100644 --- a/jabgui/src/main/java/org/jabref/gui/importer/actions/OpenDirectoryLibraryAction.java +++ b/jabgui/src/main/java/org/jabref/gui/importer/actions/OpenDirectoryLibraryAction.java @@ -1,5 +1,7 @@ package org.jabref.gui.importer.actions; +import java.io.IOException; +import java.nio.file.Files; import java.nio.file.Path; import javax.swing.undo.UndoManager; @@ -10,6 +12,7 @@ import org.jabref.gui.StateManager; import org.jabref.gui.actions.SimpleCommand; import org.jabref.gui.clipboard.ClipBoardManager; +import org.jabref.gui.desktop.os.NativeDesktop; import org.jabref.gui.preferences.GuiPreferences; import org.jabref.gui.util.DirectoryDialogConfiguration; import org.jabref.gui.util.UiTaskExecutor; @@ -27,11 +30,15 @@ import org.jabref.model.util.FileUpdateMonitor; import com.airhacks.afterburner.injection.Injector; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /// Opens a directory as a library: the main table fills from the Hayagriva `.yml` sidecars and /// `.pdf` files found in the directory tree (see [DirectoryLibraryScanner]). public class OpenDirectoryLibraryAction extends SimpleCommand { + private static final Logger LOGGER = LoggerFactory.getLogger(OpenDirectoryLibraryAction.class); + private final LibraryTabContainer tabContainer; private final DialogService dialogService; private final GuiPreferences preferences; @@ -90,6 +97,20 @@ public void openDirectory(Path root) { .executeWith(taskExecutor); } + /// Sidecar files whose last entry was deleted are trashed or deleted per the preference; + /// the paired PDF is never touched. + private void disposeFile(Path file) { + try { + if (preferences.getFilePreferences().moveToTrash() && NativeDesktop.get().moveToTrashSupported()) { + NativeDesktop.get().moveToTrash(file); + } else { + Files.delete(file); + } + } catch (IOException e) { + LOGGER.error("Could not remove sidecar {}", file, e); + } + } + private void showLibraryTab(DirectoryLibraryScanner.ScanResult scanResult) { // The synchronous factory keeps the DIRECTORY location: the ParserResult-based one // reconstructs a fresh (LOCAL) context from database + metadata on loading success @@ -114,7 +135,8 @@ private void showLibraryTab(DirectoryLibraryScanner.ScanResult scanResult) { preferences.getImportFormatPreferences(), preferences.getFilePreferences(), preferences.getCitationKeyPatternPreferences()); DirectoryLibrarySynchronizer synchronizer = new DirectoryLibrarySynchronizer( - databaseContext, scanResult.catalog(), pdfEntryFactory, UiTaskExecutor::runInJavaFXThread); + databaseContext, scanResult.catalog(), pdfEntryFactory, this::disposeFile, + UiTaskExecutor::runInJavaFXThread); databaseContext.attachDirectorySynchronizer(synchronizer); synchronizer.startWatching(Injector.instantiateModelOrService(DirectoryMonitor.class)); diff --git a/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibraryCatalog.java b/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibraryCatalog.java index fcb1f0dfc104..b05d29f4ff8d 100644 --- a/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibraryCatalog.java +++ b/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibraryCatalog.java @@ -36,6 +36,27 @@ public Optional sourceOf(BibEntry entry) { return Optional.ofNullable(sourceByEntryId.get(entry.getId())); } + public void removeEntry(BibEntry entry) { + EntrySource source = sourceByEntryId.remove(entry.getId()); + if (source != null) { + List ids = entryIdsByFile.get(source.yamlFile()); + if (ids != null) { + ids.remove(entry.getId()); + if (ids.isEmpty()) { + entryIdsByFile.remove(source.yamlFile()); + } + } + } + } + + /// Records the Hayagriva key the entry was last written under (after a citation-key edit). + public void updateHayagrivaKey(BibEntry entry, String hayagrivaKey) { + EntrySource source = sourceByEntryId.get(entry.getId()); + if (source != null) { + sourceByEntryId.put(entry.getId(), new EntrySource(source.yamlFile(), hayagrivaKey)); + } + } + /// Entry ids of all entries read from the given file, in file order. public List entryIdsIn(Path yamlFile) { return List.copyOf(entryIdsByFile.getOrDefault(yamlFile, List.of())); diff --git a/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizer.java b/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizer.java index 764a417e691a..4d74021694e2 100644 --- a/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizer.java +++ b/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizer.java @@ -6,6 +6,7 @@ import java.nio.charset.StandardCharsets; import java.nio.file.Files; import java.nio.file.Path; +import java.nio.file.StandardCopyOption; import java.security.MessageDigest; import java.security.NoSuchAlgorithmException; import java.time.Clock; @@ -13,8 +14,10 @@ import java.time.Instant; import java.util.ArrayList; import java.util.HashMap; +import java.util.HashSet; import java.util.HexFormat; import java.util.LinkedHashMap; +import java.util.LinkedHashSet; import java.util.List; import java.util.Locale; import java.util.Map; @@ -27,17 +30,23 @@ import java.util.function.Consumer; import org.jabref.logic.bibtex.FileFieldWriter; +import org.jabref.logic.exporter.HayagrivaEntryWriter; import org.jabref.logic.importer.ParserResult; import org.jabref.logic.importer.fileformat.HayagrivaImporter; import org.jabref.logic.util.DirectoryMonitor; import org.jabref.logic.util.StandardFileType; import org.jabref.logic.util.io.FileUtil; import org.jabref.model.database.BibDatabaseContext; +import org.jabref.model.database.event.EntriesAddedEvent; +import org.jabref.model.database.event.EntriesRemovedEvent; import org.jabref.model.entry.BibEntry; import org.jabref.model.entry.LinkedFile; +import org.jabref.model.entry.event.EntriesEvent; import org.jabref.model.entry.event.EntriesEventSource; +import org.jabref.model.entry.event.EntryChangedEvent; import org.jabref.model.entry.field.StandardField; +import com.google.common.eventbus.Subscribe; import org.apache.commons.io.IOCase; import org.apache.commons.io.filefilter.FileFilterUtils; import org.apache.commons.io.filefilter.IOFileFilter; @@ -64,7 +73,17 @@ /// grace period spanning two poll cycles: a create whose parsed entries equal a staged /// deletion's entries is treated as a move (the [BibEntry] instances survive, preserving /// selection and undo history); only unmatched deletions are committed. +/// +/// The outbound direction subscribes to entry events (relayed through the +/// [org.jabref.logic.util.CoarseChangeFilter] installed by +/// [BibDatabaseContext#attachDirectorySynchronizer]) and persists user changes back into the +/// sidecar files: edits rewrite the entry's file read-modify-write, the first user edit of an +/// entry without a sidecar creates one (next to its PDF, sharing the base name), a citation-key +/// edit renames the YAML map key, and deleting an entry removes it from its file (disposing the +/// file once its last entry is gone — the paired PDF is never touched). Writes are debounced +/// per file; [#flush] forces them, and shutdown flushes implicitly. // [impl->req~directory-library.inbound-sync~1] +// [impl->req~directory-library.write-back~1] @NullMarked public class DirectoryLibrarySynchronizer implements FileAlterationListener { @@ -77,6 +96,10 @@ public class DirectoryLibrarySynchronizer implements FileAlterationListener { private static final Set YAML_EXTENSIONS = Set.of("yml", "yaml"); private static final String PDF_EXTENSION = "pdf"; + /// Collects the keystroke-level bursts the CoarseChangeFilter still lets through into one + /// write per file. + private static final Duration WRITE_DEBOUNCE = Duration.ofMillis(500); + private final BibDatabaseContext databaseContext; private final DirectoryLibraryCatalog catalog; private final PdfEntryFactory pdfEntryFactory; @@ -88,6 +111,10 @@ public class DirectoryLibrarySynchronizer implements FileAlterationListener { private final Map stagedDeletions = new HashMap<>(); private final Map lastWrittenFingerprints = new HashMap<>(); + private final HayagrivaEntryWriter entryWriter = new HayagrivaEntryWriter(); + private final Set dirtyFiles = new LinkedHashSet<>(); + private final Consumer fileDisposer; + private boolean writeScheduled; private @Nullable FileAlterationObserver observer; private @Nullable DirectoryMonitor directoryMonitor; @@ -98,18 +125,21 @@ private record StagedDeletion(List entries, Instant expiry) { public DirectoryLibrarySynchronizer(BibDatabaseContext databaseContext, DirectoryLibraryCatalog catalog, PdfEntryFactory pdfEntryFactory, + Consumer fileDisposer, Consumer modelUpdateMarshaller) { - this(databaseContext, catalog, pdfEntryFactory, modelUpdateMarshaller, Clock.systemUTC()); + this(databaseContext, catalog, pdfEntryFactory, fileDisposer, modelUpdateMarshaller, Clock.systemUTC()); } DirectoryLibrarySynchronizer(BibDatabaseContext databaseContext, DirectoryLibraryCatalog catalog, PdfEntryFactory pdfEntryFactory, + Consumer fileDisposer, Consumer modelUpdateMarshaller, Clock clock) { this.databaseContext = databaseContext; this.catalog = catalog; this.pdfEntryFactory = pdfEntryFactory; + this.fileDisposer = fileDisposer; this.root = databaseContext.getDirectoryLibraryRoot().orElseThrow( () -> new IllegalArgumentException("Context is not a directory library")); this.modelUpdateMarshaller = modelUpdateMarshaller; @@ -144,9 +174,15 @@ public void shutdown() { if (observer != null && directoryMonitor != null) { directoryMonitor.removeObserver(observer); } + flush(); syncExecutor.shutdown(); } + /// Writes all pending sidecar changes now (they are otherwise debounced). + public synchronized void flush() { + writeDirtyFiles(); + } + /// Registers the fingerprint of a file this application just wrote itself, so the next /// change event for it is recognized as a self-echo and not re-imported. Consumed on match. public synchronized void recordWrittenFile(Path file, byte[] content) { @@ -193,6 +229,143 @@ public void onStop(FileAlterationObserver observer) { syncExecutor.execute(this::commitExpiredStagedDeletions); } + @Subscribe + public void listen(EntryChangedEvent event) { + if (event.isFilteredOut() || !isUserChange(event)) { + return; + } + BibEntry entry = event.getBibEntry(); + syncExecutor.execute(() -> handleLocalChange(entry)); + } + + @Subscribe + public void listen(EntriesAddedEvent event) { + if (!isUserChange(event)) { + return; + } + List entries = List.copyOf(event.getBibEntries()); + syncExecutor.execute(() -> entries.forEach(this::handleLocalChange)); + } + + @Subscribe + public void listen(EntriesRemovedEvent event) { + if (!isUserChange(event)) { + return; + } + List entries = List.copyOf(event.getBibEntries()); + syncExecutor.execute(() -> handleLocalRemoval(entries)); + } + + private static boolean isUserChange(EntriesEvent event) { + return event.getEntriesEventSource() == EntriesEventSource.LOCAL + || event.getEntriesEventSource() == EntriesEventSource.UNDO; + } + + synchronized void handleLocalChange(BibEntry entry) { + Path file = catalog.sourceOf(entry) + .map(DirectoryLibraryCatalog.EntrySource::yamlFile) + .orElseGet(() -> assignSidecar(entry)); + dirtyFiles.add(file); + scheduleWrite(); + } + + synchronized void handleLocalRemoval(List entries) { + Set affectedFiles = new LinkedHashSet<>(); + for (BibEntry entry : entries) { + catalog.sourceOf(entry).ifPresent(source -> { + affectedFiles.add(source.yamlFile()); + catalog.removeEntry(entry); + }); + } + for (Path file : affectedFiles) { + if (catalog.entryIdsIn(file).isEmpty()) { + dirtyFiles.remove(file); + if (Files.exists(file)) { + fileDisposer.accept(file); + } + } else { + dirtyFiles.add(file); + scheduleWrite(); + } + } + } + + /// The first user change of an entry without a source materializes its sidecar: next to the + /// entry's PDF (sharing the base name, per the pairing convention), or named after the + /// citation key for entries without a file. + private Path assignSidecar(BibEntry entry) { + Optional pairedFile = entry.getFiles().stream() + .filter(linkedFile -> !linkedFile.isOnlineLink()) + .findFirst() + .map(linkedFile -> root.resolve(linkedFile.getLink()).normalize()); + Path sidecar; + if (pairedFile.isPresent() && pairedFile.get().startsWith(root)) { + Path parent = pairedFile.get().getParent(); + sidecar = parent.resolve(FileUtil.getBaseName(pairedFile.get()) + ".yml"); + } else { + String baseName = entry.getCitationKey().filter(key -> !key.isBlank()).orElse("entry"); + sidecar = root.resolve(baseName + ".yml"); + int counter = 1; + while (Files.exists(sidecar)) { + sidecar = root.resolve(baseName + "-" + counter++ + ".yml"); + } + } + catalog.register(entry, sidecar, entry.getCitationKey().orElse("")); + return sidecar; + } + + private synchronized void scheduleWrite() { + if (writeScheduled) { + return; + } + writeScheduled = true; + syncExecutor.schedule(this::writeDirtyFiles, WRITE_DEBOUNCE.toMillis(), TimeUnit.MILLISECONDS); + } + + private synchronized void writeDirtyFiles() { + writeScheduled = false; + List files = List.copyOf(dirtyFiles); + dirtyFiles.clear(); + files.forEach(this::writeFile); + } + + private void writeFile(Path file) { + List entries = entriesOf(file); + if (entries.isEmpty()) { + return; + } + List keyedEntries = new ArrayList<>(); + Set usedKeys = new HashSet<>(); + for (BibEntry entry : entries) { + String previousKey = catalog.sourceOf(entry) + .map(DirectoryLibraryCatalog.EntrySource::hayagrivaKey) + .orElse(""); + String targetKey = entry.getCitationKey() + .filter(key -> !key.isBlank()) + .orElse(previousKey.isBlank() ? "entry" : previousKey); + String uniqueKey = targetKey; + int counter = 1; + while (!usedKeys.add(uniqueKey)) { + uniqueKey = targetKey + "-" + counter++; + } + keyedEntries.add(new HayagrivaEntryWriter.KeyedEntry(previousKey, uniqueKey, entry)); + } + try { + String existingDocument = Files.exists(file) ? Files.readString(file, StandardCharsets.UTF_8) : null; + String document = entryWriter.mergeIntoDocument(existingDocument, keyedEntries); + byte[] content = document.getBytes(StandardCharsets.UTF_8); + recordWrittenFile(file, content); + // Written atomically: the polling watcher (or another process) must never see a + // half-written sidecar + Path temporary = file.resolveSibling(file.getFileName() + ".jabref-write.tmp"); + Files.write(temporary, content); + Files.move(temporary, file, StandardCopyOption.REPLACE_EXISTING, StandardCopyOption.ATOMIC_MOVE); + keyedEntries.forEach(keyedEntry -> catalog.updateHayagrivaKey(keyedEntry.entry(), keyedEntry.targetKey())); + } catch (IOException e) { + LOGGER.error("Could not write sidecar {}", file, e); + } + } + synchronized void handleFileCreated(Path file) { commitExpiredStagedDeletions(); if (isYaml(file)) { diff --git a/jablib/src/main/java/org/jabref/logic/directorylibrary/PdfEntryFactory.java b/jablib/src/main/java/org/jabref/logic/directorylibrary/PdfEntryFactory.java index 391456b39d1a..7db0f1067b86 100644 --- a/jablib/src/main/java/org/jabref/logic/directorylibrary/PdfEntryFactory.java +++ b/jablib/src/main/java/org/jabref/logic/directorylibrary/PdfEntryFactory.java @@ -20,6 +20,7 @@ import org.jabref.model.entry.BibEntry; import org.jabref.model.entry.LinkedFile; import org.jabref.model.entry.event.EntriesEventSource; +import org.jabref.model.entry.field.InternalField; import org.jabref.model.entry.field.StandardField; import org.jabref.model.entry.identifier.DOI; import org.jabref.model.entry.types.StandardEntryType; @@ -65,12 +66,17 @@ public PdfEntryFactory(ImportFormatPreferences importFormatPreferences, } /// Generates a citation key for the entry if it has none. Call after the entry has been - /// inserted into the database, so the uniqueness check sees the whole library. + /// inserted into the database, so the uniqueness check sees the whole library. The key is + /// set with [EntriesEventSource#SHARED]: it is system-initiated and must not count as a + /// user edit (a user edit is what materializes a sidecar). public void generateCitationKeyIfMissing(BibEntry entry, BibDatabaseContext databaseContext) { if (entry.getCitationKey().isPresent()) { return; } - new CitationKeyGenerator(databaseContext, citationKeyPatternPreferences).generateAndSetKey(entry); + String key = new CitationKeyGenerator(databaseContext, citationKeyPatternPreferences).generateKey(entry); + if (!key.isBlank()) { + entry.setField(InternalField.KEY_FIELD, key, EntriesEventSource.SHARED); + } } public BibEntry createEntry(Path pdf, Path root, BibDatabaseContext databaseContext) { diff --git a/jablib/src/main/java/org/jabref/logic/exporter/HayagrivaEntryWriter.java b/jablib/src/main/java/org/jabref/logic/exporter/HayagrivaEntryWriter.java index f69cf7f7c552..fee39dfd81d8 100644 --- a/jablib/src/main/java/org/jabref/logic/exporter/HayagrivaEntryWriter.java +++ b/jablib/src/main/java/org/jabref/logic/exporter/HayagrivaEntryWriter.java @@ -20,6 +20,7 @@ import org.jspecify.annotations.NullMarked; import org.jspecify.annotations.Nullable; +import tools.jackson.core.JacksonException; import tools.jackson.databind.JsonNode; import tools.jackson.databind.node.ArrayNode; import tools.jackson.databind.node.ObjectNode; @@ -49,6 +50,41 @@ public class HayagrivaEntryWriter { .enable(YAMLWriteFeature.MINIMIZE_QUOTES) .build()); + /// One entry of a directory-library sidecar during write-back: `previousKey` locates the + /// entry's existing YAML node (empty or unknown for new entries), `targetKey` is the key to + /// write (differs from `previousKey` after a citation-key edit). + public record KeyedEntry(String previousKey, String targetKey, BibEntry entry) { + } + + /// Merges the given entries into an existing Hayagriva document (read-modify-write, see + /// class doc). The result contains exactly the given entries in the given order — top-level + /// keys without a corresponding entry are dropped, because their entries no longer belong + /// to this file — while inside each kept entry everything JabRef does not own survives. + public String mergeIntoDocument(@Nullable String existingDocument, List entries) { + ObjectNode existingRoot; + if (existingDocument == null || existingDocument.isBlank()) { + existingRoot = MAPPER.createObjectNode(); + } else { + JsonNode parsed; + try { + parsed = MAPPER.readTree(existingDocument); + } catch (JacksonException e) { + // Write-back only touches files it imported, so this is unexpected; rebuilding + // from scratch keeps the user's current entries authoritative + parsed = null; + } + existingRoot = parsed instanceof ObjectNode objectNode ? objectNode : MAPPER.createObjectNode(); + } + ObjectNode result = MAPPER.createObjectNode(); + for (KeyedEntry keyedEntry : entries) { + ObjectNode entryNode = existingRoot.get(keyedEntry.previousKey()) instanceof ObjectNode existing + ? existing + : MAPPER.createObjectNode(); + result.set(keyedEntry.targetKey(), mergeIntoNode(keyedEntry.entry(), entryNode)); + } + return MAPPER.writeValueAsString(result); + } + public String serialize(SequencedMap keyedEntries) { ObjectNode root = MAPPER.createObjectNode(); keyedEntries.forEach((citationKey, entry) -> root.set(citationKey, toEntryNode(entry))); diff --git a/jablib/src/main/java/org/jabref/model/database/BibDatabaseContext.java b/jablib/src/main/java/org/jabref/model/database/BibDatabaseContext.java index a64a61e06acb..75b2671b2497 100644 --- a/jablib/src/main/java/org/jabref/model/database/BibDatabaseContext.java +++ b/jablib/src/main/java/org/jabref/model/database/BibDatabaseContext.java @@ -77,6 +77,9 @@ public class BibDatabaseContext { @Nullable private DirectoryLibrarySynchronizer directorySynchronizer; + @Nullable + private CoarseChangeFilter directoryListener; + private DatabaseLocation location; public BibDatabaseContext() { @@ -289,6 +292,10 @@ public Optional getDirectoryLibraryRoot() { public void attachDirectorySynchronizer(DirectoryLibrarySynchronizer directorySynchronizer) { this.directorySynchronizer = directorySynchronizer; + // Relays entry events keystroke-filtered to the synchronizer's write-back direction, + // mirroring convertToSharedDatabase + this.directoryListener = new CoarseChangeFilter(this); + directoryListener.registerListener(directorySynchronizer); } public @Nullable DirectoryLibrarySynchronizer getDirectorySynchronizer() { @@ -302,6 +309,18 @@ public void convertToLocalDatabase() { } dbmsListener.shutdown(); } + if (directoryListener != null) { + if (directorySynchronizer != null) { + directoryListener.unregisterListener(directorySynchronizer); + } + directoryListener.shutdown(); + this.directoryListener = null; + } + if (directorySynchronizer != null) { + // Flushes pending sidecar writes and stops the directory watcher + directorySynchronizer.shutdown(); + this.directorySynchronizer = null; + } if (directorySynchronizer != null) { directorySynchronizer.shutdown(); directorySynchronizer = null; diff --git a/jablib/src/test/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizerTest.java b/jablib/src/test/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizerTest.java index 9f9473b68962..beb6eb39a7c2 100644 --- a/jablib/src/test/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizerTest.java +++ b/jablib/src/test/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizerTest.java @@ -9,6 +9,7 @@ import java.time.Instant; import java.time.ZoneId; import java.time.ZoneOffset; +import java.util.ArrayList; import java.util.List; import java.util.Optional; @@ -28,6 +29,7 @@ import org.mockito.Answers; import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; import static org.junit.jupiter.api.Assertions.assertSame; import static org.junit.jupiter.api.Assertions.assertTrue; import static org.mockito.Mockito.mock; @@ -72,6 +74,8 @@ public Clock withZone(ZoneId zone) { private final SteppingClock clock = new SteppingClock(); + private final List disposedFiles = new ArrayList<>(); + private BibDatabaseContext context; private DirectoryLibrarySynchronizer synchronizer; @@ -79,7 +83,8 @@ private void openLibrary() throws IOException { PdfEntryFactory pdfEntryFactory = offlinePdfEntryFactory(); DirectoryLibraryScanner.ScanResult scanResult = new DirectoryLibraryScanner(pdfEntryFactory).scan(root); context = scanResult.databaseContext(); - synchronizer = new DirectoryLibrarySynchronizer(context, scanResult.catalog(), pdfEntryFactory, Runnable::run, clock); + synchronizer = new DirectoryLibrarySynchronizer(context, scanResult.catalog(), pdfEntryFactory, + disposedFiles::add, Runnable::run, clock); } /// GROBID off and no identifiers in the fixtures, so no network is touched @@ -264,4 +269,121 @@ void deletedPdfRemovesStubButKeepsSidecarEntry() throws IOException { assertEquals(1, entries().size()); assertTrue(entries().getFirst().getFiles().isEmpty()); } + + @Test + void localEditRewritesSidecarPreservingUnknownContent() throws IOException { + Path sidecar = root.resolve("smith2020.yml"); + Files.writeString(sidecar, ARTICLE_YAML + " tongus: 2\n"); + openLibrary(); + BibEntry entry = entries().getFirst(); + + entry.setField(StandardField.NOTE, "rewritten by JabRef"); + synchronizer.handleLocalChange(entry); + synchronizer.flush(); + + String written = Files.readString(sidecar); + assertTrue(written.contains("rewritten by JabRef")); + assertTrue(written.contains("tongus")); + } + + @Test + void firstEditOfStubEntryCreatesSidecarNextToPdf() throws IOException { + Files.createFile(root.resolve("loose.pdf")); + openLibrary(); + BibEntry stub = entries().getFirst(); + + stub.setField(StandardField.AUTHOR, "Doe, John"); + synchronizer.handleLocalChange(stub); + synchronizer.flush(); + + Path sidecar = root.resolve("loose.yml"); + assertTrue(Files.exists(sidecar)); + assertTrue(Files.readString(sidecar).contains("Doe, John")); + } + + @Test + void newEntryWithoutFileGetsCitationKeyNamedSidecar() throws IOException { + openLibrary(); + BibEntry entry = new BibEntry(org.jabref.model.entry.types.StandardEntryType.Article) + .withCitationKey("fresh2026") + .withField(StandardField.TITLE, "Fresh Entry"); + context.getDatabase().insertEntry(entry); + + synchronizer.handleLocalChange(entry); + synchronizer.flush(); + + assertTrue(Files.exists(root.resolve("fresh2026.yml"))); + } + + @Test + void citationKeyEditRenamesYamlKey() throws IOException { + Path sidecar = root.resolve("smith2020.yml"); + Files.writeString(sidecar, ARTICLE_YAML); + openLibrary(); + BibEntry entry = entries().getFirst(); + + entry.setCitationKey("smith2021"); + synchronizer.handleLocalChange(entry); + synchronizer.flush(); + + String written = Files.readString(sidecar); + assertTrue(written.contains("smith2021:")); + assertFalse(written.contains("smith2020:")); + } + + @Test + void deletingLastEntryDisposesSidecarOnly() throws IOException { + Path sidecar = root.resolve("smith2020.yml"); + Files.writeString(sidecar, ARTICLE_YAML); + Files.createFile(root.resolve("smith2020.pdf")); + openLibrary(); + BibEntry entry = entries().getFirst(); + + context.getDatabase().removeEntries(List.of(entry)); + synchronizer.handleLocalRemoval(List.of(entry)); + synchronizer.flush(); + + assertEquals(List.of(sidecar), disposedFiles); + assertTrue(Files.exists(root.resolve("smith2020.pdf"))); + } + + @Test + void deletingOneEntryOfMultiEntryFileRewritesRemainder() throws IOException { + Path file = root.resolve("collection.yml"); + Files.writeString(file, """ + first: + type: article + title: First + second: + type: article + title: Second + """); + openLibrary(); + BibEntry first = entries().getFirst(); + + context.getDatabase().removeEntries(List.of(first)); + synchronizer.handleLocalRemoval(List.of(first)); + synchronizer.flush(); + + String written = Files.readString(file); + assertFalse(written.contains("first:")); + assertTrue(written.contains("second:")); + assertEquals(List.of(), disposedFiles); + } + + @Test + void ownSidecarWritesAreNotReimported() throws IOException { + Path sidecar = root.resolve("smith2020.yml"); + Files.writeString(sidecar, ARTICLE_YAML); + openLibrary(); + BibEntry entry = entries().getFirst(); + + entry.setField(StandardField.NOTE, "written back"); + synchronizer.handleLocalChange(entry); + synchronizer.flush(); + synchronizer.handleFileChanged(sidecar); + + assertEquals(1, entries().size()); + assertEquals(Optional.of("written back"), entry.getField(StandardField.NOTE)); + } } From a09bc85b345abe38215524d60f709ae5f9c88cb1 Mon Sep 17 00:00:00 2001 From: Oliver Kopp Date: Tue, 14 Jul 2026 05:02:23 +0200 Subject: [PATCH 2/8] Link CHANGELOG entry to its pull request Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0eab69bd23c9..ea2047d7f757 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,7 +11,7 @@ Note that this project **does not** adhere to [Semantic Versioning](https://semv ### Added -- Directory libraries now save into their Hayagriva sidecar files: edits are written back automatically (debounced; Ctrl+S forces the write and no longer creates a `.bib`), the first edit of a PDF-only entry creates its sidecar, renaming a citation key renames the YAML key, and deleting an entry removes it from its file (the file is trashed once empty, the PDF stays). Hand-written YAML content that JabRef does not understand survives rewrites. [TODO] +- Directory libraries now save into their Hayagriva sidecar files: edits are written back automatically (debounced; Ctrl+S forces the write and no longer creates a `.bib`), the first edit of a PDF-only entry creates its sidecar, renaming a citation key renames the YAML key, and deleting an entry removes it from its file (the file is trashed once empty, the PDF stays). Hand-written YAML content that JabRef does not understand survives rewrites. [#739](https://github.com/JabRef/jabref-koppor/pull/739) - Directory libraries now stay in sync with external file changes: creating, editing, deleting, or renaming `.yml`/`.pdf` files in the opened folder updates the open library live, and renames keep the affected entries (selection and undo history survive). [#738](https://github.com/JabRef/jabref-koppor/pull/738) - We added "Open folder as library" (File menu): JabRef opens a directory as a library, filling the main table from the Hayagriva `.yml` files and PDFs found in the folder tree. A PDF next to a `.yml` sidecar of the same name is linked to the sidecar's entry; PDFs without a sidecar appear as stub entries titled after the file. Edits are not yet written back to the files. [#737](https://github.com/JabRef/jabref-koppor/pull/737) - We added "Open folder as library" (File menu): JabRef opens a directory as a library, filling the main table from the Hayagriva `.yml` files and PDFs found in the folder tree. A PDF next to a `.yml` sidecar of the same name is linked to the sidecar's entry; PDFs without a sidecar appear as entries with metadata extracted from the PDF itself (falling back to the file name). Edits are not yet written back to the files. [#737](https://github.com/JabRef/jabref-koppor/pull/737) From b385dcc7c187ed64579f749226f7ba5531513732 Mon Sep 17 00:00:00 2001 From: Oliver Kopp Date: Tue, 14 Jul 2026 05:07:57 +0200 Subject: [PATCH 3/8] Align ternary continuation to the IntelliJ code style Co-Authored-By: Claude Fable 5 --- .../java/org/jabref/logic/exporter/HayagrivaEntryWriter.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/jablib/src/main/java/org/jabref/logic/exporter/HayagrivaEntryWriter.java b/jablib/src/main/java/org/jabref/logic/exporter/HayagrivaEntryWriter.java index fee39dfd81d8..3ab518a80d9f 100644 --- a/jablib/src/main/java/org/jabref/logic/exporter/HayagrivaEntryWriter.java +++ b/jablib/src/main/java/org/jabref/logic/exporter/HayagrivaEntryWriter.java @@ -78,8 +78,8 @@ public String mergeIntoDocument(@Nullable String existingDocument, List Date: Fri, 17 Jul 2026 01:59:07 +0200 Subject: [PATCH 4/8] Write Markdown sidecars and never strand a typing burst's tail New sidecars are now created as Markdown files: the Hayagriva data as YAML frontmatter, the comment fields as a markdownlint-clean notes body (# Notes intro, one ## comment- section per per-user comment); body sections under foreign headings survive rewrites. Plain .yml sidecars keep being written as pure Hayagriva. The write debounce is now trailing-edge and re-armed by every change event, including the keystroke events the CoarseChangeFilter marks as filtered. Previously only the first keystroke after a field switch scheduled a write, so the write captured a mid-typing value and the rest of the burst was never persisted (and Ctrl+S had nothing left to flush). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Q5Y5GTihE9rZxKrvPqnHV4 --- CHANGELOG.md | 10 +- docs/requirements/directory-library.md | 19 ++- .../DirectoryLibrarySynchronizer.java | 40 +++-- .../directorylibrary/MarkdownSidecar.java | 146 +++++++++++++++--- .../DirectoryLibrarySynchronizerTest.java | 77 ++++++++- .../directorylibrary/MarkdownSidecarTest.java | 24 +++ 6 files changed, 259 insertions(+), 57 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 14ddeb59e1fd..1d04e6d7844e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,15 +11,9 @@ Note that this project **does not** adhere to [Semantic Versioning](https://semv ### Added -- Directory libraries now save into their Hayagriva sidecar files: edits are written back automatically (debounced; Ctrl+S forces the write and no longer creates a `.bib`), the first edit of a PDF-only entry creates its sidecar, renaming a citation key renames the YAML key, and deleting an entry removes it from its file (the file is trashed once empty, the PDF stays). Hand-written YAML content that JabRef does not understand survives rewrites. [#739](https://github.com/JabRef/jabref-koppor/pull/739) -- Directory libraries now stay in sync with external file changes: creating, editing, deleting, or renaming `.yml`/`.pdf` files in the opened folder updates the open library live, and renames keep the affected entries (selection and undo history survive). [#738](https://github.com/JabRef/jabref-koppor/pull/738) +- Directory libraries now save into their sidecar files: edits are written back automatically (debounced until typing pauses; Ctrl+S forces the write and no longer creates a `.bib`), the first edit of a PDF-only entry creates a Markdown sidecar (`X.md` with the Hayagriva data as frontmatter and the comment fields as notes body), renaming a citation key renames the YAML key, and deleting an entry removes it from its file (the file is trashed once empty, the PDF stays). Hand-written content that JabRef does not understand survives rewrites. [#739](https://github.com/JabRef/jabref-koppor/pull/739) - Directory libraries now stay in sync with external file changes: creating, editing, deleting, or renaming `.yml`/`.md`/`.pdf` files in the opened folder updates the open library live, and renames keep the affected entries (selection and undo history survive). [#738](https://github.com/JabRef/jabref-koppor/pull/738) -- We added "Open folder as library" (File menu): JabRef opens a directory as a library, filling the main table from the Hayagriva `.yml` files and PDFs found in the folder tree. A PDF next to a `.yml` sidecar of the same name is linked to the sidecar's entry; PDFs without a sidecar appear as stub entries titled after the file. Edits are not yet written back to the files. [#737](https://github.com/JabRef/jabref-koppor/pull/737) -- We added "Open folder as library" (File menu): JabRef opens a directory as a library, filling the main table from the Hayagriva `.yml` files and PDFs found in the folder tree. A PDF next to a `.yml` sidecar of the same name is linked to the sidecar's entry; PDFs without a sidecar appear as entries with metadata extracted from the PDF itself (falling back to the file name). Edits are not yet written back to the files. [#737](https://github.com/JabRef/jabref-koppor/pull/737) -- We added "Open folder as library" (File menu): JabRef opens a directory as a library, filling the main table from the Hayagriva `.yml` files and PDFs found in the folder tree. A PDF next to a `.yml` sidecar of the same name is linked to the sidecar's entry; PDFs without a sidecar appear immediately and are enriched in the background with metadata extracted from the PDF itself and a generated citation key. Edits are not yet written back to the files. [#737](https://github.com/JabRef/jabref-koppor/pull/737) -- We added "Open folder as library" (File menu): JabRef opens a directory as a library, filling the main table from the Hayagriva `.yml` files and PDFs found in the folder tree. A PDF next to a `.yml` sidecar of the same name is linked to the sidecar's entry; PDFs without a sidecar appear immediately and are enriched in the background with metadata extracted from the PDF itself and a generated citation key; a missing DOI is looked up online and its metadata fills the remaining empty fields. Edits are not yet written back to the files. [#737](https://github.com/JabRef/jabref-koppor/pull/737) -- We added "Open folder as library" (File menu): JabRef opens a directory as a library, filling the main table from the Hayagriva `.yml` files and PDFs found in the folder tree. A PDF next to a `.yml` sidecar of the same name is linked to the sidecar's entry; PDFs without a sidecar appear immediately and are enriched in the background with metadata extracted from the PDF itself and a generated citation key; a missing DOI is looked up online and its metadata fills the remaining empty fields. Directory libraries that were open on shutdown are reopened on the next start. Edits are not yet written back to the files. [#737](https://github.com/JabRef/jabref-koppor/pull/737) -- We added "Open folder as library" (File menu): JabRef opens a directory as a library, filling the main table from the Hayagriva `.yml` files, the Markdown notes files with a Hayagriva YAML frontmatter (`.md`; the notes body maps to the entry's comment fields), and the PDFs found in the folder tree. A PDF next to a sidecar of the same name is linked to the sidecar's entry; PDFs without a sidecar appear immediately and are enriched in the background with metadata extracted from the PDF itself and a generated citation key; a missing DOI is looked up online and its metadata fills the remaining empty fields. Directory libraries that were open on shutdown are reopened on the next start. Edits are not yet written back to the files. [#737](https://github.com/JabRef/jabref-koppor/pull/737) +- We added "Open folder as library" (File menu): JabRef opens a directory as a library, filling the main table from the Hayagriva `.yml` files, the Markdown notes files with a Hayagriva YAML frontmatter (`.md`; the notes body maps to the entry's comment fields), and the PDFs found in the folder tree. A PDF next to a sidecar of the same name is linked to the sidecar's entry; PDFs without a sidecar appear immediately and are enriched in the background with metadata extracted from the PDF itself and a generated citation key; a missing DOI is looked up online and its metadata fills the remaining empty fields. Directory libraries that were open on shutdown are reopened on the next start. [#737](https://github.com/JabRef/jabref-koppor/pull/737) - The HTTP import endpoint (`POST /libraries/{id}/entries`) now accepts CSL-JSON (`application/vnd.citationstyles.csl+json`), mapping each item to the correct entry type (e.g. conference paper, book chapter, thesis) via the citation-js-based mapping. [#16151](https://github.com/JabRef/jabref/pull/16151) - We added a new "Main" tab to the entry editor showing all fields of an entry in a single scrollable list, with one-click chips for adding optional fields and a free-form box for adding arbitrary fields. Identifiers, files and links, bibliometrics, comments, and meta fields (groups, owner, timestamps, special fields) live in collapsible sections — collapsed when empty — each offering chips for its unset fields. [#12711](https://github.com/JabRef/jabref/issues/12711) - We added auto-detection import for drag-and-dropped library files. [#15391](https://github.com/JabRef/jabref/issues/15391) diff --git a/docs/requirements/directory-library.md b/docs/requirements/directory-library.md index cd46e379b02f..9498c613273c 100644 --- a/docs/requirements/directory-library.md +++ b/docs/requirements/directory-library.md @@ -33,14 +33,21 @@ the last-opened list and routed back through the directory-library opener. Needs: impl ## User changes are written back into the sidecar files -`req~directory-library.write-back~1` +`req~directory-library.write-back~2` -A directory library persists into its Hayagriva sidecar files: user edits rewrite the entry's -file read-modify-write (content JabRef does not understand survives), the first user edit of an -entry without a sidecar creates one (next to its PDF, sharing the base name, or named after the -citation key), a citation-key edit renames the YAML map key, and deleting an entry removes it +A directory library persists into its sidecar files: user edits rewrite the entry's +file read-modify-write (content JabRef does not understand survives, including body sections of +Markdown sidecars under foreign headings), the first user edit of an +entry without a sidecar creates a Markdown sidecar `X.md` (next to its PDF, sharing the base +name, or named after the citation key) whose frontmatter carries the Hayagriva data and whose +markdownlint-clean body carries the comment fields (`# Notes` intro for the comment, one +`## comment-` section per per-user comment); in plain `.yml` sidecars the comment fields +are written as extension keys. A citation-key edit renames the YAML map key, and deleting an +entry removes it from its file — the file itself is trashed/deleted once its last entry is gone, the paired PDF -is never touched. Writes are debounced per file; Save (Ctrl+S) flushes them and must never +is never touched. Writes are debounced per file with a trailing-edge debounce that is re-armed +by every change event — including the keystroke events the CoarseChangeFilter marks as +filtered, so the tail of a typing burst is never lost; Save (Ctrl+S) flushes them and must never write a `.bib` file ("Save as" remains the explicit `.bib` snapshot). Closing needs no save prompt. System-initiated changes (background enrichment, generated citation keys, inbound synchronization) do not create or rewrite sidecars. diff --git a/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizer.java b/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizer.java index f4dce757131e..fc3ecf1b1a1d 100644 --- a/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizer.java +++ b/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizer.java @@ -26,6 +26,7 @@ import java.util.Set; import java.util.concurrent.Executors; import java.util.concurrent.ScheduledExecutorService; +import java.util.concurrent.ScheduledFuture; import java.util.concurrent.TimeUnit; import java.util.function.Consumer; @@ -86,7 +87,7 @@ /// file once its last entry is gone — the paired PDF is never touched). Writes are debounced /// per file; [#flush] forces them, and shutdown flushes implicitly. // [impl->req~directory-library.inbound-sync~2] -// [impl->req~directory-library.write-back~1] +// [impl->req~directory-library.write-back~2] @NullMarked public class DirectoryLibrarySynchronizer implements FileAlterationListener { @@ -99,8 +100,9 @@ public class DirectoryLibrarySynchronizer implements FileAlterationListener { private static final Set SIDECAR_EXTENSIONS = Set.of("yml", "yaml", MarkdownSidecar.MARKDOWN_EXTENSION); private static final String PDF_EXTENSION = "pdf"; - /// Collects the keystroke-level bursts the CoarseChangeFilter still lets through into one - /// write per file. + /// Collects keystroke-level bursts into one write per file. Trailing edge: every change + /// event re-arms the timer, so the write fires once typing pauses and always persists the + /// latest state. private static final Duration WRITE_DEBOUNCE = Duration.ofMillis(500); private final BibDatabaseContext databaseContext; @@ -118,7 +120,7 @@ public class DirectoryLibrarySynchronizer implements FileAlterationListener { private final HayagrivaEntryWriter entryWriter = new HayagrivaEntryWriter(); private final Set dirtyFiles = new LinkedHashSet<>(); private final Consumer fileDisposer; - private boolean writeScheduled; + private @Nullable ScheduledFuture scheduledWrite; private @Nullable FileAlterationObserver observer; private @Nullable DirectoryMonitor directoryMonitor; @@ -236,9 +238,12 @@ public void onStop(FileAlterationObserver observer) { @Subscribe public void listen(EntryChangedEvent event) { - if (event.isFilteredOut() || !isUserChange(event)) { + if (!isUserChange(event)) { return; } + // Events the CoarseChangeFilter marks as filtered (the keystrokes of a typing burst) + // still re-arm the debounce: the write captures the entry's state at fire time, so the + // tail of a burst — which produces only filtered events — is never lost. BibEntry entry = event.getBibEntry(); syncExecutor.execute(() -> handleLocalChange(entry)); } @@ -295,9 +300,9 @@ synchronized void handleLocalRemoval(List entries) { } } - /// The first user change of an entry without a source materializes its sidecar: next to the - /// entry's PDF (sharing the base name, per the pairing convention), or named after the - /// citation key for entries without a file. + /// The first user change of an entry without a source materializes its sidecar — a Markdown + /// sidecar (see [MarkdownSidecar]): next to the entry's PDF (sharing the base name, per the + /// pairing convention), or named after the citation key for entries without a file. private Path assignSidecar(BibEntry entry) { Optional pairedFile = entry.getFiles().stream() .filter(linkedFile -> !linkedFile.isOnlineLink()) @@ -306,13 +311,13 @@ private Path assignSidecar(BibEntry entry) { Path sidecar; if (pairedFile.isPresent() && pairedFile.get().startsWith(root)) { Path parent = pairedFile.get().getParent(); - sidecar = parent.resolve(FileUtil.getBaseName(pairedFile.get()) + ".yml"); + sidecar = parent.resolve(FileUtil.getBaseName(pairedFile.get()) + "." + MarkdownSidecar.MARKDOWN_EXTENSION); } else { String baseName = entry.getCitationKey().filter(key -> !key.isBlank()).orElse("entry"); - sidecar = root.resolve(baseName + ".yml"); + sidecar = root.resolve(baseName + "." + MarkdownSidecar.MARKDOWN_EXTENSION); int counter = 1; while (Files.exists(sidecar)) { - sidecar = root.resolve(baseName + "-" + counter++ + ".yml"); + sidecar = root.resolve(baseName + "-" + counter++ + "." + MarkdownSidecar.MARKDOWN_EXTENSION); } } catalog.register(entry, sidecar, entry.getCitationKey().orElse("")); @@ -320,15 +325,14 @@ private Path assignSidecar(BibEntry entry) { } private synchronized void scheduleWrite() { - if (writeScheduled) { - return; + if (scheduledWrite != null) { + scheduledWrite.cancel(false); } - writeScheduled = true; - syncExecutor.schedule(this::writeDirtyFiles, WRITE_DEBOUNCE.toMillis(), TimeUnit.MILLISECONDS); + scheduledWrite = syncExecutor.schedule(this::writeDirtyFiles, WRITE_DEBOUNCE.toMillis(), TimeUnit.MILLISECONDS); } private synchronized void writeDirtyFiles() { - writeScheduled = false; + scheduledWrite = null; List files = List.copyOf(dirtyFiles); dirtyFiles.clear(); files.forEach(this::writeFile); @@ -357,7 +361,9 @@ private void writeFile(Path file) { } try { String existingDocument = Files.exists(file) ? Files.readString(file, StandardCharsets.UTF_8) : null; - String document = entryWriter.mergeIntoDocument(existingDocument, keyedEntries); + String document = MarkdownSidecar.hasMarkdownExtension(file) + ? markdownSidecar.merge(existingDocument, keyedEntries) + : entryWriter.mergeIntoDocument(existingDocument, keyedEntries); byte[] content = document.getBytes(StandardCharsets.UTF_8); recordWrittenFile(file, content); // Written atomically: the polling watcher (or another process) must never see a diff --git a/jablib/src/main/java/org/jabref/logic/directorylibrary/MarkdownSidecar.java b/jablib/src/main/java/org/jabref/logic/directorylibrary/MarkdownSidecar.java index 293b3d132cd7..a63f5528aba4 100644 --- a/jablib/src/main/java/org/jabref/logic/directorylibrary/MarkdownSidecar.java +++ b/jablib/src/main/java/org/jabref/logic/directorylibrary/MarkdownSidecar.java @@ -6,27 +6,34 @@ import java.nio.charset.StandardCharsets; import java.nio.file.Files; import java.nio.file.Path; +import java.util.ArrayList; +import java.util.HashSet; import java.util.List; import java.util.Locale; import java.util.Optional; +import java.util.Set; +import org.jabref.logic.exporter.HayagrivaEntryWriter; import org.jabref.logic.importer.ParserResult; import org.jabref.logic.importer.fileformat.HayagrivaImporter; +import org.jabref.logic.importer.fileformat.HayagrivaMapping; import org.jabref.logic.util.io.FileUtil; import org.jabref.model.entry.BibEntry; +import org.jabref.model.entry.field.Field; import org.jabref.model.entry.field.FieldFactory; import org.jabref.model.entry.field.StandardField; import org.jspecify.annotations.NullMarked; +import org.jspecify.annotations.Nullable; /// A directory-library sidecar in Markdown form: `X.md` next to `X.pdf`. The YAML frontmatter /// (between two `---` lines) is a regular Hayagriva document carrying the bibliographic data; /// the Markdown body below carries JabRef's long-form notes. The text under the `# Notes` /// heading is the entry's comment field, and every `## comment-` section is the -/// corresponding per-user comment field. Body content under other headings is not imported and -/// stays file-only, so the file remains a normal, markdownlint-clean Markdown note (usable in -/// Obsidian and plain editors); Typst users extract the frontmatter to obtain a plain Hayagriva -/// file. +/// corresponding per-user comment field. Body content under other headings is not imported but +/// survives rewrites verbatim, so the file remains a normal, markdownlint-clean Markdown note +/// (usable in Obsidian and plain editors); Typst users extract the frontmatter to obtain a +/// plain Hayagriva file. /// /// The body always describes the frontmatter's first entry; JabRef-authored Markdown sidecars /// contain exactly one entry. @@ -35,18 +42,25 @@ public class MarkdownSidecar { public static final String MARKDOWN_EXTENSION = "md"; static final String FRONTMATTER_DELIMITER = "---"; - - /// Field name (and name prefix) of JabRef's comment fields, see - /// [org.jabref.model.entry.field.UserSpecificCommentField]. - private static final String COMMENT_FIELD_PREFIX = "comment-"; + static final String NOTES_HEADING = "# Notes"; private static final String SECTION_HEADING_PREFIX = "## "; private final HayagrivaImporter importer = new HayagrivaImporter(); + private final HayagrivaEntryWriter entryWriter = new HayagrivaEntryWriter(); /// The split of a sidecar's raw text into its Hayagriva frontmatter and its notes body. record Document(String frontmatter, String body) { } + /// A `##` section of the notes body, with its content stripped of surrounding blank lines. + private record Section(String heading, String content) { + } + + /// The parsed notes body: the intro under the (dropped) `# Notes` document heading, and the + /// `##` sections in file order. + private record Body(String intro, List
sections) { + } + public static boolean hasMarkdownExtension(Path file) { return MARKDOWN_EXTENSION.equals(FileUtil.getFileExtension(file).orElse("").toLowerCase(Locale.ROOT)); } @@ -83,6 +97,32 @@ public ParserResult read(Path file) throws IOException { return result; } + /// Merges the given entries into a Markdown sidecar document (read-modify-write): the + /// frontmatter through [HayagrivaEntryWriter#mergeIntoDocument] — with the comment fields + /// stripped, they live in the body — and the body by regenerating the intro and the + /// `## comment-` sections from the first entry while keeping foreign sections + /// verbatim. The document heading is normalized to `# Notes`. + public String merge(@Nullable String existingDocument, List entries) { + Optional existing = Optional.ofNullable(existingDocument).flatMap(MarkdownSidecar::split); + List frontmatterEntries = entries.stream() + .map(keyed -> new HayagrivaEntryWriter.KeyedEntry(keyed.previousKey(), keyed.targetKey(), withoutCommentFields(keyed.entry()))) + .toList(); + String frontmatter = entryWriter.mergeIntoDocument(existing.map(Document::frontmatter).orElse(null), frontmatterEntries); + String body = entries.isEmpty() ? "" : renderBody(existing.map(Document::body).orElse(""), entries.getFirst().entry()); + + StringBuilder document = new StringBuilder(); + document.append(FRONTMATTER_DELIMITER).append('\n'); + document.append(frontmatter); + if (!frontmatter.isEmpty() && !frontmatter.endsWith("\n")) { + document.append('\n'); + } + document.append(FRONTMATTER_DELIMITER).append('\n'); + if (!body.isEmpty()) { + document.append('\n').append(body); + } + return document.toString(); + } + /// Splits the raw text at the frontmatter delimiters: the first line must be `---`, the /// frontmatter runs until the next `---` line, the body is everything below. static Optional split(String content) { @@ -103,32 +143,96 @@ static Optional split(String content) { /// The intro under the (optional) `# Notes` document heading becomes the comment field; /// every `## comment-` section becomes the equally named per-user comment field. static void applyBody(BibEntry entry, String body) { - String currentSection = ""; + Body parsed = parseBody(body); + if (!parsed.intro().isEmpty()) { + entry.setField(StandardField.COMMENT, parsed.intro()); + } + for (Section section : parsed.sections()) { + if (isCommentSection(section.heading()) && !section.content().isEmpty()) { + entry.setField(FieldFactory.parseField(section.heading()), section.content()); + } + } + } + + private static Body parseBody(String body) { + String currentHeading = ""; boolean titleSkipped = false; StringBuilder currentText = new StringBuilder(); + String intro = ""; + List
sections = new ArrayList<>(); for (String line : body.lines().toList()) { if (line.startsWith(SECTION_HEADING_PREFIX)) { - applySection(entry, currentSection, currentText.toString()); - currentSection = line.substring(SECTION_HEADING_PREFIX.length()).strip(); + if (currentHeading.isEmpty()) { + intro = currentText.toString().strip(); + } else { + sections.add(new Section(currentHeading, currentText.toString().strip())); + } + currentHeading = line.substring(SECTION_HEADING_PREFIX.length()).strip(); currentText.setLength(0); - } else if (!titleSkipped && currentSection.isEmpty() && currentText.toString().isBlank() && line.startsWith("# ")) { + } else if (!titleSkipped && currentHeading.isEmpty() && currentText.toString().isBlank() && line.startsWith("# ")) { titleSkipped = true; } else { currentText.append(line).append('\n'); } } - applySection(entry, currentSection, currentText.toString()); + if (currentHeading.isEmpty()) { + intro = currentText.toString().strip(); + } else { + sections.add(new Section(currentHeading, currentText.toString().strip())); + } + return new Body(intro, sections); } - private static void applySection(BibEntry entry, String section, String text) { - String value = text.strip(); - if (value.isEmpty()) { - return; + /// Regenerates the notes body: the entry's comment as the intro, existing comment sections + /// updated in place (dropped when cleared), foreign sections kept verbatim in their + /// position, and comment fields without an existing section appended in name order. + private static String renderBody(String existingBody, BibEntry entry) { + Body existing = parseBody(existingBody); + List blocks = new ArrayList<>(); + commentValue(entry, StandardField.COMMENT).ifPresent(blocks::add); + + Set existingHeadings = new HashSet<>(); + for (Section section : existing.sections()) { + if (isCommentSection(section.heading())) { + existingHeadings.add(section.heading()); + commentValue(entry, FieldFactory.parseField(section.heading())) + .ifPresent(value -> blocks.add(SECTION_HEADING_PREFIX + section.heading() + "\n\n" + value)); + } else if (!section.content().isEmpty()) { + blocks.add(SECTION_HEADING_PREFIX + section.heading() + "\n\n" + section.content()); + } else { + blocks.add(SECTION_HEADING_PREFIX + section.heading()); + } } - if (section.isEmpty()) { - entry.setField(StandardField.COMMENT, value); - } else if (section.startsWith(COMMENT_FIELD_PREFIX)) { - entry.setField(FieldFactory.parseField(section), value); + + entry.getFields().stream() + .map(Field::getName) + .filter(MarkdownSidecar::isCommentSection) + .filter(name -> !existingHeadings.contains(name)) + .sorted() + .forEach(name -> commentValue(entry, FieldFactory.parseField(name)) + .ifPresent(value -> blocks.add(SECTION_HEADING_PREFIX + name + "\n\n" + value))); + + if (blocks.isEmpty()) { + return ""; } + return NOTES_HEADING + "\n\n" + String.join("\n\n", blocks) + "\n"; + } + + private static Optional commentValue(BibEntry entry, Field field) { + return entry.getField(field).map(String::strip).filter(value -> !value.isEmpty()); + } + + private static boolean isCommentSection(String heading) { + return heading.startsWith(HayagrivaMapping.USER_COMMENT_PREFIX); + } + + private static BibEntry withoutCommentFields(BibEntry entry) { + BibEntry copy = new BibEntry(entry); + copy.clearField(StandardField.COMMENT); + copy.getFields().stream() + .filter(field -> field.getName().startsWith(HayagrivaMapping.USER_COMMENT_PREFIX)) + .toList() + .forEach(copy::clearField); + return copy; } } diff --git a/jablib/src/test/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizerTest.java b/jablib/src/test/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizerTest.java index 8159dde878ec..6376714ba97d 100644 --- a/jablib/src/test/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizerTest.java +++ b/jablib/src/test/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizerTest.java @@ -22,7 +22,10 @@ import org.jabref.logic.importer.util.GrobidPreferences; import org.jabref.model.database.BibDatabaseContext; import org.jabref.model.entry.BibEntry; +import org.jabref.model.entry.event.EntriesEventSource; +import org.jabref.model.entry.event.FieldChangedEvent; import org.jabref.model.entry.field.StandardField; +import org.jabref.model.entry.field.UserSpecificCommentField; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.io.TempDir; @@ -329,7 +332,7 @@ void localEditRewritesSidecarPreservingUnknownContent() throws IOException { } @Test - void firstEditOfStubEntryCreatesSidecarNextToPdf() throws IOException { + void firstEditOfStubEntryCreatesMarkdownSidecarNextToPdf() throws IOException { Files.createFile(root.resolve("loose.pdf")); openLibrary(); BibEntry stub = entries().getFirst(); @@ -338,13 +341,15 @@ void firstEditOfStubEntryCreatesSidecarNextToPdf() throws IOException { synchronizer.handleLocalChange(stub); synchronizer.flush(); - Path sidecar = root.resolve("loose.yml"); + Path sidecar = root.resolve("loose.md"); assertTrue(Files.exists(sidecar)); - assertTrue(Files.readString(sidecar).contains("Doe, John")); + String written = Files.readString(sidecar); + assertTrue(written.startsWith("---\n")); + assertTrue(written.contains("Doe, John")); } @Test - void newEntryWithoutFileGetsCitationKeyNamedSidecar() throws IOException { + void newEntryWithoutFileGetsCitationKeyNamedMarkdownSidecar() throws IOException { openLibrary(); BibEntry entry = new BibEntry(org.jabref.model.entry.types.StandardEntryType.Article) .withCitationKey("fresh2026") @@ -354,7 +359,69 @@ void newEntryWithoutFileGetsCitationKeyNamedSidecar() throws IOException { synchronizer.handleLocalChange(entry); synchronizer.flush(); - assertTrue(Files.exists(root.resolve("fresh2026.yml"))); + assertTrue(Files.exists(root.resolve("fresh2026.md"))); + } + + @Test + void commentEditsLandInTheMarkdownBody() throws IOException { + Files.createFile(root.resolve("loose.pdf")); + openLibrary(); + BibEntry stub = entries().getFirst(); + + stub.setField(StandardField.COMMENT, "First thoughts."); + stub.setField(new UserSpecificCommentField("koppor"), "Per-user thoughts."); + synchronizer.handleLocalChange(stub); + synchronizer.flush(); + + String written = Files.readString(root.resolve("loose.md")); + assertTrue(written.contains("# Notes\n\nFirst thoughts.\n\n## comment-koppor\n\nPer-user thoughts."), + () -> "unexpected body: " + written); + assertFalse(written.contains("comment:"), () -> "comment leaked into the frontmatter: " + written); + } + + @Test + void markdownRewriteKeepsForeignBodySections() throws IOException { + Path sidecar = root.resolve("smith2020.md"); + Files.writeString(sidecar, MARKDOWN_SIDECAR + """ + + ## Reading list + + Follow-up papers. + """); + openLibrary(); + BibEntry entry = entries().getFirst(); + + entry.setField(StandardField.COMMENT, "Updated comment text."); + synchronizer.handleLocalChange(entry); + synchronizer.flush(); + + String written = Files.readString(sidecar); + assertTrue(written.contains("Updated comment text.")); + assertTrue(written.contains("## Reading list\n\nFollow-up papers."), () -> "foreign section lost: " + written); + assertFalse(written.contains("Shared comment text.")); + } + + @Test + void filteredKeystrokeEventsStillTriggerTheDebouncedWrite() throws IOException, InterruptedException { + Path sidecar = root.resolve("smith2020.yml"); + Files.writeString(sidecar, ARTICLE_YAML); + openLibrary(); + BibEntry entry = entries().getFirst(); + + // The CoarseChangeFilter marks every keystroke of a same-field burst as filtered; only + // relying on unfiltered events would strand the burst's tail (it never gets one) + entry.setField(StandardField.NOTE, "typed letter by letter", EntriesEventSource.LOCAL); + FieldChangedEvent keystroke = new FieldChangedEvent(entry, StandardField.NOTE, "typed letter by letter", "first version"); + keystroke.setFilteredOut(true); + synchronizer.listen(keystroke); + + Instant deadline = Instant.now().plusSeconds(10); + while (!Files.readString(sidecar).contains("typed letter by letter")) { + if (Instant.now().isAfter(deadline)) { + throw new AssertionError("debounced write did not happen; file: " + Files.readString(sidecar)); + } + Thread.sleep(25); + } } @Test diff --git a/jablib/src/test/java/org/jabref/logic/directorylibrary/MarkdownSidecarTest.java b/jablib/src/test/java/org/jabref/logic/directorylibrary/MarkdownSidecarTest.java index 0e82768ded6f..9692022f9839 100644 --- a/jablib/src/test/java/org/jabref/logic/directorylibrary/MarkdownSidecarTest.java +++ b/jablib/src/test/java/org/jabref/logic/directorylibrary/MarkdownSidecarTest.java @@ -6,9 +6,12 @@ import java.util.List; import java.util.Optional; +import org.jabref.logic.exporter.HayagrivaEntryWriter; import org.jabref.model.entry.BibEntry; import org.jabref.model.entry.field.FieldFactory; import org.jabref.model.entry.field.StandardField; +import org.jabref.model.entry.field.UserSpecificCommentField; +import org.jabref.model.entry.types.StandardEntryType; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.io.TempDir; @@ -135,6 +138,27 @@ void multiParagraphCommentKeepsItsInnerBlankLine() throws IOException { assertEquals(Optional.of("First paragraph.\n\nSecond paragraph."), entry.getField(StandardField.COMMENT)); } + @Test + void mergeRoundTripsCommentsThroughTheBody() throws IOException { + BibEntry entry = new BibEntry(StandardEntryType.Article) + .withCitationKey("smith2020") + .withField(StandardField.TITLE, "A Test Article") + .withField(StandardField.COMMENT, "Shared comment text.") + .withField(new UserSpecificCommentField("koppor"), "Per-user comment text."); + + String document = sidecar.merge(null, List.of(new HayagrivaEntryWriter.KeyedEntry("", "smith2020", entry))); + Path file = write("smith2020.md", document); + + assertTrue(document.startsWith("---\n"), () -> "missing frontmatter: " + document); + assertTrue(document.contains("\n# Notes\n\nShared comment text.\n\n## comment-koppor\n\nPer-user comment text.\n"), + () -> "unexpected body: " + document); + BibEntry reimported = sidecar.read(file).getDatabase().getEntries().getFirst(); + assertEquals(entry.getField(StandardField.COMMENT), reimported.getField(StandardField.COMMENT)); + assertEquals(entry.getField(new UserSpecificCommentField("koppor")), + reimported.getField(new UserSpecificCommentField("koppor"))); + assertEquals(entry.getField(StandardField.TITLE), reimported.getField(StandardField.TITLE)); + } + @Test void emptyBodySetsNoCommentFields() throws IOException { Path file = write("smith2020.md", """ From 094341a52baf03aa801513acfcccc2dc07beabd4 Mon Sep 17 00:00:00 2001 From: Oliver Kopp Date: Thu, 3 Sep 2026 04:22:01 +0200 Subject: [PATCH 5/8] Fix write-back requirement marker, CHANGELOG duplicates, and formatting Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01Vr3E1Gg5DRU4LQDDVnhPys --- CHANGELOG.md | 6 ------ .../java/org/jabref/gui/exporter/SaveDatabaseAction.java | 2 +- .../org/jabref/logic/directorylibrary/MarkdownSidecar.java | 4 ++-- 3 files changed, 3 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9bd1011145ea..f10ef38f2010 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,12 +14,6 @@ Note that this project **does not** adhere to [Semantic Versioning](https://semv - Directory libraries now save into their sidecar files: edits are written back automatically (debounced until typing pauses; Ctrl+S forces the write and no longer creates a `.bib`), the first edit of a PDF-only entry creates a Markdown sidecar (`X.md` with the Hayagriva data as frontmatter and the comment fields as notes body), renaming a citation key renames the YAML key, and deleting an entry removes it from its file (the file is trashed once empty, the PDF stays). Hand-written content that JabRef does not understand survives rewrites. [#739](https://github.com/JabRef/jabref-koppor/pull/739) - Directory libraries now stay in sync with external file changes: creating, editing, deleting, or renaming `.yml`/`.md`/`.pdf` files in the opened folder updates the open library live, and renames keep the affected entries (selection and undo history survive). [#738](https://github.com/JabRef/jabref-koppor/pull/738) - We added "Open folder as library" (File menu): JabRef opens a directory as a library, filling the main table from the Hayagriva `.yml` files, the Markdown notes files with a Hayagriva YAML frontmatter (`.md`; the notes body maps to the entry's comment fields), and the PDFs found in the folder tree. A PDF next to a sidecar of the same name is linked to the sidecar's entry; PDFs without a sidecar appear immediately and are enriched in the background with metadata extracted from the PDF itself and a generated citation key; a missing DOI is looked up online and its metadata fills the remaining empty fields. Directory libraries that were open on shutdown are reopened on the next start. [#737](https://github.com/JabRef/jabref-koppor/pull/737) -- We added "Open folder as library" (File menu): JabRef opens a directory as a library, filling the main table from the Hayagriva `.yml` files and PDFs found in the folder tree. A PDF next to a `.yml` sidecar of the same name is linked to the sidecar's entry; PDFs without a sidecar appear as stub entries titled after the file. Edits are not yet written back to the files. [#737](https://github.com/JabRef/jabref-koppor/pull/737) -- We added "Open folder as library" (File menu): JabRef opens a directory as a library, filling the main table from the Hayagriva `.yml` files and PDFs found in the folder tree. A PDF next to a `.yml` sidecar of the same name is linked to the sidecar's entry; PDFs without a sidecar appear as entries with metadata extracted from the PDF itself (falling back to the file name). Edits are not yet written back to the files. [#737](https://github.com/JabRef/jabref-koppor/pull/737) -- We added "Open folder as library" (File menu): JabRef opens a directory as a library, filling the main table from the Hayagriva `.yml` files and PDFs found in the folder tree. A PDF next to a `.yml` sidecar of the same name is linked to the sidecar's entry; PDFs without a sidecar appear immediately and are enriched in the background with metadata extracted from the PDF itself and a generated citation key. Edits are not yet written back to the files. [#737](https://github.com/JabRef/jabref-koppor/pull/737) -- We added "Open folder as library" (File menu): JabRef opens a directory as a library, filling the main table from the Hayagriva `.yml` files and PDFs found in the folder tree. A PDF next to a `.yml` sidecar of the same name is linked to the sidecar's entry; PDFs without a sidecar appear immediately and are enriched in the background with metadata extracted from the PDF itself and a generated citation key; a missing DOI is looked up online and its metadata fills the remaining empty fields. Edits are not yet written back to the files. [#737](https://github.com/JabRef/jabref-koppor/pull/737) -- We added "Open folder as library" (File menu): JabRef opens a directory as a library, filling the main table from the Hayagriva `.yml` files and PDFs found in the folder tree. A PDF next to a `.yml` sidecar of the same name is linked to the sidecar's entry; PDFs without a sidecar appear immediately and are enriched in the background with metadata extracted from the PDF itself and a generated citation key; a missing DOI is looked up online and its metadata fills the remaining empty fields. Directory libraries that were open on shutdown are reopened on the next start. Edits are not yet written back to the files. [#737](https://github.com/JabRef/jabref-koppor/pull/737) -- We added "Open folder as library" (File menu): JabRef opens a directory as a library, filling the main table from the Hayagriva `.yml` files, the Markdown notes files with a Hayagriva YAML frontmatter (`.md`; the notes body maps to the entry's comment fields), and the PDFs found in the folder tree. A PDF next to a sidecar of the same name is linked to the sidecar's entry; PDFs without a sidecar appear immediately and are enriched in the background with metadata extracted from the PDF itself and a generated citation key; a missing DOI is looked up online and its metadata fills the remaining empty fields. Directory libraries that were open on shutdown are reopened on the next start. Edits are not yet written back to the files. [#737](https://github.com/JabRef/jabref-koppor/pull/737) - We added a "Commit and push" button which allows to commit and then push in one go for Git operations. [#16339](https://github.com/JabRef/jabref/issues/16339) - We added the option to close and reopen the PDF preview in the unlinked files dialog. [#16159](https://github.com/JabRef/jabref/issues/16159) - We added the ability for LibreOffice BST citations to use style-defined labels. [forum#3764]([https://github.com/JabRef/jabref/issues/16357](https://discourse.jabref.org/t/feature-request-custom-citation-styles-from-bst/3764)) diff --git a/jabgui/src/main/java/org/jabref/gui/exporter/SaveDatabaseAction.java b/jabgui/src/main/java/org/jabref/gui/exporter/SaveDatabaseAction.java index 9b460262c16c..cc682394735e 100644 --- a/jabgui/src/main/java/org/jabref/gui/exporter/SaveDatabaseAction.java +++ b/jabgui/src/main/java/org/jabref/gui/exporter/SaveDatabaseAction.java @@ -224,7 +224,7 @@ private SaveResult save(BibDatabaseContext bibDatabaseContext, SaveDatabaseMode if (bibDatabaseContext.getLocation() == DatabaseLocation.DIRECTORY) { // A directory library persists into its sidecar files; saving means flushing the // debounced writes, never writing a .bib ("Save as" remains the explicit snapshot) - // [impl->req~directory-library.write-back~1] + // [impl->req~directory-library.write-back~2] DirectoryLibrarySynchronizer synchronizer = bibDatabaseContext.getDirectorySynchronizer(); if (synchronizer != null) { synchronizer.flush(); diff --git a/jablib/src/main/java/org/jabref/logic/directorylibrary/MarkdownSidecar.java b/jablib/src/main/java/org/jabref/logic/directorylibrary/MarkdownSidecar.java index a63f5528aba4..8ad42a0ef0af 100644 --- a/jablib/src/main/java/org/jabref/logic/directorylibrary/MarkdownSidecar.java +++ b/jablib/src/main/java/org/jabref/logic/directorylibrary/MarkdownSidecar.java @@ -105,8 +105,8 @@ public ParserResult read(Path file) throws IOException { public String merge(@Nullable String existingDocument, List entries) { Optional existing = Optional.ofNullable(existingDocument).flatMap(MarkdownSidecar::split); List frontmatterEntries = entries.stream() - .map(keyed -> new HayagrivaEntryWriter.KeyedEntry(keyed.previousKey(), keyed.targetKey(), withoutCommentFields(keyed.entry()))) - .toList(); + .map(keyed -> new HayagrivaEntryWriter.KeyedEntry(keyed.previousKey(), keyed.targetKey(), withoutCommentFields(keyed.entry()))) + .toList(); String frontmatter = entryWriter.mergeIntoDocument(existing.map(Document::frontmatter).orElse(null), frontmatterEntries); String body = entries.isEmpty() ? "" : renderBody(existing.map(Document::body).orElse(""), entries.getFirst().entry()); From 4573732510c8b25266eb9cff8c8f5df260aa1c56 Mon Sep 17 00:00:00 2001 From: Oliver Kopp Date: Mon, 7 Sep 2026 00:16:10 +0200 Subject: [PATCH 6/8] Apply IntelliJ formatter after upstream resync Fixes the CI format check. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01DDcHNMt9fPWnpYaHheFvry --- .../directorylibrary/DirectoryLibrarySynchronizer.java | 3 ++- .../org/jabref/logic/exporter/HayagrivaEntryWriter.java | 9 ++++++--- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizer.java b/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizer.java index c8208aef10ed..b0b7d27b2cb1 100644 --- a/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizer.java +++ b/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizer.java @@ -219,7 +219,8 @@ Path sidecarOf(BibEntry entry) { /// Waits until every event queued so far has been handled (tests). void awaitPendingEvents() throws InterruptedException, ExecutionException { - syncExecutor.submit(() -> { }).get(); + syncExecutor.submit(() -> { + }).get(); } /// Stops watching and writes what is still pending. Events already queued (the last diff --git a/jablib/src/main/java/org/jabref/logic/exporter/HayagrivaEntryWriter.java b/jablib/src/main/java/org/jabref/logic/exporter/HayagrivaEntryWriter.java index 3d6ac8edaf77..876fd667275a 100644 --- a/jablib/src/main/java/org/jabref/logic/exporter/HayagrivaEntryWriter.java +++ b/jablib/src/main/java/org/jabref/logic/exporter/HayagrivaEntryWriter.java @@ -352,9 +352,12 @@ private ObjectNode parentNode(String type, String title) { private String booktitleParentType(EntryType entryType) { return switch (entryType) { - case StandardEntryType.InProceedings -> "proceedings"; - case StandardEntryType.InBook -> "book"; - default -> "anthology"; + case StandardEntryType.InProceedings -> + "proceedings"; + case StandardEntryType.InBook -> + "book"; + default -> + "anthology"; }; } From cb0d0865f959254bc420bcab737e8ae146bca59d Mon Sep 17 00:00:00 2001 From: Oliver Kopp Date: Mon, 7 Sep 2026 11:13:53 +0200 Subject: [PATCH 7/8] Renumber the directory-as-library ADR to 0072 Upstream added a 0071 ADR after this stack claimed the number, so the MADR duplicate-ID check failed. Co-Authored-By: Claude Opus 4.8 --- ....md => 0072-directory-as-library-with-hayagriva-sidecars.md} | 2 +- docs/requirements/directory-library.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) rename docs/decisions/{0071-directory-as-library-with-hayagriva-sidecars.md => 0072-directory-as-library-with-hayagriva-sidecars.md} (99%) diff --git a/docs/decisions/0071-directory-as-library-with-hayagriva-sidecars.md b/docs/decisions/0072-directory-as-library-with-hayagriva-sidecars.md similarity index 99% rename from docs/decisions/0071-directory-as-library-with-hayagriva-sidecars.md rename to docs/decisions/0072-directory-as-library-with-hayagriva-sidecars.md index 5229eadcaf1f..79ff20e89107 100644 --- a/docs/decisions/0071-directory-as-library-with-hayagriva-sidecars.md +++ b/docs/decisions/0072-directory-as-library-with-hayagriva-sidecars.md @@ -1,5 +1,5 @@ --- -nav_order: 0071 +nav_order: 0072 parent: Decision Records --- diff --git a/docs/requirements/directory-library.md b/docs/requirements/directory-library.md index 74abfad2b742..e9f361f7c779 100644 --- a/docs/requirements/directory-library.md +++ b/docs/requirements/directory-library.md @@ -19,7 +19,7 @@ online and the metadata behind it fills only the fields the PDF did not provide. and `.md` files without a Hayagriva frontmatter are skipped; unparseable Hayagriva files are reported as warnings without aborting the scan. Scanning must not write or modify any file in the directory. -See [ADR 71](../decisions/0071-directory-as-library-with-hayagriva-sidecars.md) for more details. +See [ADR 72](../decisions/0072-directory-as-library-with-hayagriva-sidecars.md) for more details. Needs: impl From ea7b65575fd7f32cd7060adc5c6e18a4849e76b8 Mon Sep 17 00:00:00 2001 From: Oliver Kopp Date: Wed, 23 Sep 2026 15:53:45 +0200 Subject: [PATCH 8/8] Use unnamed variables where the value is unused Co-Authored-By: Claude Opus 5.5 --- .../logic/directorylibrary/DirectoryLibrarySynchronizer.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizer.java b/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizer.java index b0b7d27b2cb1..90488fa7e14f 100644 --- a/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizer.java +++ b/jablib/src/main/java/org/jabref/logic/directorylibrary/DirectoryLibrarySynchronizer.java @@ -232,7 +232,7 @@ public List shutdown() { syncExecutor.shutdown(); try { syncExecutor.awaitTermination(2, TimeUnit.SECONDS); - } catch (InterruptedException e) { + } catch (InterruptedException _) { Thread.currentThread().interrupt(); } return flush();