Skip to content

Open folder as library (directory as library, phase 1) - #737

Open
koppor wants to merge 24 commits into
hayagriva-writerfrom
open-directory-library
Open

koppor wants to merge 24 commits into
hayagriva-writerfrom
open-directory-library

Conversation

@koppor

@koppor koppor commented Jul 13, 2026 •

Copy link
Copy Markdown
Member

📚 Directory-as-library stack — bottom → top, each builds on the one below:

  1. Programmatic Hayagriva writer (directory-as-library, phase 0) #736 · Hayagriva YAML writer
  2. Open folder as library (directory as library, phase 1) #737 · Open folder as library ← this PR
  3. Live inbound sync for directory libraries (phase 2) #738 · Inbound file sync
  4. Write-back to Hayagriva sidecars (directory as library, phase 3) #739 · Write-back to sidecars
  5. Groups panel mirrors the directory structure (phase 4, #10930) #740 · Directory-structure groups
  6. Pattern-driven pair renames for directory libraries (phase 5a) #741 · Pattern-driven pair renames
  7. Mirror directory libraries into a .bib with git-sync merge-back #743 · .bib mirror + merge-back
  8. Convert .bib libraries into directory libraries #744 · Convert .bib → directory
  9. Make directory libraries reachable through the REST API #745 · REST API support
  10. Split DirectoryLibrarySynchronizer into collaborators #760 · Split the synchronizer

Related issues and pull requests

Phase 1 of the "directory as library" plan — see PLAN.md and phase 0 (#736, merged state assumed). Related to JabRef#10930 (directory-mirroring groups build on this later); no issue is closed by this PR.

PR Description

🤖 A new File > Open folder as library entry opens a directory as a JabRef library, filling the table from the Hayagriva sidecar files and PDFs found in the tree. A PDF beside a matching sidecar is linked to its entry; a PDF without one appears at once and gains its metadata and a citation key in the background, so opening never blocks. Directory libraries reopen on the next start.

jabref-contrib-policy:4.2:reviewed​:ok

Analogies

Like honey, the sidecar convention binds PDF and metadata together without any glue you can see. Like chocolate broken along its grooves, the directory tree splits naturally into entries — one square per sidecar, nested pieces included. And like the moon, most of this feature's mass (sync, write-back, groups) is still on the far side; this PR is the bright half you can already navigate by.

Steps to test

  1. Create a folder with a Hayagriva sidecar pair (smith2020.yml + smith2020.pdf), a bare PDF, and a subfolder with another pair.
  2. File > Open folder as library..., select the folder.
  3. The table fills instantly: sidecar entries (with journal/booktitle from Hayagriva parents and the PDF linked), plus a stub for the bare PDF that gains extracted metadata and a citation key moments later. The tab is titled after the folder:

Main table after opening a folder as library

  1. Closing the tab with modifications prompts "Save before closing" with the directory path; "Save as" writes a regular .bib.
  2. Quit JabRef and start it again: the folder tab is restored with the session:

Directory library restored after restart

Guarded by DirectoryLibraryScannerTest (pairing, nesting, gitignore/hidden skipping, non-Hayagriva YAML, broken-file warnings, no-write guarantee).

AI usage

Claude Code (model claude-fable-5).

AI CHECKLIST.md walkthrough

1. Code self-review

Nullability and control flow
  • No == null / != null checks in new code except Path.getFileName()/getParent() nullable returns of the JDK API.
  • No Objects.requireNonNull(...).
  • New classes annotated with @NullMarked.
  • Optional consumed with ifPresent / map / or — no isPresent() + get() blocks.
  • [/] StringUtil.isBlank(...) — no null-or-blank string checks needed.
Exceptions
  • No catch (Exception e); the scanner propagates IOException, the action surfaces failures via the background task's onFailure.
  • No throw new RuntimeException(...) / IllegalStateException(...).
  • Logged exception passed as last logger argument (visitFileFailed debug log).
Style and idioms
  • New BibEntry built with withers.
  • Modern Java: List.of(), records, Files.walkFileTree, text blocks in tests.
  • [/] Regexes — none added.
  • Background work uses org.jabref.logic.util.BackgroundTask.
  • No commented-out code, no trivial comments, no AI-disclosure comments.
  • Markdown Javadoc (///) with Markdown syntax.
User-facing text
  • All user-facing strings localized (Open folder as library..., error/warning messages).
  • Sentence case, no trailing !, no : labels.
  • Variance via placeholders ('%0').
Security
  • [/] No HTML output.
Tests
  • DirectoryLibraryScannerTest (11 tests) covers scan behavior incl. the no-write guarantee; GitIgnoreFileFilterTest moved along with the class and stays green.
  • Tests assert object contents with plain JUnit asserts, no @DisplayName, exceptions propagate, @TempDir used.

2. Verification commands

  • ./gradlew :jablib:check — 10721 tests, 2 failures in RemoteCommunicationTest only (local port occupied by a running JabRef instance on this machine; unrelated, pass on CI).
  • ./gradlew :jablib:checkstyleMain :jablib:checkstyleTest :jabgui:checkstyleMain.
  • ./gradlew :jablib:modernizer.
  • ./gradlew :rewriteRun left no diff (dry-run clean).
  • ./gradlew :jablib:javadoc.
  • ./gradlew traceRequirements (new req~directory-library.scan~1 covered by impl marker).
  • npx markdownlint-cli2 on the changed/added Markdown.
  • [/] IntelliJ-format docker image — not runnable here (no docker group); CI format job is the backstop.

3. Documentation

  • CHANGELOG.md entry added (TODO placeholder replaced with this PR's link after creation).
  • Issue search done; Dynamic group mirroring the file system. jabref#10930 related (linked, not closed).
  • Requirement added: docs/requirements/directory-library.md.
  • ADR added: docs/decisions/0072-directory-as-library-with-hayagriva-sidecars.md.

4. Pull request

  • PR body built from .github/PULL_REQUEST_TEMPLATE.md, every section filled.
  • All checklist items kept and marked.
  • All HTML comments removed.
  • PR created from a body file via gh api (same-owner cross-repo head needs head_repo).
  • CHANGELOG.md TODO replaced with the real PR link after creation, committed and pushed.

Checklist

  • I own the copyright of the code submitted and I license it under the MIT license
  • If AI tools were used, I disclosed them in the "AI usage" section and reviewed, understood, and take full ownership of all AI-generated code
  • I manually tested my changes in running JabRef (opened a demo folder via the new menu item on the X display; see screenshot)
  • I added JUnit tests for changes (if applicable)
  • I added screenshots in the PR description (if change is visible to the user)
  • [/] I added a screenshot in the PR description showing a library with a single entry with me as author and as title the issue number
  • I added one sentence (max 20 words) to CHANGELOG.md describing the change from the user's point of view (if the change is visible to the user)
  • I checked the user documentation for up to dateness and submitted a pull request to our user documentation repository — deferred until the feature stabilizes (write-back phase); tracked in PLAN.md phase 5

🤖 Generated with Claude Code

koppor and others added 2 commits July 13, 2026 23:13
A directory can now be opened as a library: the main table fills from
the Hayagriva .yml sidecars and PDFs found in the tree. Sidecar entries
link their same-basename PDF relative to the root; PDFs without a
sidecar become stub entries. The context gets the new
DatabaseLocation.DIRECTORY with an empty database path, which keeps
autosave/backup/change-monitor and path-dependent actions off by
default; "Save as" converts the tab into a regular .bib library.
GitIgnoreFileFilter moves to jablib for reuse by the scanner.

Edits are not yet written back to the files; live synchronization and
write-back follow in later steps (see PLAN.md on directory-as-library).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
koppor and others added 6 commits July 14, 2026 02:03
PDFs without a sidecar now go through PdfMergeMetadataImporter (the
standard PDF import pipeline: embedded BibTeX, XMP, content heuristics,
plus GROBID/identifier lookups as configured) instead of becoming
filename-only stubs; the stub remains the fallback when nothing usable
can be extracted. The new PdfEntryFactory carries this behavior so the
file watcher can reuse it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Entries created for PDFs without a sidecar get a citation key from the
configured pattern after insertion, so the uniqueness check sees the
whole library. Sidecar entries keep their Hayagriva key untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Opening a folder no longer blocks on PDF parsing or identifier lookups:
the scan inserts quick stub entries for sidecar-less PDFs and the new
PdfEnrichmentTask extracts their metadata in the background afterwards,
enriching each stub in place (progress shown in the task overlay,
cancellable). Enrichment mutations carry EntriesEventSource.SHARED
because they are system-initiated, not user edits; the citation key is
generated per entry once its metadata arrived.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When the PDF itself yields no DOI, the DOI is looked up via CrossRef
(the same reverse search as the magnifier button next to the DOI field)
and the metadata behind it merges into the entry with set-if-empty
semantics (BibEntry#mergeWith): PDF-extracted values are never
overwritten, the DOI metadata only fills the gaps. PDFs that already
carry a DOI keep being enriched by PdfMergeMetadataImporter itself.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Directory libraries were skipped by the session store because they have
no database path; their root directory now stands in, and paths that
turn out to be directories are routed from openFiles to the
directory-library opener on the next start.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The directory-library spec gains a Markdown sidecar form: X.md next to
X.pdf, whose YAML frontmatter is a regular Hayagriva document and whose
body carries the long-form notes (# Notes for the comment field, one
## comment-<name> section per per-user comment). Long-form Markdown
notes read terribly as YAML block scalars; as a Markdown body the
folder doubles as a plain notes collection (Obsidian, any editor). The
scanner imports such files next to plain .yml/.yaml Hayagriva files.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Q5Y5GTihE9rZxKrvPqnHV4
koppor and others added 9 commits September 3, 2026 03:18
Resync layer 2 of the stack. Conflicts: import-block collision in
UnlinkedFilesCrawler (kept all three imports), and a semantic conflict
from upstream's UndoManager -> GuiUndoManager refactor — retyped
OpenDirectoryLibraryAction's undo manager to GuiUndoManager to match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DDcHNMt9fPWnpYaHheFvry
Upstream took 0066 in the meantime; the MADR check rejects the duplicate id.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vr3E1Gg5DRU4LQDDVnhPys
A failed "Save as" no longer strips the directory library of its root; the background PDF enrichment is cancelled with the tab, logs failures, and no longer marks the library as modified; opening an already open folder raises its tab; extensions match case-insensitively and gitignored PDFs are not paired. Optional/null idioms follow the checklist.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vr3E1Gg5DRU4LQDDVnhPys
Fixes the CI format check.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DDcHNMt9fPWnpYaHheFvry
Upstream dropped the undo manager from LibraryTab.createLibraryTab and
added a GitHandlerRegistry; OpenDirectoryLibraryAction and its callers
follow suit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DDcHNMt9fPWnpYaHheFvry
Upstream added a 0071 ADR after this stack claimed the number, so the
MADR duplicate-ID check failed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
koppor and others added 3 commits September 8, 2026 00:21
# Conflicts:
#	jabgui/src/main/java/org/jabref/gui/LibraryTab.java
Main took 0072.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CajtxjcCEjS87f4hoTBTJD
koppor and others added 3 commits September 13, 2026 21:48
A folder whose `.gitignore` ignores everything — a `*` catch-all next to a
`.gitkeep`, the common shape of a scratch PDF folder — opened as an empty
library, because the scanner ran every PDF and sidecar through the gitignore
filter. The filter now applies only to non-content files; hidden files and
ignored subtrees are still skipped.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DDcHNMt9fPWnpYaHheFvry
A directory library resolves every linked file against its single root, so the
library- and user-specific file directory overrides in the library properties
do not apply. Both rows are now disabled when the library is a directory library.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DDcHNMt9fPWnpYaHheFvry

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant