Persist reverse-link projection for scalable backlink queries

[brylie]

Problem

Updated 2026-09-12, after #257 (closing #83) reintroduced the Backlinks panel: the description below no longer matches the current implementation. #257 could not "preserve the current in-memory index as the live UI path" (this issue's own original scope bullet) because that path was already gone — $lib/data/links.ts#listIncomingLinks's in-memory index is scoped to one Y.Doc, which since #120's per-Document sharding can only ever be one Document's own shard, structurally unable to answer "what links to this Document" workspace-wide. That's exactly why the panel built on it was removed when #120 shipped.

#257 instead added services/documents.ts#listBacklinks: a server-side scan that fans out across every Document's own shard (the same fanOutCatalogedAndUncataloged pattern search_workspace already established in #191), calling the stateless listOutgoingLinks(doc, documentId) once per fanned-out Document. It has no persisted index at all, so "lost on process restart" no longer describes the actual cost — the real cost now is that this full fan-out scan runs synchronously on every Document page load (+page.server.ts's SSR load), not just an on-demand search. That's a materially higher-frequency hot path than anything this issue was originally scoped against, and it's also not live: a backlink created while the target Document is open doesn't appear until the next navigation/refresh (docs/specifications/internal-links.md §5 documents this tradeoff).

As the workspace grows, backlinks still need a durable, rebuildable projection so a target Document's incoming links can be retrieved through an indexed query without scanning every Document's shard on every page load.

Scope

• Extend the rebuildable read-model work in persistence.md's record_index read model is speced but not implemented #70 with a reverse-link projection derived from canonical ID-backed page_link blocks and inline record: wiki links.
• Store source record ID, source Document ID, target Document or Collection ID, link kind, and enough current source context to render backlink results safely.
• Index the projection by target ID for incoming-link queries.
• Keep the projection synchronized with link create, edit, move, and delete operations; define rebuild and recovery behavior from authoritative Yjs data.
• Make server-side backlink retrieval permission-filtered before source titles, context, counts, or identifiers are returned — services/documents.ts#listBacklinks already does this the same way listDocuments does; preserve that contract rather than reimplementing it.
• Define one permission-filtered backlink query service used by the UI and exposed through MCP; the future REST surface in Public REST API with feature parity to MCP tools, sharing the service layer #50 must adapt the same contract rather than reimplement it. (If the MCP-exposure half ships sooner as its own issue against the current listBacklinks, this becomes "migrate that MCP tool onto the durable projection" rather than building it from scratch here.)
• Replace services/documents.ts#listBacklinks's per-request shard fan-out with the durable query path once it exists; do not introduce a second canonical link write path.
• Document the storage boundary so the SQLite implementation can migrate to the planned Postgres read model without changing the backlink contract.
• Run npm run benchmark:workspace against a fixture with a realistic Document count, before and after this lands, to get real numbers on the fan-out-scan cost this issue removes from the Document page-load path — CLAUDE.md's capacity-benchmark guidance calls for this on any PR changing shard-resolution/fan-out behavior, and it wasn't run for Navigate backlinks to their exact referring block (closes #83) #257 since the page-load-frequency implication wasn't obvious at the time.

Non-goals

• Changing the canonical Yjs representation of internal links.
• Graph visualization (Graph view of internal-link relationships (Obsidian/Foam-style) #48) or broken-link remediation (Broken-link cleanup/reporting once #21's backlink index exists #47).
• Selecting SQLite versus Postgres as the long-term primary data store.

Done when: a restarted service can answer a permitted Document's backlinks from an indexed, rebuildable projection without loading or scanning the full workspace, the Document page-load path no longer pays a per-load shard fan-out cost, and the result remains correct after link edits, source deletion, and projection rebuild.

[brylie]

Blocked by #70: #70 establishes the rebuildable persistence/read-model boundary. This follow-up applies that boundary to canonical internal-link edges, adding an indexed and permission-filtered reverse-link projection for scalable backlink queries. Keeping it separate lets #70 decide the shared projection approach without expanding its search and collection-query scope.

[brylie]

Set Priority to P2: durable reverse-link persistence improves scalability and restart behavior, but the current incremental in-memory index is functional. This work is sequenced after #70’s shared read-model decision and implementation.

[brylie]

Moved out of `0.4.0` — this is a scalability/durability optimization over the existing in-memory backlink index, which #81's own body already keeps as the live UI path until this lands, so it isn't blocking Table/Board/Calendar coherence work. It's also blocked by #70 (persistence.md's record_index read model), which itself has no milestone yet — pulling #81 into 0.4.0 without #70 would leave it unshippable there anyway.

The #70 → #81 dependency stays tracked via GitHub's native issue-dependency graph (already set). Revisit both together whenever the record_index/read-model work is next prioritized.

[brylie]

Filed #258 (P1, Size S) to expose the current listBacklinks implementation via MCP, sooner than this issue's own durable rebuild — see #258 for why: docs/prd.md's "Backlinks" requirement explicitly frames MCP read/write parity as part of what completes it, and this repo's PR #257 shipped the UI half only. No native blocks/blocked-by between the two issues — #258 doesn't need this issue's durable projection to ship, and this issue doesn't need #258 either. Once this issue's durable projection lands, it should migrate/subsume #258's MCP tool onto it rather than reimplementing the tool from scratch — already noted in this issue's own updated Scope section above.