diff --git a/docs/clipboard-semantics.md b/docs/clipboard-semantics.md new file mode 100644 index 00000000..a68ec3dc --- /dev/null +++ b/docs/clipboard-semantics.md @@ -0,0 +1,264 @@ +# Standard-View Clipboard USFM Semantics (PT-4201) + +## Overview + +This document specifies the copy/paste behavior for USFM content in Platform.Bible's Standard-view scripture editor. These semantics ensure that clipboard operations preserve USFM fidelity, allowing users to copy from the Standard view, paste into external editors, and vice versa while maintaining marker integrity and positional accuracy. + +--- + +## Semantics Decisions (S1–S7) + +### S1. Copy (Standard view) — `text/plain` is valid USFM of the selection + +Already ~true because markers are real text nodes; this plan closes the exceptions: + +- **Note callers:** `\f` + caller + content + `\f*` with the note's USJ `caller` value (`+`, `-`, or literal) — currently the caller is silently dropped (and rewrites to `+` on re-paste). This deliberately _exceeds_ P9, where plain-text copy carries only the caller glyph and full notes survive only in CF_HTML. Rationale: in P10 there is no CF_HTML fidelity carrier for external apps, and the ticket requires "copy from Standard → paste into a plain-text editor: markers present and correctly placed". + +- **Collapsed notes** contribute their full (visually hidden) `\f …\f*` bytes. "What you copy" > "what you see", for fidelity. This is documented, not changed. + +- **Display-NBSP handling, and ONE set of bytes for both readable flavors:** `text/plain` inverts EVERY display-NBSP to a plain space and leaves a data-NBSP as the `~` the display already shows. `text/html` carries those SAME bytes — the selection's USFM, HTML-escaped, one `

…

` per `\n`-separated line (`usfmToClipboardHtml`, `whitespaceDisplay.plugin.utils.ts`). Lexical's own DOM export (`$getHtmlContent`) is deliberately NOT the html carrier, because it is lossy in two independent ways: `ImmutableNoteCallerNode.exportDOM` carries a collapsed note's caller as a `data-caller` ATTRIBUTE with no text at all, and `UnknownNode.exportDOM` returns `{element: null}` for every kind, which stops `@lexical/html`'s node walk before the construct's own display children — so an export-derived html reached a consumer with the note caller missing and every figure/sidebar/periph/ref/optbreak absent. Paratext 9 is that consumer in practice: its paste reads an incoming fragment's TEXT as USFM, so a P10→P9 note paste arrived with an empty caller. The block-per-line shape mirrors Lexical's own `TextNode` export, so an html consumer that re-imports through `$generateNodesFromDOM` (`StructureKeyboardPlugin`'s DROP sanitizer, and any foreign consumer that rebuilds nodes from html) still sees one block per line; `white-space: pre-wrap` is what keeps an edge space and a run of spaces alive through such a re-import, which is why neither flavor needs to carry an NBSP the other one does not. Consequence: no NBSP reaches either readable flavor, and the two decode to the same document text — a consumer's flavor choice cannot change what the user pasted. `application/x-lexical-editor` stays unnormalized, so a paste back into a Standard-view editor round-trips the display form exactly. A genuine data-NBSP is a literal `~` on screen (the USFM byte form PT9 also shows and copies), so it ships as `~` in every flavor. + +- **Byte fidelity around note-internal markers:** Copied text must not contain spaces the source USFM lacks. Live repro (2026-08-07): copying `\x - \xo 1:3: \xo*\xt 2Cor 4:6\xt*\x*Den` produced `… \xo* \xt 2Cor 4:6\xt* \x*Den` — phantom spaces after each closing marker (display-separator NBSPs leaking through the blanket NBSP→space mapping). The copy walker must be source-faithful, mirroring the serialization inverse, not a blanket `replaceAll`. + +- **An attribute the display does not show is still copied, whenever the file has bytes for it:** the same rule the note-caller bullet above states, applied wherever the display and the file disagree. A COLLAPSED note deliberately renders no `\cat` run (`createNote`, `usj-editor.adaptor.ts` — the category displays only in the editable EXPANDED layout, mirroring `\va`/`\vp`), so the copy walker contributes the file's own `\cat …\cat*` span itself, placed where the file has it (`\f + \cat People\cat*\fr 1:1 …`, with no trailing space — a space there re-tokenizes into a stray text child inside the note). A spanning TABLE CELL was a display gap rather than a view decision: USFM tables carry no pipe attributes at all, so a cell's width lives in its MARKER NAME (`\thc3-4`) and the tokenizer trims it off into `colspan` on the way in — the cell's display glyph now re-encodes it (`tableCellMarkerWithSpan`, shared with the `UnknownNode` byte builder), so what is shown, copied, and re-tokenized is the cell the file describes rather than a one-column cell. Both were found by sweeping the repo's real-world fixture (`2sa.usj.ts`) through the copy→paste round trip; both are pinned on the copied bytes in `clipboardCopyFidelity.test.tsx`. + +- **Multi-block selections:** One `\n` between blocks; a selection starting mid-paragraph omits that paragraph's own `\p ` glyph (matches P9's rendered-text copy). Pinned, not changed. + +- **Copying nothing leaves the clipboard untouched, and a read-only construct is copied by selecting it from outside:** a copy triggered by anything other than a real browser clipboard event (the Ctrl+C/Ctrl+X key handling in `ClipboardPlugin`, the context menu's Cut/Copy items, `EditorRef.copy()`/`.cut()`) dispatches `COPY_COMMAND`/`CUT_COMMAND` with a `null` payload, and a null payload has to be turned into a real clipboard event before anything can be written: `@lexical/clipboard`'s `copyToClipboard` appends a hidden placeholder element to the editor root, points the DOM selection at it, and runs `document.execCommand("copy")` to provoke one. Its own handler for that synthesized event declines an empty Lexical selection _before_ calling `preventDefault`, so the browser's default copy then runs against the DOM selection it was handed — the placeholder — and the user's real clipboard contents are replaced by a character that was never in the document. Live report (2026-08-26): trying to copy the marker text of an uneditable construct (`\fig`) yielded one stray character. The construct only makes it _likelier_: `UnknownNode` renders `contentEditable="false"` and its marker/attribute glyphs are `ImmutableTypedTextNode` decorators with `isKeyboardSelectable() === false`, so a click or drag inside one leaves the caret in the prose beside it instead of selecting anything — but a plain collapsed caret anywhere reaches the same path, in every view mode, and cut does too. **Rule:** a copy with nothing selected writes nothing at all. Enforced at two layers, both of which decide INSIDE the update where the selection is authoritative — a check in front of the dispatch would read the last COMMITTED selection, which Lexical is a microtask behind, and would silently no-op a `copy()` made in the same tick as the selection it is copying. (1) `registerEmptyCopyGuard` (`clipboard.utils.ts`, shared-react), a `COPY_COMMAND`/`CUT_COMMAND` listener `ClipboardPlugin` registers at `COMMAND_PRIORITY_LOW` — above `@lexical/rich-text`'s EDITOR fallback, which is what synthesizes the event, and below every feature handler. One registration covers every leg (the keyboard shortcuts, the context menu's Cut/Copy, `EditorRef.copy()`/`.cut()`), since all of them dispatch the same command against the same editor. (2) `$handleCopyForStandardView` CLAIMS a null-payload dispatch it cannot build data for, rather than declining it into that fallback. A real clipboard event is still declined by layer 2 at a collapsed caret (the browser already writes nothing, and declining leaves it un-`preventDefault`ed). A NodeSelection is claimed by neither layer — it has real content, and Lexical's own path copies it correctly. **How a read-only construct IS copied:** by a selection that CONTAINS it. Such a selection — measured at the Lexical level as two ELEMENT points on the parent paragraph either side of the block — reaches every one of the block's display glyphs through the same walker as ordinary prose, so it copies the construct's real USFM bytes; no node-selection state over these constructs exists to recover, and none was invented. **Unverified assumption:** that a real drag or click over a `contentEditable=false` block is what a browser resolves INTO that selection shape (on the reasoning that it will not place a selection endpoint inside one). jsdom cannot observe DOM-range → Lexical selection resolution — the same boundary S6 already records for the WI-2 quirk — so this needs a real-browser check before it is treated as fact. If a browser instead leaves the caret collapsed beside the block, the leak is still closed (nothing is written) but the user gets nothing rather than the figure, which would be a selection-affordance gap to file separately. Pinned in `whitespaceDisplay.plugin.utils.test.tsx` (`"copying an empty selection leaves the clipboard alone"`, including the node-selection carve-out, plus the figure-alone copy), `CommandMenuPlugin.gate.test.tsx` (the same rule in a hidden-marker view where no Standard-view handler is registered at all, and through `EditorRef.copy()`/`.cut()`), and shared-react's `ClipboardPlugin.test.tsx` (the shortcuts, and the same-tick select-then-copy case) and `ContextMenuPlugin.test.tsx` (the menu leg). + +- **Optbreak (`\optbreak`, displayed as the literal token `//`) is a non-backslash marker token the copy walker must respect like any other content:** `$selectionToUsfmText` contributes an optbreak's `//` via its `ImmutableTypedTextNode` display child (the `$isDecoratorNode` branch), with no padding added on either side — the significant flanking spaces PT9 preserves byte-for-byte around an optbreak come entirely from the surrounding plain-text nodes, never from the walker itself, and since every TextNode's NBSP is inverted to a plain space unconditionally (the same rule as any other content), `text/plain` never carries an NBSP adjacent to `//`. Pinned in `optbreakClipboardFidelity.test.tsx`'s "copy characterization" describe (2026-08-13). + +- **An opaque construct copies out on ONE line:** the walk's ordinary rule puts a `\n` at every non-inline element boundary it crosses, which would spread a single construct over several lines — a sidebar's nested `\p`, and every row and cell of a table, are real block-level nodes. Pasting that text replays each `\n` as a paragraph split, and Tier 2 re-tokenizes strictly one paragraph at a time, so the `\esb`/`\esbe` and `\tr`/`\th`/`\tc` assembly the fragment tokenizer already implements never saw the whole construct in one pass. `$startsBlockLine` (`whitespaceDisplay.plugin.utils.ts`) therefore suppresses the line break for a block boundary with an opaque ancestor ABOVE it — the construct's own outermost node still starts its own line, so a table stays separated from the prose around it. USFM markers self-delimit, so the one-line form is valid USFM of the same document; it is simply not the line-per-marker layout a USFM writer emits, which is the deliberate trade for a paste that survives. Pinned by the corpus sweep's now-unskipped `"sidebar (esb)"` and `"table with header and cells"` fixtures and by `unknownClipboardFidelity.test.tsx`. A `\periph` peripheral division needs the rule just as much, for the same reason and with nothing periph-specific about it: the division marker carries no closing bytes at all, so what ends its attribute run and what opens its content are the SAME thing — the next marker — and both spellings tokenize identically (pinned in `usfmFragmentToUsj.test.ts`). It is the paste's line splitting, not the byte layout, that decides whether one re-tokenization pass ever sees the division together with the blocks inside it; measured both ways and pinned in `unknownClipboardFidelity.test.tsx`. Tables needed one more thing on the paste side: `usfm.sty` classifies `\th1`/`\tc1`/… as CHARACTER styles, so with a real project stylesheet a cell marker reached the fragment tokenizer as a char token and its table-cell assembly — which only ran for paragraph tokens — never saw it, turning every cell into a loose `char` span inside an empty row. ParatextData derives a cell from the marker NAME, so the assembly now recognizes a cell marker on either token kind, guarded on an open `\tr` row. Pinned sheet-backed (`createMarkerLookup(defaultStyleInfo)`, where the cell markers really are Character) in `usfmFragmentToUsj.test.ts`'s `"cell markers under a real stylesheet"` describe, including the align-infix names (`\thc3` → center, `\thr5` → end) and both guards — a cell-named span with no open row stays an ordinary char span, and a nested `\+tc1` is never a cell. + +### S2. Paste (Standard view, internal) — same-namespace `application/x-lexical-editor` payload reconstructs the exact node tree (existing) + +**Fixed gap (2026-08-13, extended to every kind 2026-09-10), lexical-JSON flavor only:** the "exact node tree" promise did not hold for a selection containing an `UnknownNode` (figure, sidebar, periph, ref, optbreak). `@lexical/clipboard`'s JSON generator (`$appendNodesToJSON`, the function behind the `application/x-lexical-editor` flavor) computes node exclusion from `currentNode.excludeFromCopy('html')` — the literal string `'html'`, hardcoded for EVERY copy-out destination `$appendNodesToJSON` itself handles, not only actual HTML generation (`'clone'` is never passed by any Lexical-shipped code path in the installed version). This change touches ONLY that flavor, and doubly so: Standard view's `text/html` is not a DOM export at all (S1 — it renders the copy walker's own USFM), and even where Lexical's exporter does run, `$appendNodesToHTML` (`@lexical/html`) computes the same `excludeFromCopy('html')` value but returns early on `UnknownNode.exportDOM()`'s unconditional `{element: null}` BEFORE ever consulting it. + +`UnknownNode.excludeFromCopy` returned `destination !== "clone"`, excluding every `UnknownNode` unconditionally from the lexical-JSON flavor. An excluded node is not dropped silently — `$appendNodesToJSON` hoists its own children into the parent's list in its place. For an optbreak, whose only child is the `//` `ImmutableTypedTextNode` display token (a content-free `DecoratorNode` with no meaning once separated from its owning `UnknownNode`), that stranded a loose decorator on paste: `$parseSerializedNode` reconstructed the bare decorator, not a recognized optbreak, and nothing re-tokenizes a decorator's text — the discretionary line break silently vanished. Live report (TJ, 2026-08-13, most likely a pre-branch or parent-branch build): copying a selection containing an optbreak put `//` on the clipboard correctly, but pasting that same clipboard back did not restore it. + +Fixed by narrowing `UnknownNode.excludeFromCopy` (`UnknownNode.ts`) to leave a CHILD-BEARING "optbreak" out of the exclusion (a genuinely childless live optbreak — an emptied husk mid-settle — still falls back to excluded, matching `text/plain`'s "nothing here" for that shape), plus an `isSelected` override closing a SECOND, independent carrier-disagreement gap measured while verifying the fix: a selection whose ending boundary resolves to an ELEMENT-type point ON the construct at offset 0 (touching the wrapper without covering any of its content) still marked a CHILD-BEARING node "selected" under Lexical's default `isSelected`, producing an entry in the copy that `text/plain` has nothing corresponding to — `excludeFromCopy` alone cannot see this (it has no visibility into which children a given selection will include), so the override closes it at its actual source, mirroring `$selectionToUsfmText`'s own `getNodes()`-based walk so both carriers agree. The override tests child MEMBERSHIP in `getNodes()`, deliberately, and NOT the narrower "does the selection cover a child's content" — the two differ for exactly one shape, and the narrower rule is the worse of the two. That boundary reaches the children two different ways depending on what the first child IS: a construct whose display bytes lead (a figure's `\fig ` glyph, an optbreak's `//`) starts with a DECORATOR, so the element point stays one, no child is in `getNodes()`, and both carriers agree; a construct with no display bytes (a `ref`, whose container USFM never carried) starts with a real `TextNode`, and Lexical normalizes the same point into a TEXT point at that child's offset 0 — the child is then in `getNodes()` contributing zero characters, so `text/plain` emits nothing while this predicate still answers true. Answering false there was measured, and it is a SILENT STRUCTURAL LOSS rather than the agreement it looks like: excluding the wrapper makes `$appendNodesToJSON` hoist its children in its place, `createUnknown` stamps `mode:"token"` on every text child, and `$sliceSelectedTextNodeContent` refuses to slice a token node — so the copy carries the construct's characters with the node and its attributes gone (`[verse, "See ", {"type":"text","mode":"token","text":"Genesis 1:1"}]`, no `ref`, no `loc`), which is precisely the convincing-lie hazard this pair exists to prevent. Membership keeps the construct whole (`[verse, "See ", {"type":"unknown","tag":"ref","unknownAttributes":{"loc":"GEN 1:1"},"children":["Genesis 1:1"]}]`), making the lexical flavor a SUPERSET of `text/plain` at that one boundary. That residual is what the flavor-omission rule below closes, at the layer that can see it: a boundary ON a construct is a selection reaching INTO one, so no lexical flavor is written for it at all and there is nothing left to disagree. Closing it WITHIN the JSON generator would still need a lever Lexical does not offer — `exportNodeToJSON` requires every ElementNode's `exportJSON()` to return a `children` array, so a node cannot say "drop me AND my children". Both rules key on the node's own CHILD COUNT rather than on its kind, so they cover every `UnknownNode` construct alike — the kinds share one display-decorator shape (`unknownDisplayParts` builds each kind's marker/attribute bytes as the same content-free `ImmutableTypedTextNode`s), and so shared one hoisting failure. Pinned end-to-end for optbreak (all three real-world payload shapes: plain-only, plain+html without the lexical flavor, and the full sync/native payload carrying it; plus both childless-shape edges — a genuine husk, and the boundary-selection case) in `optbreakClipboardFidelity.test.tsx`, per kind for figure/sidebar/periph/ref across the same three shapes in `unknownClipboardFidelity.test.tsx` — whose own `"a construct's own boundary"` describe pins what each of those boundaries now puts on the clipboard — text carriers only, for a decorator-led kind (`figure`) and a text-led one (`ref`) alike, with the selection at the construct's start and extended over its first child — rather than leaving the shared predicate inferred from optbreak alone. Every one of those assertions reads the construct's own CONTENT BYTES, never just its tag — a tag-only assertion cannot see a copy that dropped the wrapper while hoisting its characters out, which is exactly how the lossy variant passed review once — plus direct unit pins on `excludeFromCopy` in `UnknownNode.test.ts`. Tables are NOT part of this: `table`/`table:row`/`table:cell` are their own `ImmutableTable*` nodes now, never `UnknownNode`s, so they never took the exclusion at all — measured, and pinned in the same per-kind sweep. + +**The internal flavor is written only for a selection that can carry a construct WHOLE:** `$getStandardViewClipboardData` (`whitespaceDisplay.plugin.utils.ts`) omits `application/x-lexical-editor` outright whenever an END of the selection lies inside an opaque construct (`$selectionReachesIntoOpaqueBlock`, `OpaqueBlockGuardPlugin.tsx` — a caret parked in one, or a range that reaches in and stops partway through). The flavor exists to carry a construct WHOLE, and a selection cutting through one cannot be carried whole: `createUnknown` (`usj-editor.adaptor.ts`) builds a construct's text children in token mode, and `$sliceSelectedTextNodeContent` (`@lexical/selection`) refuses to slice a token-mode TextNode, so it carries the node's full text or none of it. A figure caption selected from its third character to its seventh therefore serialized as the COMPLETE figure — wrapper, `file`, `size`, and the caption's whole text — while `text/plain`/`text/html` carried the four selected characters; a native paste of that clipboard inserted a second figure where the user asked for four letters, and a save persisted the invented bytes with no error. The text flavors have no such floor: they are exactly the selected bytes, and Tier 2 re-tokenizes them back into whatever construct those bytes actually spell (an unclosed `\fig` becomes a char span, not a figure). So a cut-through selection ships those two and nothing else, and the carriers agree by there being only one kind of carrier; a selection whose ends are both OUTSIDE still ships the flavor and still takes the node-tree fast path. + +A consequence recorded rather than traded away: a `ref` holds exactly one child, so a boundary at its own END covers its content whole yet is still INSIDE it, and the copy loses the `ref` wrapper and its `loc` (USFM has no bytes for the container). The rule keys on where the selection's ENDS are, not on how much of the construct they enclose. That shape is not what a drag produces — `UnknownNode` renders `contentEditable="false"`, so a browser resolves a drag ending "at the ref" to a point beside it rather than inside — and the loss is asserted, not hidden, in `unknownClipboardFidelity.test.tsx`'s `"a construct's own boundary"` describe. The rule itself is pinned in `partialConstructClipboard.test.tsx`: both cut-through shapes (caption-interior, and prose-into-caption), the whole-construct control that must keep the fast path, and the async-clipboard shape a real Ctrl+V delivers. + +Which paste paths the flavor reaches is worth stating, because it is why this over-carry never showed up in hand testing: the keyboard Ctrl+V and the context-menu Paste both go through `pasteSelection` (`clipboard.utils.ts`, shared-react), which rebuilds a `DataTransfer` from `navigator.clipboard.read()` — an API that exposes only sanctioned MIME types, so the private flavor is never in it and those pastes always take the text path. The flavor reaches a paste only through a genuine NATIVE paste event (an OS/Electron Edit-menu "Paste", a browser-delivered `paste`) or a programmatic `PASTE_COMMAND` carrying one. Which paste path a host uses is not something this editor controls, so the rule is enforced at the COPY. + +Otherwise unchanged, EXCEPT: a paste whose selection touches an attribute display run (a char span's `|attrs` +list, a milestone's attribute run, or a verse's `\va`/`\vp` value) never takes this path, even when +the clipboard carries the same-namespace flavor — attribute-context paste always claims plain-text +insertion instead (see S3's "Paste inside an attribute display run" subsection below). This closes a +gap the fact recorded in S3 ("the private Lexical flavor is dead on Ctrl+V") does not cover: that +fact is about the RECONSTRUCTED `DataTransfer` a programmatic/reconstructed paste dispatch uses +(`clipboard.utils.ts`'s `pasteSelection`), not about a genuine LIVE NATIVE `paste` event, whose +`clipboardData` is the browser's own and can still carry the flavor for a same-page/session copy — +this is the same "live native paste event that still has it" case S2's own node-tree fast path is +written to keep for ordinary content. + +### S3. Paste (Standard view, external) — plain text IS the fidelity carrier + +Any paste without a same-namespace Lexical payload is treated as a USFM text fragment: take `text/plain` (fall back to text derived from `text/html` with block boundaries → `\n`), normalize NBSPs **positionally**, insert as text, let Tier 2 re-tokenize markers. + +- **One extraction layer for every paste claim:** `getPastePayload` (`whitespaceDisplay.plugin.utils.ts`) is the single place a `PASTE_COMMAND` payload is read. It performs the jsdom-safe clipboard duck-check, decodes `text/html` through `htmlPasteText`, normalizes `\r\n` and bare `\r` to `\n` BEFORE any caller tests for a line break, and reports whether the clipboard carries this editor's own `application/x-lexical-editor` flavor. The carrier rule it applies is three-way: a **Paratext 9** clipboard's html is decoded to USFM and WINS over that clipboard's own `text/plain` (see "Paste from Paratext 9" below); every other source's `text/plain` wins whenever it carries anything; a clipboard with only `text/html` falls back to that html's decoded text. Four handlers race on `PASTE_COMMAND` — the in-note `\fp` claim at CRITICAL, this Standard-view claim and the character-stack line replay at HIGH, the paragraph-split arm at LOW — and they must agree byte-for-byte on what was pasted, so none of them reads the clipboard itself. What each claim DOES with `isInternal` still differs by design (the in-note claim covers internal pastes; the Standard-view claim declines them unless the selection touches an attribute run). + +- **Paste from Paratext 9 — the html carrier is decoded, because P9's own `text/plain` is strictly less faithful:** P9 writes `text/plain` as the selection's VISIBLE text and keeps the USFM its own paste reads in `CF_HTML`, as escaped `` comments inside spans it marks `exclude` (see "Paratext 9 Reference Behavior" below for the exact shape). A collapsed note therefore renders — and copies as plain text — as its caller GLYPH alone (`a`, or `*` for a `-` caller), so reading the plain carrier pasted one stray character where a footnote belonged. `paratext9HtmlToUsfm` (`paratext9Clipboard.utils.ts`) implements P9's reverse XSLT (`Standard-Reverse.xslt`) over the DOM — `text()` bytes, `usfm:` comments unescaped with newlines flattened to spaces, `exclude`/`include` modes, `div`/`tr`/`br` newlines, `span[class="attribute"]`, and the `usfmopen`/`usfmclosed` class encoding with its `nested` `+` prefix — and every NBSP becomes a plain space, which is P9's OWN copy rule (its stylesheets emit display NBSPs after every opening marker, and a user-supplied non-breaking space reaches the clipboard as the `~` byte). Newline runs are then collapsed and trimmed exactly as `htmlPasteText` does, so a P9 paste splits into lines like any other multi-line paste. The decoder's SIGNATURE is what keeps this from becoming a general "prefer html" rule: it answers `undefined` unless the fragment carries a `usfm:` comment or an element with both a `usfm_` class and `usfmopen`/`usfmclosed`. Two deliberate consequences of that narrowness: a P9 STANDARD-view fragment with no note in it matches neither clause and keeps using its `text/plain` (P9 renders every marker there as literal text, so the two carriers already agree); and P10's own html never matches, since no P10 node emits `usfmopen`/`usfmclosed` and P10 writes no `usfm:` comments — even though P10's own DOM export does stamp `usfm_` classes, which is why the class clause requires both halves. Not implemented: P9's ruby-gloss reconstruction (`span[@id^='ruby:']`), which needs project ruby settings, and the `@colspan` re-encoding P9 applies to a `usfmopen` table cell. + +- **A multi-line payload is replayed line by line through `INSERT_PARAGRAPH_COMMAND`**, not through a bare `selection.insertParagraph()` per newline. Going through the command is what makes a paste landing inside a character-style stack close and reopen that stack at every line break, exactly as Enter does — a raw split tears the span, dropping its closing marker and leaving every line after the first outside the reopened style. The selection is removed up front rather than relying on the first `insertText` to replace it, so a payload whose first line is empty (a leading newline) cannot split a still-selected range. `MarkerEditContext.splitExpected` is armed before the first line goes in — the command's own handler arms it for each dispatch, but the first line is inserted before any dispatch runs — so every fresh paragraph the paste creates gets the marker prefix that keeps it from being read as marker-deleted and merged back. A line that ALREADY opens with its own paragraph-marker literal is the one exception: the prefix is not injected there (`$paraMarkerDeletionTransform`, `markerEditDeletion.utils.ts`), because the marker the user pasted is the paragraph's marker and a `\p ` in front of it would settle as a stray empty paragraph. `\b` is where that shows, since a blank-line marker carries neither content nor a separator. Foreign HTML formatting is dropped by design — Standard view is markers-as-text, exactly like P9's Standard view where the reformat pipeline re-tokenizes everything. This generalizes the existing NBSP-gated handler and kills both live-observed corruptions plus the latent doubled-glyph path (re-imported Standard-view HTML producing a `CharNode` via `data-marker` importDOM _and_ literal `\nd` text, since MarkerNode has no importDOM). + +- **A paste inserts what was pasted; it never edits the host paragraph's marker.** A whole-paragraph copy carries its paragraph's own `\p ` literal (S1), so pasting one at an existing paragraph's content start puts two paragraph-marker occurrences on the line, and Tier 2 splits it — leaving the host behind as an empty paragraph ahead of the pasted one. That is what Paratext 9 does with the same bytes (`NormalizeTokenUsfm`, `ParatextData/UsfmToken.cs`, emits a line break before every Paragraph token), and it is what typing the same bytes does here, so paste needs no rule of its own and has none. An earlier build deduplicated that pair by dropping the HOST's glyph, which silently retagged the user's paragraph — `\p` became `\q1` — and required a whole paste-provenance apparatus (a per-commit armed flag plus a set of pended node keys, to carry "this rebuild came from a paste" across a deferred settle) to keep it out of typed input. Both are gone. + +- **Positional NBSP normalization (replaces the blanket NBSP→`~`), three-part rule:** (1) a leading NBSP — string-start or right after a newline — is a structural separator with nothing in front of it to match against and becomes a space; (2) an NBSP immediately FOLLOWING a marker token (`\marker`, nested `\+marker`, either's closer, or a milestone's anonymous `\*` self-closer) is the required opener/closer separator and becomes a space; (3) an NBSP immediately PRECEDING a marker token is a structural spacer with no source counterpart and is DROPPED entirely (neither spaced nor kept as data) — `createNote` (`usj-editor.adaptor.ts`) appends a spacer after EVERY note child, not just the first, so one sits directly before `\ft`/`\f*` and every other child after the caller; a browser-hop/html-derived copy of a collapsed note therefore carries structural NBSPs on BOTH sides of its interior markers, not only after the opener. Every remaining NBSP is user data → `~`. This is P9's `PostprocessUsfm` model. Live repro that mandates it (2026-08-07): pasting P10's own copied footnote back produced `\f~ \fr~1:1 ~ \ft~Caller test.~ \f*` (every display-NBSP became `~`, breaking marker recognition); a browser-hop paste of `\nd …\nd*` produced `\nd~light … \nd*` with an unmatched pair. **Rule precedence when both position rules match the same NBSP:** a closed-children note shape (`\xo*` immediately followed by NBSP immediately followed by `\xt`) sits after a closing marker AND before the next opening marker at once; the after-marker rule wins and the NBSP becomes a space rather than being dropped, because `normalizePastedNbsp`'s `.replace` chain applies `AFTER_MARKER_NBSP` before `BEFORE_MARKER_NBSP`, consuming the NBSP into a space before the before-marker pass ever sees it. The tie-break is correct, not incidental: an after-closer NBSP in ordinary body text (`\nd*and`) is a real content space that dropping would silently lose, and the case is reachable only via a foreign NBSP-preserving carrier — P10's own `text/plain` copy (S1) never emits an NBSP in this position to begin with. + +- **Optbreak (`//`) is a non-backslash marker token the positional NBSP rule does not recognize (characterized, not a live gap):** `AFTER_MARKER_NBSP`/`BEFORE_MARKER_NBSP` (`whitespaceDisplay.plugin.utils.ts`) only match `\`-shaped tokens, so an NBSP adjacent to `//` is not treated as display whitespace and falls to the blanket `~` rule like any other interior NBSP. This is UNREACHABLE via P10's own copy, and DOUBLY so: (1) S1 above establishes `text/plain` never carries an NBSP next to `//` to begin with, and the plain payload is always preferred when present; (2) P10's own `text/html` is those same bytes (S1), so even with `text/plain` absent the decoded html carries the `//` with the same plain flanking spaces and nothing for the positional rule to mistake. Reachable only via a genuinely foreign clipboard source supplying its OWN `text/html` (never P10's) with no `text/plain` at all: a synthetic `

before // after

`-shaped payload settles to `before~// after` — the optbreak token itself still recognizes correctly (the tokenizer's `//` split is a plain string `.split("//")`, unaffected by an adjacent `~`), and the NBSP itself is genuine data FROM that foreign source (not a byte P10 invented) — the `~` it becomes is the correct display form for real data-NBSP under the "every remaining NBSP is user data → `~`" rule above, just landing one position earlier than a marker-adjacent rule would recognize. Characterized (not extended/fixed — no live repro reaches it) in `optbreakClipboardFidelity.test.tsx`'s "copy characterization" describe (2026-08-13). + +- **The private Lexical flavor is dead on Ctrl+V — mechanically verified, not inferred:** `pasteSelection` (`clipboard.utils.ts`) rebuilds its `DataTransfer` from `navigator.clipboard.read()`, Chromium's async Clipboard-API read. That API, with no `unsanitized`/custom-format opt-in (none is used here), exposes only a fixed, sanctioned MIME allow-list (`text/plain`, `text/html`, and a short list of others) — `application/x-lexical-editor` is not one of them, so the rebuilt `DataTransfer` a real Ctrl+V dispatches can **never** contain it, by construction of the read API itself. This is a static fact about the code path (confirmed by reading `clipboard.utils.ts`), not an inference from the live tilde-corruption symptom — that symptom is merely consistent with it. Consequence: the USFM text carrier IS the internal path too — acceptable once S1's copy is byte-faithful — and the sync `ClipboardEvent` path keeps the node-tree fast path when the flavor is present (S2). + +- **Structure-protected documents (Simple interface mode) get the same BYTES and a different MECHANISM:** `$handlePasteForStandardView` handles a paste in a `structureProtectionMode: "protected"` document itself, applying every byte rule above — the `\c`/`\id` strip, the positional NBSP mapping, the Paratext 9 html decode, the attribute-context path. Protection changes exactly two things. (1) A selection `StructureKeyboardPlugin` refuses to replace — `$shouldBlockSelectionReplacement` (`structureKeyboard.utils.ts`): a range spanning a paragraph boundary, or one containing a verse marker, the mutable `VerseNode`'s own caret included — is DECLINED here, so that refusal keeps exactly one owner. Both plugins register `PASTE_COMMAND` at `COMMAND_PRIORITY_HIGH` and the marker engine mounts first, so claiming such a paste would starve the sanitizer's block. (2) A multi-line payload never splits a paragraph: each `\n` becomes a single space and the whole payload goes in as ONE `insertText`, with no `INSERT_PARAGRAPH_COMMAND` and no `splitExpected` arming — the same convention `$sanitizeNodesForProtectedStructure` uses for a boundary it removes. + + Declining the whole paste instead — handing it to the sanitizer's `$sanitizeAndInsert`, which reads `text/html` and nothing else — made the protected mode STRICTLY LESS safe than the unprotected one on three counts: a pasted `\c 7` never reached the strip, so it created a second chapter node and poisoned every subsequent save (the exact corruption the strip exists for, reachable only under protection); NBSPs never reached the positional rule; and a P9 clipboard's `usfm:` comments were never decoded, so a P9 footnote arrived as the caller glyph its `text/plain` shows. It was never protecting this view from marker BYTES either: the marker engine has no protection gate, so a pasted or typed `\v`/`\p` literal tokenizes into a real marker in both modes, and `$sanitizeNodesForProtectedStructure` only strips verse/para NODES out of an html DOM import — a shape Standard view's own `text/html` no longer carries at all (S1). So protection in Standard view governs selection replacement and paragraph splitting; marker bytes are content here. + + One residual belongs to the refusal rule rather than to this handler: a collapsed caret at the end of the editable `\v ` glyph counts as containing a verse marker, so a paste there is refused, while the same paste at offset 0 of the prose immediately after it — the same screen position — succeeds. + +- **Marker-bearing lines own their markers:** A pasted line starting with a paragraph-marker literal does NOT also get the host paragraph's cloned prefix (no doubled/empty paragraphs). Marker-free lines inherit the host marker. **Live-verified correct on c7e666fa (2026-08-07)** — `\p one\n\p two` and `tail\n\q1 line` both produced exactly the target structure. + +- **Pasted `\c` / `\id`:** Strip during paste normalization (approved). Live-verified harm (2026-08-07): pasting `\c 2` mid-chapter put a chapter node in the editor and poisoned the save loop — PDP rejects every save with "Multiple chapter markers present", the error surfaces only in the renderer log, disk and other editors silently stop updating. (P9 destroys pasted `\c` at save with a user-facing error.) + +- **Paste inside an attribute display run — paste ≡ typing:** a selection that TOUCHES attribute-display text at either end — a char span's bare `|attrs` list, a milestone's attribute run, or a verse's `\va`/`\vp` value — is inserted exactly as the SAME characters typed at the SAME caret (or over the same selection) would be: one `selection.insertText`, with each `\n` becoming a single space, per-newline, not run-collapsed (attribute values are single-line; there is no multi-line attribute byte shape to collapse INTO — `"a\n\nb"` pastes as `"a b"`, two spaces). A selection that only PARTLY sits inside attribute-display text (one end in, one end out) takes this same INSERTION MECHANISM — see below for the two corruption shapes that earns it. The existing attribute pend/settle machinery (`$textNodeTier2Transform`'s attribute-tagged early return, `$resolvePendingMarkers`) then re-tokenizes the displayed bytes back into node state identically whether they arrived by typing or paste. + + **The two BODY-content paste rules are skipped only for a selection wholly inside ONE attribute node** — the `\c`/`\id` strip and the positional NBSP mapping, which are the only two places a paste deliberately diverges from typing the same bytes. What earns that carve-out is paste ≡ TYPING: the bytes land where the same keystrokes would land, and the attribute-context contract is that a paste in a value behaves as typing there does. It is NOT that the bytes never re-tokenize — measured, that is false. The `"attribute"` tag survives the insertion, but the caret-departure settle re-tokenizes the paragraph, so a pasted `\c 5` in a value DOES become a chapter marker, and a pasted interior NBSP is read as display whitespace and saved as a plain space. Both happen IDENTICALLY when the same bytes are TYPED at the same caret, which is why this is the typed-`\c` hole recorded as Deferred item 2 — inherited by the paste path, not introduced by it. Extending the strip to cover it would eat bytes out of an attribute value that were never a chapter token — the very regression the carve-out exists to prevent — and would break the equivalence. Both halves are pinned SETTLED, on concrete USJ, in `attributeContextPasteFidelity.test.tsx`. A selection that merely TOUCHES an attribute run does not have that property, and pretending it does was a real data-fidelity hole: removing such a range takes the bytes out of the run — and when the range covers a char span's closing glyph it deletes the closer, so the whole span re-tokenizes around whatever just arrived. Measured both ways round (a range starting in body text and reaching INTO the run, and one starting inside the run and reaching PAST the closer), pasting `\c 5` over such a selection produced a real chapter node mid-paragraph with the rest of the paragraph stranded outside it — the exact save-loop poisoning the strip exists to prevent — and a pasted interior NBSP was read as display whitespace and saved as a plain space instead of surviving as data. So a touching-but-not-wholly-inside selection gets body content's rules for the bytes, while keeping the attribute path's insertion mechanism. One deliberate residual comes with that split: a MULTI-LINE payload landing in body content this way still collapses per newline rather than splitting the paragraph, because splitting a paragraph whose char span the removal has just cut in half is the corruption this path exists to avoid. + + TJ's live repro (filed 2026-08-11, against a pre-branch build): existing span `\nd asdf|who="hi"\nd*`, caret at the end of the `who="hi"` run, paste plain text `sid="things"` — the `who` attribute display and the closing `\nd*` glyph both vanished from the editor, the pasted text rendered outside the span, and the saved file diverged from the editor. `sid="things"` carries no NBSP; the pre-branch build's paste handler still had its OLD NBSP-gated form (see the "positional NBSP normalization" bullet above — that gate was generalized to claim every external paste, NBSP or not, on 2026-08-07), so the most plausible mechanism is that OLD gate declining an NBSP-free paste outright and falling through to Lexical's own default rich-paste node insertion — NOT a same-namespace `application/x-lexical-editor` flavor on the clipboard (this document's own "private Lexical flavor is dead on Ctrl+V" fact, above, still holds for the reconstructed-`DataTransfer` paste path this repro most likely used). + + What matters for the current code is the SHAPE the corruption takes once ANY handler declines an attribute-context paste to Lexical's default rich-paste node insertion — reproduced directly here (`attributeContextPasteFidelity.test.tsx`'s "root cause" describe): it has no notion that an attribute run's text must stay inside its one tagged TextNode, and merges the run, the closing glyph, and even the FOLLOWING paragraph's sibling text into one plain node, destroying the attribute display and the closing marker and leaving the pasted bytes loose in body content. On the current code (a single-line, NBSP-free, flavor-free external paste is already safe since 2026-08-07's generalization), the confirmed regression classes this paste-≡-typing rule closes are: (1) a live native paste event that still carries a same-namespace `application/x-lexical-editor` flavor (S2's own documented case for when that reaches a real handler); (2) a multi-line plain-text payload, which the ordinary pipeline would split into real paragraphs via `INSERT_PARAGRAPH_COMMAND`; (3) a marker-bearing payload (`\c 5`) landing wholly INSIDE one attribute node, where the ordinary pipeline's `\c`/`\id` strip would eat bytes out of an attribute VALUE that were never a chapter token (bytes merely TOUCHING a run do get that strip — see the body-rules paragraph above); (4) a MIXED selection (one end in the run, one end out) combined with either (1) or (2) — the selection touches attribute context, so it must not reach either risky branch merely because its other end sits outside. + + Pre-existing precedence, UNCHANGED by this rule: the CRITICAL-priority in-note multi-line `PASTE_COMMAND` claim (`MarkerEditPlugin.tsx`) still runs before the Standard-view external-paste handler and still wins for a multi-line payload whose selection touches EXPANDED note content — an attribute run that happens to sit inside an expanded note's content is reached by this rule only when that in-note claim itself declines. See `$handlePasteForStandardView`'s doc comment (`whitespaceDisplay.plugin.utils.ts`) for the full mechanism. + +### S4. Paste-as-plain-text (Ctrl+Shift+V / context menu) — narrows the payload to `text/plain` + +Under S3 this is semantically identical to a normal external paste in Standard view. There is NO "paste literally, don't tokenize" mode; P9 has none either (no Paste Special exists in P9). Live-confirmed identical 2026-08-07. Documented + pinned as equivalence, no new UI. TJ note: the equivalence is Standard-view-scoped — other views may legitimately differentiate the two commands later; out of scope here. + +### S5. Hidden-marker views (`formatted`, `paragraph-structure`) + +- **Copy-out is prose** (no marker text) — existing, gets a gate test protecting Standard-view handlers from leaking there. + +- **The paste gate** (swallow anything containing `\` **or `/`**) is over-broad — it eats URLs, dates, "and/or" (live-confirmed 2026-08-07). **Pre-existing upstream behavior** (`CommandMenuPlugin` on `origin/main`, predates the standard-view branches) — per TJ, NOT fixed here; recorded in the semantics doc's deferred list. Structural apply-markers-on-paste in formatted views likewise out of scope. + +### S6. Cut = copy + `removeText()` (existing) + +Including the empty case: a cut with nothing selected writes nothing and removes nothing, by the same rule and the same two enforcement layers as copy (see S1's "Copying nothing leaves the clipboard untouched"). + +The WI-2 filed quirk — a selection-delete ending exactly at a just-settled char-span boundary absorbing one adjacent character — gets a targeted regression pin. **Outcome (2026-08-07):** does NOT reproduce at the Lexical selection level. Pinned as regression armor for both `CUT_COMMAND` and a plain `removeText()`, each given a selection whose focus is an element point mirroring DOM `range.setEndAfter(spanElement)` — the exact boundary shape the live repro's programmatic DOM selection used. Both variants produced byte-exact, correctly-scoped results, narrowing the live repro's root cause to the DOM Range → Lexical selection-resolution layer (`applyDOMRange`/`$internalResolveSelectionPoints`), which is not exercised by a Lexical-level `RangeSelection` and so remains covered only by the E2E selection steps. + +### S7. Undo — every paste, including rebuilds it triggers, is one undo step + +This holds for BOTH paste shapes, and the earlier qualification recorded here no longer applies. + +- A paste whose rebuild happens SYNCHRONOUSLY, within the paste's own commit (a terminated marker, or a marker-bearing line's Tier 2 rebuild via `armPasteRebuildDedup`) is one commit and one undo step, as it always was. +- A paste whose literal text PENDS and only re-tokenizes on a LATER caret departure (e.g. a bare `//` with no terminator) is still TWO commits — the insertion and the departure-triggered settle — but only ONE undo step, because a settle is never its own history entry: it merges into the entry holding the edit it completes. A single press therefore lands on the true pre-paste state rather than stranding the user on a half-settled literal they never typed. Which carrier the clipboard happened to provide (plain payload vs. the same-namespace Lexical flavor) is invisible to undo. + +The multi-line `INSERT_PARAGRAPH_COMMAND` replay does not change this either: every dispatch and every `insertText` runs inside the ONE `editor.update()` the `PASTE_COMMAND` dispatch is already in, so a three-line paste is still a single history entry. + +Measured, not assumed: pinned for optbreak in `optbreakClipboardFidelity.test.tsx`'s "undo after a PLAIN-payload paste is ONE step" and "undo after the lexical-flavor paste restores the pre-paste (empty host) USJ in one step" pins, and for the synchronous shape in `markerPasteFidelity.test.tsx`'s `undoAndSettle` pins. + +--- + +## Paratext 9 Reference Behavior + +Understanding P9's clipboard model provides context for P10's design decisions: + +- **Data formats:** P9 writes three clipboard formats (`CopySelectionToClipboard`, `HtmlEditor/FirefoxHtmlEditor/IHtmlEditorCopyPaste.cs`): (1) `text/plain`, the selection's VISIBLE DOM text with every NBSP replaced by a space; in Standard view markers are rendered as literal text, so plain-text copy includes them; (2) `CF_HTML` (or `text/html`), the fidelity carrier — the selected range's serialized DOM fragment wrapped by `HtmlAdaptorUtils.FormatAsHtml`'s `…`; (3) a private `ParatextLanguageId` format for app-internal use. + +- **The exact html shape P9's Standard view copies**, and what P10 does with it: a paragraph is `
\p …
` (the separator NBSP is a sibling of the glyph span, not inside it), a verse `\v 1 `, a char style `\nd …\nd*`, an attribute list `|src="…"`, and a COLLAPSED NOTE `a` — the note's real bytes exist ONLY in that `usfm:` comment, the visible `a` is the rendered caller glyph (a `-` caller renders as `*`), and the `exclude`class is what tells P9's own reverse XSLT to drop the glyph while still reading the comment. Comment escaping is`XsltExtensions.EscapeComment`: every character except `a-zA-Z`becomes`%`plus four uppercase hex digits, because an html comment may contain neither`--`nor`>`. **P10 decodes this** (`paratext9HtmlToUsfm`, S3's "Paste from Paratext 9" above), so a P9→P10 footnote paste lands as a real note instead of the caller glyph its `text/plain`carries. P9's FORMATTED views encode markers in`usfmopen`/`usfmclosed` classes instead of as literal text; the decoder implements that rule too, but no copy from those views has been verified against it. + +- **Paste workflow:** P9's paste is HTML-first and unsanitized. All normalization happens in a subsequent reformat pipeline: `CleanHtml` → Unicode NFC/NFD per project settings → reverse XSLT → `PostprocessUsfm` (removing FEFF marks; applying NBSP policy — token-leading NBSP → space, interior kept per `AllowInvisibleChars` setting) → `UsfmToken.NormalizeUsfm` (full re-tokenization: whitespace collapse, newlines inserted before paragraph/verse markers, RTL mark handling). + +- **Paste Special:** P9 has no Paste Special command. Both normal paste and plain-text paste ride the same normalization path. + +- **Structural errors:** P9 destroys pasted `\c` markers at save time with a user-facing error message: "You cannot put a \c marker in the middle of a chapter". + +- **Unknown markers:** Pasted unknown USFM markers are kept and flagged red in the UI, never stripped. + +--- + +## Known Accepted Asymmetries vs Paratext 9 + +These differences are documented and intentional; they are not implemented as workarounds: + +1. **P9→P10 paste from P9's Formatted view is unverified:** P9's Formatted-view copy encodes markers in `CF_HTML` CSS classes (`usfmopen`/`usfmclosed`) rather than as literal text. `paratext9HtmlToUsfm` (`paratext9Clipboard.utils.ts`) DOES implement that rule — including the `nested` `+` prefix — and its `usfm_` + `usfmopen` class pair is one of the decoder's two signatures, so such a fragment is decoded rather than pasted as prose. But no copy taken from a running P9 Formatted view has been measured against it: the rule is implemented from `Standard-Reverse.xslt` and pinned on synthetic fragments only, and the `@colspan` re-encoding P9 applies to a Formatted/Preview table cell is not implemented at all. P9's Standard-view copy, by contrast, pastes into P10 through both carriers — its plain text for everything P9 renders as literal marker text, its html for the notes that ride only there (S3's "Paste from Paratext 9"). + +2. **P10 formatted-view copy loses note callers in plain text:** P9 Standard-view copy preserved the caller glyph in plain text (though full notes survived only in `CF_HTML`). P10's formatted-view copy produces prose-only output, so note callers do not appear in plain text. This is acceptable — prose copy stays clean and users can switch to Standard view for marker-aware copy. + +--- + +## Known Lossy Constructs — Copy→Paste Round Trip + +Found by a corpus-style sweep (`clipboardCorpusRoundTrip.test.tsx`): for every fixture in the +shared USJ round-trip corpus (`corpus-data.ts`) and for the repo's one real-world fixture +(`2sa.usj.ts`), select the chapter's content, copy, paste into a fresh editor holding the same +chapter header, and compare the resulting USJ to the source. Three of the remaining failures are +INHERENT — the plain-text `text/plain` carrier (S3) has no bytes capable of representing the +construct or the attribute, so no paste-side fix is possible without a different carrier. One is +ACCEPTED normalization that matches Paratext 9's own behavior, not a bug at all. Items 1–2 and 4 +stay in the sweep as `it.skip`, not deleted, so a future engine change un-skips one automatically +instead of the gap going unnoticed — which is how the sidebar, table, milestone and `\periph` +entries that used to sit here left the list. Item 3 is not a skip: it is a single attribute inside a +fixture that otherwise sweeps clean, so the sweep asserts it as a NAMED carve-out — both its +presence in the source and its absence after the paste — rather than skipping 143 items' worth of +coverage over it. + +1. **Cross-reference `` target wrapper (inherent):** USJ's `ref` element is a wrapper USFM + itself never carried (`unknownUsfm.utils.ts`'s own doc comment: "USJ invented this container, + USFM never carried it... only its child text renders"). Source content + `["See ", {type:"ref", loc:"GEN 1:1", content:["Genesis 1:1"]}, " for details."]` copies as + plain `"See Genesis 1:1 for details."` with no marker bytes anywhere marking the wrapper's + extent; paste re-tokenizes it as ordinary prose (`["See Genesis 1:1 for details."]`, the `ref` + wrapper gone). A raw USFM export of this same fixture has the identical gap — not specific to + clipboard mechanics. + +2. **`closed="false"` char span followed by more paragraph content (inherent):** a `closed="false"` + span has, by definition, no closing marker byte anywhere in its own USFM. When such a span is + not the last thing in its paragraph (`Tell the Lord plainly.`), the + copied text (`\nd Lord plainly.`) carries no byte marking where the span's content ends and the + trailing prose resumes, so paste has nothing to stop at "Lord" on — it swallows the rest of the + paragraph into the span (`{marker:"nd", content:["Lord plainly."]}`, the top-level `" plainly."` + string gone). The sibling `"unclosed note (closed=false)"` fixture, whose unclosed span IS the + last thing in its paragraph, has no such trailing content to lose and round-trips clean — + confirming the ambiguity is specifically about trailing content after an implicit close, not + `closed="false"` itself. + +3. **A verse's derived `sid` (inherent):** USJ/USX carry `sid="2SA 1:1"` on a verse; USFM carries + `\v 1` and nothing else — the identifier is DERIVED from book + chapter + verse when + ParatextData produces USX, so there are no bytes anywhere for the plain-text carrier to copy and + nothing for paste to rebuild one from. Measured rather than assumed: a plain load→save of the + 2SA fixture keeps all 26 of its verse sids, so the loss belongs to the carrier and not to the + adaptors. The CHAPTER's `sid` survives the same round trip only because the sweep seeds the + target editor with the chapter header. Asserted by name in the sweep's + `"real-world multi-construct fixture (2sa)"` describe, on both sides (the source really carries + them; the paste carries none), so the carve-out can neither be vacuous nor outlive the gap. + +4. **Paragraph-leading space swallowed on paste (ACCEPTED normalization — matches Paratext 9, not a + bug):** isolated with a minimal non-corpus repro: pasting the literal text `"\p X"` (marker, its + own required separator, and a SECOND, real content-leading space) into a fresh empty `"\p"` host + produces `"\p X"` — one space, not two. The mechanism is `consumeSeparator()` + (`usfmFragmentToUsj.ts`), whose own comment states exactly this: "Consume the separator + whitespace after an opening marker (PT9 skips it) — all leading whitespace, not just a single + space." That mirrors Paratext 9's own `NormalizeUsfm` re-tokenization pass (see "Paratext 9 + Reference Behavior" above: "whitespace collapse, newlines inserted before paragraph/verse + markers"), which likewise collapses a whitespace run after a marker during its own paste reformat + pipeline — P10 doing the same is parity with P9, not a divergence from it. Corpus symptom: + copying ` Leading space precedes this text.` (source content + `" Leading space precedes this text."`) round-trips through paste to + `"Leading space precedes this text."` — the leading space is gone, the same as it would be in + P9. Kept in the sweep's skip list (the byte-level comparison genuinely differs from the source) + but is NOT a fix candidate. + +--- + +## Deferred / Out of Scope + +The following items are recorded as deferred and not addressed in this plan: + +1. **Pre-existing upstream CommandMenuPlugin `/`-swallow:** The paste gate in hidden-marker views inspects only `text/plain` (`event.clipboardData?.getData("text/plain")`) and blocks the whole paste — no partial insertion — whenever that string contains `\` or `/`, so it also eats URLs, dates, and common phrases like "and/or" outright; the only feedback is a `logger?.info(...)` call, which never reaches the user (no toast, no visible indication the paste was dropped). The check is `text/plain`-only, so a clipboard payload carrying **just** `text/html` (no `text/plain` key at all) skips the gate entirely — confirmed by exercising the paste path directly (2026-08-07): an html-only, backslash-bearing payload reaches Lexical's own HTML-import fallback instead of being blocked, unlike the byte-identical `text/plain` payload. This behavior originates in the `CommandMenuPlugin` on `origin/main` and predates the standard-view branches. As of 2026-08-07, this is not fixed here per TJ's guidance; it belongs to a separate upstream cleanup task. + +2. **Typing `\c` mid-chapter poisons the save loop:** If a user manually types `\c 2` in the middle of a chapter, it corrupts the editor state. The PDP error "Multiple chapter markers present" surfaces only in the renderer log, and disk/other editors silently stop updating. This is a data-fidelity issue tracked separately (WI-10's data-fidelity audit) and is not addressed by paste normalization. + +3. **Structural marker application on paste in hidden-marker views:** When a user pastes USFM markers into a hidden-marker (formatted/paragraph-structure) view, there is no logic to apply those markers as structural elements. This is by design — hidden-marker views are not marker-aware and should reject or convert marker input. Implementation is out of scope. + +4. **P9-style CF_HTML class parsing — now implemented, not yet verified live:** P9's Formatted-view copy encodes markers in `CF_HTML` CSS classes (`usfmopen`/`usfmclosed`) rather than as literal text, and P10 used to ignore them. `paratext9HtmlToUsfm` (`paratext9Clipboard.utils.ts`) implements the rule, so such a fragment decodes to USFM instead of arriving as prose. What remains open is verification and one sub-rule, both recorded as accepted asymmetry #1: no copy from a running P9 Formatted view has been measured against the decoder, and the `@colspan` re-encoding P9 applies to a Formatted/Preview table cell is not implemented. + +5. **Custom `text/usfm` MIME type / "Copy as USFM" command:** A dedicated `text/usfm` MIME type or separate "Copy as USFM" command was considered and rejected. Once `text/plain` IS USFM (S1), a separate command adds no value; the existing single-copy-format approach is sufficient. + +6. **Popover expanded-note `isStandardView` gating issue:** An issue with note-expansion behavior in popovers is tracked separately and is not addressed in the current plan. + +7. **Formatted-view copy includes CSS-hidden collapsed-note content as plain prose:** hidden-marker views (S5) hide a collapsed note's content — `\ft` text and the rest — from view with CSS; the content stays in the DOM/editor state rather than being excluded from it. A prose copy in these views is not scoped to only the visibly-rendered text, so the hidden collapsed-note body is included in the copied prose alongside the surrounding paragraph's own text. This is a pre-existing consequence of how hidden-marker views render (CSS-hide, not exclude) rather than something introduced by this plan, and is out of scope here. + +8. **A foreign clipboard that kept an NBSP only in `text/html` loses it (accepted trade-off in the paste carrier choice):** outside the Paratext 9 clause (S3), `getPastePayload` resolves a paste to `text/plain` whenever the clipboard carries any, falling back to the decoded `text/html` only when it does not. A stronger rule is possible and genuinely better for FOREIGN sources — choose the carrier the NBSP survived in (`plainText.includes(NBSP) ? plainText : htmlText`) — because some sources collapse ` ` to a plain space in their `text/plain` while keeping the real NBSP in their `text/html`; with the presence rule, `{"text/plain": "3 000", "text/html": "

3 000

"}` inserts `3 000` where the stronger rule would insert `3~000`. What the presence rule stands on is that preferring html buys NOTHING on P10's own copy: both of its readable flavors are the same USFM bytes (S1), neither carries an NBSP at all (display ones invert to plain spaces; a genuine data NBSP displays and copies as `~`), so "the plain text has no NBSP" is true of every P10 copy while the html has no NBSP to recover from. Foreign html-only sources are unaffected either way — they already reach the html fallback, since the rule is about which carrier wins when BOTH are present. So the remaining choice is between a rule that helps a foreign clipboard's data-NBSP and one that is simple and cannot mis-trigger; the simple one stands and the foreign-source loss is accepted. Closing it properly means a per-carrier reconciliation (take the plain text's words and the html's NBSP positions) rather than a choice between the two — more machinery than the gap justifies today. + +9. **A hidden-marker view's copy drops a CONTENT-LESS opaque construct entirely:** `UnknownNode.excludeFromCopy` keeps a construct in the `application/x-lexical-editor` flavor only while it has children of its own, which is exactly when `text/plain` has bytes for it — the two carriers agreeing is the point of the rule. In an editable-marker view a construct's own marker and attribute display bytes ARE children, so only a genuinely emptied husk is childless. In a HIDDEN-marker view `createUnknown` (`usj-editor.adaptor.ts`) builds no display children at all, so a construct with no content — a caption-less `figure`, an empty `ref`, every optbreak — has zero children and is dropped from both carriers; an internal (lexical-flavor) copy→paste inside such a view therefore loses the node and every attribute on it. This is consistent with S5 (a hidden-marker view copies the prose it shows, and it shows nothing for a construct with no content) rather than a regression in the copy rule, and closing it would mean deciding that a marker-free view copies marker-only constructs anyway — a view-semantics question, not a clipboard one. Recorded rather than guessed at. + +10. **CLOSED — a selection ending exactly at a TEXT-LED construct's own start carried it in the lexical flavor but not in `text/plain`:** closed by the flavor-omission rule in S2 — a boundary ON a construct is a selection reaching INTO one, and a copy that reaches into a construct writes the two text carriers only, so there is no longer a second carrier to disagree. The measurement below stands as the reason the narrower `isSelected` predicate was NOT the answer, and as the reason the rule had to live at the copy rather than inside the JSON generator. `UnknownNode.isSelected` counts a construct selected when one of its own children is in `selection.getNodes()`. For every construct whose display bytes lead (figure, sidebar, periph, optbreak) that boundary is an element point that reaches no child, and the carriers agree. A `ref` has no display bytes of its own, so its first child is a real `TextNode` and Lexical normalizes the same boundary into a TEXT point at that child's offset 0 — the child is in `getNodes()` while contributing zero characters, so `text/plain` correctly emits none of it while the `application/x-lexical-editor` flavor carries the whole `ref` (its `loc` attribute and child text included). The narrower "covers a child's content" predicate closes the disagreement and was rejected on measurement: excluding the wrapper makes `$appendNodesToJSON` HOIST its children in its place, and `createUnknown` stamps `mode:"token"` on every text child so `$sliceSelectedTextNodeContent` never trims the zero-width one — the copy then carries `Genesis 1:1` with the `ref` node and its `loc` silently gone, trading a visible superset for an invisible structural loss. Closing this properly needs a way for a node to exclude itself AND its children from the JSON generator; `exportNodeToJSON` requires every ElementNode's `exportJSON()` to return a `children` array, so there is no such lever without a custom node class for construct content — which is why the rule ended up at the copy instead. What each boundary puts on the clipboard is pinned (not merely characterized) in `unknownClipboardFidelity.test.tsx`'s `"a construct's own boundary"` describe, on content bytes, so neither the lossy hoisting variant nor the over-carrying one can come back unnoticed. + +## Test Mapping (S1–S7) + +| Semantic | Pinning Test | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| **S1** Copy (Standard view) — `text/plain` is valid USFM | `packages/platform/src/editor/markerEdit/clipboardCopyFidelity.test.tsx` — `"note caller fidelity"`, `"phantom-space live-repro pins (2026-08-07)"`, `"multi-paragraph selections"`, `"AttributeRunNode traversal"`, `"attributes a construct carries in bytes the display had to reconstruct"` (a collapsed note's `\cat` run; a spanning table cell's span suffix) describes; `packages/platform/src/editor/markerEdit/whitespaceDisplay.plugin.utils.test.tsx` — `"clipboard normalization"` and `"copy across an UnknownNode (figure)"` describes (display-NBSP→space, `~` data-NBSP preserved, full-USFM figure byte display, in BOTH readable flavors) | +| **S2** Paste (Standard view, internal) — Lexical node tree reconstruction | `packages/platform/src/editor/markerEdit/whitespaceDisplay.plugin.utils.test.tsx` — `"declines internal pastes (a same-namespace \`application/x-lexical-editor\` payload is present)"`and`"keeps current behavior (declines) when an \`application/x-lexical-editor\` payload is present, even if \`text/html\` carries NBSP"`(external-paste handler steps aside so the Lexical fast path owns the node-tree reconstruction);`packages/platform/src/editor/markerEdit/markerPasteFidelity.test.tsx`equivalence pins exercise the resulting tokenized tree shape end-to-end;`packages/platform/src/editor/markerEdit/partialConstructClipboard.test.tsx` — the flavor is written only for a selection that can carry a construct whole | +| **S3** Paste (Standard view, external) — plain text as fidelity carrier | `packages/platform/src/editor/markerEdit/whitespaceDisplay.plugin.utils.test.tsx` — `"paste normalization ($handlePasteForStandardView)"` describe, including its `"positional NBSP normalization"`, `"multi-line paste interplay (splitExpected arming)"`, `"structure protection"`, and `"tilde-corruption regression (2026-08-07 live repro)"` sub-describes; `packages/platform/src/editor/markerEdit/markerPasteFidelity.test.tsx` — `"multi-line marker-bearing paste semantics (live-verified 2026-08-07)"` and `"\\c/\\id strip on paste"` describes; `packages/platform/src/editor/markerEdit/protectedFigurePaste.test.tsx` — paste in a structure-protected Standard view (the figure built from plain text, from the editor's own html, and from html alone; a legacy DOM-export html inserting prose without a figure; the verse-glyph selection refusal; the `\c`/`\id` strip in both modes; a P9 `usfm:` comment decoded into a note) and `whitespaceDisplay.plugin.utils.test.tsx`'s `"structure protection"` describe (the byte rules a protected paste keeps, no paragraph split, the declined block-spanning selection, and the `Editor`→plugin wiring measured as a difference between the two modes) | +| **S3 (attribute-context paste)** Paste inside a char/milestone/verse attribute display run — paste ≡ typing | `packages/platform/src/editor/markerEdit/attributeContextPasteFidelity.test.tsx` — `"typed characterization (baseline)"`, `"paste ≡ typed (TJ's repro shape)"` (undo pinned), `"root cause: a native paste event carrying a same-namespace application/x-lexical-editor flavor must not corrupt the run"` (a confirmed regression class, not the literal pre-branch mechanism — see the S3 subsection above), `"leading-space payload"`, `"replace-selection paste inside the attribute value"` (undo pinned), `"multi-line payload collapses to a single space, per newline"` (undo pinned), `"marker-bearing payload"`, `"CUT of a selection inside the attribute value"`, `"mixed selection"` (now claims the attribute path — both the flavor and multi-line corruption shapes pinned), `"mixed selection: the pasted bytes are BODY content, not attribute-value bytes"` (the `\\c` strip and the positional NBSP rule applying to a touching selection from either end, the wholly-inside boundary that keeps value-byte semantics, and a plain payload still settling identically to the same character typed over the same range), `"milestone attribute run paste"` (including the same touching-selection rule on a second run kind), and `"verse \\va run paste"` describes | +| **S4** Paste-as-plain-text (Ctrl+Shift+V) — equivalence to external paste | `packages/platform/src/editor/markerEdit/markerPasteFidelity.test.tsx` — `"paste-as-plain-text equivalence (S4): no literal mode, plain always wins"` describe | +| **S5** Hidden-marker views — prose copy, paste gate preservation | `packages/platform/src/editor/CommandMenuPlugin.gate.test.tsx` — `"CommandMenuPlugin editable-mode gate"` and `"MarkerEditPlugin's Standard-view clipboard handlers do not leak into Formatted view"` describes | +| **S6** Cut = copy + `removeText()` — WI-2 regression pin | `packages/platform/src/editor/markerEdit/markerEditDeletion.utils.test.tsx` — `"selection-delete at a settled char-span boundary (WI-2 filed)"` (does not reproduce at the Lexical level; pinned as regression armor); `packages/platform/src/editor/markerEdit/clipboardCopyFidelity.test.tsx` — `"cut = copy + removeText"` describe pins the byte-fidelity half of the semantic (cut's clipboard bytes match copy's, and the source is removed) | +| **S7** Undo — paste and rebuilds as single step | `packages/platform/src/editor/markerEdit/markerPasteFidelity.test.tsx` — the `undoAndSettle`-based pins (multi-line marker-bearing paste, `\c`/`\id` strip, no-op `\c` paste) each asserting one `UNDO_COMMAND` dispatch restores the exact pre-paste USJ; `packages/platform/src/editor/markerEdit/optbreakClipboardFidelity.test.tsx` — the two one-step undo pins covering the pend-then-departure-settle shape on both carriers | +| **S3 (peripheral divisions)** `\periph` re-tokenizes back into a `periph` division — its title, its attribute list, and the blocks it contains | `libs/shared/src/converters/usfm/usfmFragmentToUsj.test.ts` — `"usfmFragmentToUsjContent — peripheral divisions (\\periph)"` describe: the marker line split into `alt` plus attributes, the line-per-marker spelling reading identically, following blocks taken into the division, one division ending at the next, a chapter-free contentless division, a nested sidebar, the unparseable-attribute degrade to a literal paragraph, and note context opening no division at all | +| **Corpus sweep** Copy→paste round trip across the shared USJ fixture corpus AND the repo's real-world fixture | `packages/platform/src/editor/markerEdit/clipboardCorpusRoundTrip.test.tsx` — `"corpus copy/paste round trip (Standard view)"` describe; three known-lossy fixtures kept as `it.skip` (see "Known Lossy Constructs" above); the corpus's own `"optional line break (optbreak)"`, `"sidebar (esb)"`, `"figure (USFM 3 attributes)"` and `"table with header and cells"` fixtures are swept clean, not skipped. Its `"real-world multi-construct fixture (2sa)"` describe sweeps `libs/test-data/src/data/2sa.usj.ts` (143 top-level items, every supported construct in the shapes a real project produced) to full USJ equality, with the one inherent verse-`sid` loss asserted by name on both sides. The whole file runs in about a second | +| **Optbreak (`//`) clipboard fidelity** Copy characterization (`text/plain`/`text/html`), paste round trip across all three real-world payload shapes (plain-only, plain+html, plain+html+lexical), undo (one step for both the lexical-flavor path and a plain-payload paste that pends until departure — S7), cut | `packages/platform/src/editor/markerEdit/optbreakClipboardFidelity.test.tsx`; the S2 fix itself lives in `libs/shared/src/nodes/features/UnknownNode.ts` (`excludeFromCopy`) | +| **Per-kind opaque constructs** Copy→paste round trip for figure/sidebar/periph/ref (and the `ImmutableTable*` kinds that used to be `UnknownNode`s) across all three payload shapes, plus the shared `isSelected` boundary rule on more than one kind | `packages/platform/src/editor/markerEdit/unknownClipboardFidelity.test.tsx` — one describe per kind. Figure, sidebar, periph and table round-trip on all three shapes; `ref` round-trips on the lexical flavor and has its plain-carrier LOSS asserted by name (USFM has no bytes at all for a `ref` wrapper). The periph describe also pins WHY the copy stays on one line, by pasting the line-per-marker spelling and asserting the division comes back empty with its paragraph beside it. Mounts the full `Editor`, not `MarkerEditPlugin` alone. Its `"a construct's own start boundary"` describe pins the `isSelected` rule for a decorator-led kind (`figure`, excluded from both carriers) and a text-led one (`ref`, kept whole in the lexical flavor — Deferred item 10), asserting content BYTES rather than tags | +| **S3 (deferred paste rebuild)** A pasted line whose Tier-2 rebuild PENDS settles to the same document the identical TYPED bytes do | `packages/platform/src/editor/markerEdit/markerPasteFidelity.test.tsx` — `"a DEFERRED settle treats pasted and typed bytes identically"` describe: a paste whose bytes eject content out of a milestone (so the rebuild waits for departure) is asserted with the pre-departure literal in place, so the pin cannot pass through the immediate-rebuild path, and its sibling drives the same bytes by typing and expects the identical USJ. The no-invented-marker rule for a multi-line paste is pinned in the same file's `"a multi-line paste never invents a `\p` for a line that already carries its own marker"` | +| **S1 (`text/html` flavor)** The rich-text carrier ships the same USFM bytes as `text/plain` | `packages/platform/src/editor/markerEdit/whitespaceDisplay.plugin.utils.test.tsx` — `"text/html flavor — the same USFM bytes as text/plain"` describe (no NBSP in either flavor, `~` untouched in both, a space run and a fragment-edge space surviving the html carrier, and both flavors decoding to the same document text), including its `"usfmToClipboardHtml mechanics"` unit pins (block per line, empty line, entity escaping); `packages/platform/src/editor/markerEdit/clipboardCopyFidelity.test.tsx` — `"text/html carries the same USFM bytes as text/plain"` describe (a collapsed note's `+`/`-`/literal caller, a cross-reference's `-` caller present in the html, no `data-caller`/node-class residue, a two-paragraph selection's line break, `<`/`>`/`&` escaping, and an html-ONLY paste of the editor's own copy round-tripping the note caller); `packages/platform/src/editor/markerEdit/optbreakClipboardFidelity.test.tsx` — the `"copy characterization"` describe's `text/html` pin (the `//` is present and decodes to the plain carrier's bytes) | +| **S1/S6 (empty copy)** A copy or cut with nothing selected writes nothing; a read-only construct is copied by a selection containing it | `packages/platform/src/editor/markerEdit/whitespaceDisplay.plugin.utils.test.tsx` — `"copying an empty selection leaves the clipboard alone"` describe (collapsed-caret copy, cut, and the caret-beside-a-figure shape from the live report), plus `"copies the figure's own bytes when the selection covers the figure alone"`; `packages/platform/src/editor/CommandMenuPlugin.gate.test.tsx` — `"copying an empty selection leaves the clipboard alone in a hidden-marker view"` (the rule holds where no Standard-view handler is registered); `libs/shared-react/src/plugins/usj/ClipboardPlugin.test.tsx` — the Ctrl+C/Ctrl+X key handling itself, with the paste keys pinned unaffected | +| **S3 (opaque construct materialized by a paste)** A pasted `\fig …\fig*` keeps its caption INSIDE the figure — the construct's own content is not ejected as paragraph prose | `packages/platform/src/editor/markerEdit/figurePasteFidelity.test.tsx` — mid-paragraph and end-of-paragraph USJ pins plus a display-byte-order pin whose oracle is a real LOAD of the same document; the fix itself lives in `libs/shared-react/src/plugins/usj/TextSpacingPlugin.tsx` (`$textNodeInUnknownTransform` ejects only into a wrapper that predates the update), with its boundary pinned in that plugin's own suite. These pins mount the full `Editor`: the ejection lives in a sibling plugin, so a `MarkerEditPlugin`-only harness reports the same paste clean | +| **S3 (Paratext 9 html)** A P9 clipboard's html is decoded to USFM and wins over its own `text/plain` | `packages/platform/src/editor/markerEdit/paratext9Clipboard.utils.test.ts` — three describes: the note shapes P9's plain text cannot carry (`+`/`-`/literal caller, `%XXXX` unescaping of a `--` pair and non-ASCII bytes, a folded `\cat` run), Standard-Reverse's structural rules (`exclude` with a nested `include`, `div`/`br` newlines, `usfmopen`/`usfmclosed` with the `nested` `+` prefix, `span.attribute`, NBSP→space, newline collapsing, and a note inside a real verse+paragraph fragment), and the negatives that keep the decoder off every other clipboard (Word, Google Docs, this editor's own copy built by the real producer, a note-free P9 Standard-view fragment, and a CF_HTML wrapper comment); `packages/platform/src/editor/markerEdit/markerPasteFidelity.test.tsx` — `"Paratext 9 clipboard html (P9→P10 paste)"` describe: a real `PASTE_COMMAND` carrying `text/plain: "a"` plus P9's html materializes the footnote with its caller and body and leaves the glyph out of the document, and the same for a `-` caller cross-reference | +| **E2E** Real-browser clipboard round trip (Standard view) through the real OS clipboard | `e2e-tests/tests/isolated/scripture-editor/clipboard-usfm-round-trip.spec.ts` (paranext-core repo) | + +--- diff --git a/libs/shared-react/src/plugins/usj/ClipboardPlugin.test.tsx b/libs/shared-react/src/plugins/usj/ClipboardPlugin.test.tsx new file mode 100644 index 00000000..cc061543 --- /dev/null +++ b/libs/shared-react/src/plugins/usj/ClipboardPlugin.test.tsx @@ -0,0 +1,188 @@ +import { ClipboardPlugin } from "./ClipboardPlugin"; +import { copySelection } from "./clipboard.utils"; +import { baseTestEnvironment } from "./react-test.utils"; +import { act } from "@testing-library/react"; +import { + $createNodeSelection, + $createTextNode, + $getRoot, + $setSelection, + LexicalEditor, + TextNode, +} from "lexical"; +import { $createImmutableTypedTextNode, $createParaNode } from "shared"; + +/** + * `document.execCommand("copy")` is how a clipboard write reaches the browser when there is no real + * clipboard event to fill in: `@lexical/clipboard` points the DOM selection at a hidden placeholder + * element it appends to the editor and runs it to provoke one. jsdom implements no `execCommand` at + * all, so these tests install a spy in its place — **called means a write reached the browser, not + * called means the clipboard is untouched**. That is the observable throughout; the placeholder's + * own content belongs to `@lexical/clipboard` and is never asserted on here. + */ +let execCommand: ReturnType; + +beforeEach(() => { + execCommand = vi.fn(() => true); + Object.defineProperty(document, "execCommand", { + configurable: true, + writable: true, + value: execCommand, + }); + // `pasteSelection`/`pasteSelectionAsPlainText` read the async clipboard API, which jsdom also + // does not implement. A read that never settles is enough for these tests: they assert only that + // the paste keys still reach it, not what a paste does with what it finds. + vi.stubGlobal("navigator", { + ...navigator, + clipboard: { read: vi.fn(() => new Promise(() => undefined)) }, + }); +}); + +afterEach(async () => { + // `@lexical/clipboard` keeps a MODULE-level timer handle while it waits for the clipboard event + // its `execCommand` call should provoke, and refuses to start another copy until that handle + // clears. A test that reaches the real copy path therefore silences the `execCommand` assertion + // in the NEXT test unless the window is drained here — which would make a failure show up only in + // whichever test happens to run first. The window is `EVENT_LATENCY`, 50ms. + await new Promise((resolve) => setTimeout(resolve, 60)); + Reflect.deleteProperty(document, "execCommand"); + vi.unstubAllGlobals(); +}); + +/** An editor holding one paragraph of text, with the clipboard key handling under test mounted. */ +async function clipboardEnvironment(): Promise<{ editor: LexicalEditor; text: TextNode }> { + let text: TextNode | undefined; + const { editor } = await baseTestEnvironment( + () => { + text = $createTextNode("In the beginning"); + $getRoot().append($createParaNode("p").append(text)); + }, + , + ); + if (!text) throw new Error("expected the initial text node to exist"); + return { editor, text }; +} + +/** Presses a clipboard shortcut on the editor's root element, where the plugin listens. */ +async function pressShortcut( + editor: LexicalEditor, + key: string, + shiftKey = false, +): Promise { + const rootElement = editor.getRootElement(); + if (!rootElement) throw new Error("editor has no root element to press a key on"); + const event = new KeyboardEvent("keydown", { + key, + ctrlKey: true, + shiftKey, + bubbles: true, + cancelable: true, + }); + await act(async () => { + rootElement.dispatchEvent(event); + }); + return event; +} + +describe("ClipboardPlugin — copy/cut with nothing selected", () => { + it("leaves the clipboard untouched on Ctrl+C at a collapsed caret", async () => { + const { editor, text } = await clipboardEnvironment(); + await act(async () => editor.update(() => text.select(3, 3))); + + await pressShortcut(editor, "c"); + + expect(execCommand).not.toHaveBeenCalled(); + }); + + it("leaves the clipboard untouched on Ctrl+X at a collapsed caret, and removes nothing", async () => { + const { editor, text } = await clipboardEnvironment(); + await act(async () => editor.update(() => text.select(3, 3))); + + await pressShortcut(editor, "x"); + + expect(execCommand).not.toHaveBeenCalled(); + editor.getEditorState().read(() => expect(text.getTextContent()).toBe("In the beginning")); + }); +}); + +describe("ClipboardPlugin — copy/cut with a selection", () => { + it("copies on Ctrl+C", async () => { + const { editor, text } = await clipboardEnvironment(); + await act(async () => editor.update(() => text.select(0, text.getTextContentSize()))); + + await pressShortcut(editor, "c"); + + expect(execCommand).toHaveBeenCalledWith("copy"); + }); + + it("cuts on Ctrl+X", async () => { + const { editor, text } = await clipboardEnvironment(); + await act(async () => editor.update(() => text.select(0, text.getTextContentSize()))); + + await pressShortcut(editor, "x"); + + expect(execCommand).toHaveBeenCalledWith("copy"); + }); + + it("copies a node selection — the guard is about having nothing to copy, not about ranges", async () => { + // A node selection covers real content and is not collapsed, so it copies like any other. The + // guard tests "is there anything here", NOT "is this a range": narrowing it to range selections + // would silently swallow this copy. + const { editor } = await clipboardEnvironment(); + await act(async () => + editor.update(() => { + const decorator = $createImmutableTypedTextNode("marker", "\\p"); + $getRoot().getFirstChild()?.insertBefore?.($createParaNode("p").append(decorator)); + const nodeSelection = $createNodeSelection(); + nodeSelection.add(decorator.getKey()); + $setSelection(nodeSelection); + }), + ); + + await pressShortcut(editor, "c"); + + expect(execCommand).toHaveBeenCalledWith("copy"); + }); +}); + +describe("ClipboardPlugin — the guard reads the live selection, not the committed one", () => { + // Lexical commits on a microtask, so the last COMMITTED selection lags a selection made earlier + // in the same synchronous tick. A guard that read the committed state would see "nothing + // selected" here and silently copy nothing — and this is the ordinary shape of the public + // `EditorRef.copy()` path: select something programmatically, then copy it. + it("copies a selection made earlier in the same synchronous tick", async () => { + const { editor, text } = await clipboardEnvironment(); + + await act(async () => { + editor.update(() => text.select(0, text.getTextContentSize())); + copySelection(editor); + }); + + expect(execCommand).toHaveBeenCalledWith("copy"); + }); + + it("copies a selection made inside the same editor.update() as the copy call", async () => { + const { editor, text } = await clipboardEnvironment(); + + await act(async () => + editor.update(() => { + text.select(0, text.getTextContentSize()); + copySelection(editor); + }), + ); + + expect(execCommand).toHaveBeenCalledWith("copy"); + }); +}); + +describe("ClipboardPlugin — paste keys are unaffected", () => { + it("claims Ctrl+V and Ctrl+Shift+V regardless of the selection", async () => { + const { editor, text } = await clipboardEnvironment(); + await act(async () => editor.update(() => text.select(3, 3))); + + // A collapsed caret is exactly where a paste belongs, so the empty-selection rule copy and cut + // now follow must not reach these. + expect((await pressShortcut(editor, "v")).defaultPrevented).toBe(true); + expect((await pressShortcut(editor, "v", true)).defaultPrevented).toBe(true); + }); +}); diff --git a/libs/shared-react/src/plugins/usj/ClipboardPlugin.tsx b/libs/shared-react/src/plugins/usj/ClipboardPlugin.tsx index 9158db69..cbcca185 100644 --- a/libs/shared-react/src/plugins/usj/ClipboardPlugin.tsx +++ b/libs/shared-react/src/plugins/usj/ClipboardPlugin.tsx @@ -1,7 +1,12 @@ -import { pasteSelection, pasteSelectionAsPlainText } from "./clipboard.utils"; +import { + copySelection, + cutSelection, + pasteSelection, + pasteSelectionAsPlainText, + registerEmptyCopyGuard, +} from "./clipboard.utils"; import { useLexicalComposerContext } from "@lexical/react/LexicalComposerContext"; -import { IS_APPLE } from "@lexical/utils"; -import { COPY_COMMAND, CUT_COMMAND } from "lexical"; +import { IS_APPLE, mergeRegister } from "@lexical/utils"; import { useEffect } from "react"; export function ClipboardPlugin(): null { @@ -14,10 +19,10 @@ export function ClipboardPlugin(): null { if (!shiftKey && key.toLowerCase() === "c") { event.preventDefault(); - editor.dispatchCommand(COPY_COMMAND, null); + copySelection(editor); } else if (!shiftKey && key.toLowerCase() === "x") { event.preventDefault(); - editor.dispatchCommand(CUT_COMMAND, null); + cutSelection(editor); } else if (key.toLowerCase() === "v") { event.preventDefault(); if (shiftKey) pasteSelectionAsPlainText(editor); @@ -25,15 +30,20 @@ export function ClipboardPlugin(): null { } }; - return editor.registerRootListener( - (rootElement: HTMLElement | null, prevRootElement: HTMLElement | null) => { - if (prevRootElement !== null) { - prevRootElement.removeEventListener("keydown", onKeyDown); - } - if (rootElement !== null) { - rootElement.addEventListener("keydown", onKeyDown); - } - }, + return mergeRegister( + // Every copy/cut this plugin's shortcuts synthesize — and every one the context menu or an + // editor ref synthesizes against the same editor — passes through this guard. + registerEmptyCopyGuard(editor), + editor.registerRootListener( + (rootElement: HTMLElement | null, prevRootElement: HTMLElement | null) => { + if (prevRootElement !== null) { + prevRootElement.removeEventListener("keydown", onKeyDown); + } + if (rootElement !== null) { + rootElement.addEventListener("keydown", onKeyDown); + } + }, + ), ); }, [editor]); diff --git a/libs/shared-react/src/plugins/usj/ContextMenuPlugin.test.tsx b/libs/shared-react/src/plugins/usj/ContextMenuPlugin.test.tsx new file mode 100644 index 00000000..0c0093bb --- /dev/null +++ b/libs/shared-react/src/plugins/usj/ContextMenuPlugin.test.tsx @@ -0,0 +1,132 @@ +import { ClipboardPlugin } from "./ClipboardPlugin"; +import { ContextMenuPlugin } from "./ContextMenuPlugin"; +import { baseTestEnvironment } from "./react-test.utils"; +import { act } from "@testing-library/react"; +import { $createTextNode, $getRoot, LexicalEditor, TextNode } from "lexical"; +import { $createParaNode } from "shared"; + +/** + * The context menu's Cut/Copy dispatch the same commands the keyboard shortcuts do, so they are + * covered by the same empty-copy guard (`registerEmptyCopyGuard`). This pins that the leg really + * does go through it, rather than dispatching around it — both in the shape the shipped editors + * mount (alongside `ClipboardPlugin`, which registers the guard too) and with `ContextMenuPlugin` + * mounted alone, which a host consuming the plugin on its own is free to do. + * + * Note where `onSelect` runs: inside `editor.update()` (see the plugin's Enter handler). That is + * why the guard has to live on the COMMAND and read the live selection — a check in front of the + * dispatch reading the last committed state would be both stale and, via `editor.read()`, unsafe + * to call from there. + * + * `document.execCommand("copy")` is the observable, as everywhere else in this suite: called means + * a clipboard write reached the browser, not called means the clipboard is untouched. + */ +let execCommand: ReturnType; + +beforeEach(() => { + execCommand = vi.fn(() => true); + Object.defineProperty(document, "execCommand", { + configurable: true, + writable: true, + value: execCommand, + }); +}); + +afterEach(async () => { + // Drain `@lexical/clipboard`'s module-level `EVENT_LATENCY` (50ms) handle, which otherwise makes + // a test that reached the real copy path silence the next test's assertion. + await new Promise((resolve) => setTimeout(resolve, 60)); + Reflect.deleteProperty(document, "execCommand"); +}); + +async function contextMenuEnvironment( + withClipboardPlugin = true, +): Promise<{ editor: LexicalEditor; text: TextNode }> { + let text: TextNode | undefined; + const { editor } = await baseTestEnvironment( + () => { + text = $createTextNode("In the beginning"); + $getRoot().append($createParaNode("p").append(text)); + }, + <> + {withClipboardPlugin && } + + , + ); + if (!text) throw new Error("expected the initial text node to exist"); + return { editor, text }; +} + +/** + * Opens the context menu over the editor's content and activates the option at `index` the way a + * keyboard user would (the plugin's own arrow/Enter handling), rather than by reaching for the + * rendered menu's markup. Built-in order: Cut, Copy, Paste, Paste as Plain Text. + */ +async function chooseContextMenuOption(editor: LexicalEditor, index: number): Promise { + const rootElement = editor.getRootElement(); + const target = rootElement?.firstElementChild; + if (!target) throw new Error("expected the editor to have rendered content to right-click"); + await act(async () => { + target.dispatchEvent(new MouseEvent("contextmenu", { bubbles: true, cancelable: true })); + }); + for (let step = 0; step <= index; step++) { + await act(async () => { + document.dispatchEvent(new KeyboardEvent("keydown", { key: "ArrowDown", bubbles: true })); + }); + } + await act(async () => { + document.dispatchEvent(new KeyboardEvent("keydown", { key: "Enter", bubbles: true })); + }); +} + +const COPY_OPTION = 1; +const CUT_OPTION = 0; + +describe("ContextMenuPlugin — Cut/Copy go through the empty-copy guard", () => { + it("Copy with a collapsed caret leaves the clipboard untouched", async () => { + const { editor, text } = await contextMenuEnvironment(); + await act(async () => editor.update(() => text.select(3, 3))); + + await chooseContextMenuOption(editor, COPY_OPTION); + + expect(execCommand).not.toHaveBeenCalled(); + }); + + it("Cut with a collapsed caret leaves the clipboard untouched and removes nothing", async () => { + const { editor, text } = await contextMenuEnvironment(); + await act(async () => editor.update(() => text.select(3, 3))); + + await chooseContextMenuOption(editor, CUT_OPTION); + + expect(execCommand).not.toHaveBeenCalled(); + editor.getEditorState().read(() => expect(text.getTextContent()).toBe("In the beginning")); + }); + + it("Copy with a selection copies — the menu leg is really wired to the command", async () => { + const { editor, text } = await contextMenuEnvironment(); + await act(async () => editor.update(() => text.select(0, text.getTextContentSize()))); + + await chooseContextMenuOption(editor, COPY_OPTION); + + expect(execCommand).toHaveBeenCalledWith("copy"); + }); +}); + +describe("ContextMenuPlugin mounted without ClipboardPlugin", () => { + it("Copy with a collapsed caret still leaves the clipboard untouched — the plugin carries its own guard", async () => { + const { editor, text } = await contextMenuEnvironment(false); + await act(async () => editor.update(() => text.select(3, 3))); + + await chooseContextMenuOption(editor, COPY_OPTION); + + expect(execCommand).not.toHaveBeenCalled(); + }); + + it("Copy with a selection still copies — the standalone guard does not over-claim", async () => { + const { editor, text } = await contextMenuEnvironment(false); + await act(async () => editor.update(() => text.select(0, text.getTextContentSize()))); + + await chooseContextMenuOption(editor, COPY_OPTION); + + expect(execCommand).toHaveBeenCalledWith("copy"); + }); +}); diff --git a/libs/shared-react/src/plugins/usj/ContextMenuPlugin.tsx b/libs/shared-react/src/plugins/usj/ContextMenuPlugin.tsx index 7d8efd25..75adb0ad 100644 --- a/libs/shared-react/src/plugins/usj/ContextMenuPlugin.tsx +++ b/libs/shared-react/src/plugins/usj/ContextMenuPlugin.tsx @@ -2,9 +2,14 @@ * Adapted from https://github.com/facebook/lexical/blob/main/packages/lexical-playground/src/plugins/ContextMenuPlugin/index.tsx */ -import { pasteSelection, pasteSelectionAsPlainText } from "./clipboard.utils"; +import { + copySelection, + cutSelection, + pasteSelection, + pasteSelectionAsPlainText, + registerEmptyCopyGuard, +} from "./clipboard.utils"; import { useLexicalComposerContext } from "@lexical/react/LexicalComposerContext"; -import { COPY_COMMAND, CUT_COMMAND } from "lexical"; import { ReactElement, useCallback, @@ -134,15 +139,20 @@ export function ContextMenuPlugin({ const options = useMemo(() => { const builtIn = [ + // Cut/Copy with nothing selected leave the clipboard alone rather than writing a placeholder + // over it — `registerEmptyCopyGuard` (mounted below) claims the command, so no selection + // check is needed here. They are not disabled in that case, because this option list is + // built once per editor rather than per menu opening, so its `isDisabled` flags cannot track + // the live selection. new ContextMenuOption(`Cut`, { onSelect: () => { - editor.dispatchCommand(CUT_COMMAND, null); + cutSelection(editor); }, isDisabled: isReadonly, }), new ContextMenuOption(`Copy`, { onSelect: () => { - editor.dispatchCommand(COPY_COMMAND, null); + copySelection(editor); }, }), new ContextMenuOption(`Paste`, { @@ -170,6 +180,14 @@ export function ContextMenuPlugin({ setSelectedIndex(undefined); }, []); + // This plugin is exported on its own, so a host can mount it without `ClipboardPlugin` — and its + // Cut/Copy options would then hit the unguarded synthesized-copy path, overwriting whatever the + // clipboard already held with `@lexical/clipboard`'s hidden placeholder character. Registering + // the guard here keeps the plugin self-sufficient; a second registration alongside + // `ClipboardPlugin`'s is harmless, since both listeners sit at the same priority and the first + // one to claim an empty selection stops propagation before the other runs. + useEffect(() => registerEmptyCopyGuard(editor), [editor]); + // Register context menu event on editor root useEffect(() => { const handleContextMenu = (event: MouseEvent) => { diff --git a/libs/shared-react/src/plugins/usj/DecoratorBoundarySelectionPlugin.test.tsx b/libs/shared-react/src/plugins/usj/DecoratorBoundarySelectionPlugin.test.tsx new file mode 100644 index 00000000..b2a766bc --- /dev/null +++ b/libs/shared-react/src/plugins/usj/DecoratorBoundarySelectionPlugin.test.tsx @@ -0,0 +1,150 @@ +/** + * The two edges of {@link DecoratorBoundarySelectionPlugin}'s scope, which the figure pins in + * `decoratorBoundarySelection.test.tsx` (platform) cannot reach: the DOM point form Lexical resolves + * by a route of its own, and the one decorator whose landings belong to another rule. + */ + +import { $createImmutableNoteCallerNode } from "../../nodes/usj"; +import { DecoratorBoundarySelectionPlugin } from "./DecoratorBoundarySelectionPlugin"; +import { baseTestEnvironment } from "./react-test.utils"; +import { TrailingNoteCaretGuardPlugin } from "./TrailingNoteCaretGuardPlugin"; +import { act } from "@testing-library/react"; +import { + $createTextNode, + $getRoot, + $getSelection, + $isRangeSelection, + $isTextNode, + LexicalEditor, +} from "lexical"; +import { + $createCharNode, + $createImmutableTypedTextNode, + $createMarkerNode, + $createMarkerTrailingSeparator, + $createNoteNode, + $createParaNode, + $createUnknownNode, + $isParaNode, + CURSOR_PLACEHOLDER_CHAR, + ParaNode, +} from "shared"; +import { ReactNode } from "react"; + +/** Flush the macrotask jsdom fires its native `selectionchange` on, plus the microtasks the caret + * guards defer their work by. */ +async function flushSelectionChange(): Promise { + await act(async () => { + await new Promise((resolve) => { + setTimeout(resolve, 0); + }); + }); +} + +/** Point the DOM selection at one position and let the native `selectionchange` carry it into + * Lexical, the way a click does. */ +async function putDomCaret(node: Node, offset: number): Promise { + await flushSelectionChange(); + const domSelection = document.getSelection(); + if (!domSelection) throw new Error("no DOM selection"); + await act(async () => { + domSelection.setBaseAndExtent(node, offset, node, offset); + }); + await flushSelectionChange(); +} + +/** The document's one paragraph. */ +function readParagraph(editor: LexicalEditor): ParaNode { + return editor.getEditorState().read(() => { + const para = $getRoot().getChildren().find($isParaNode); + if (!para) throw new Error("no paragraph in the tree"); + return para; + }); +} + +describe("a DOM point that IS a decorator's own element", () => { + /** `\p before \fig cap\fig*` — a read-only construct whose opening and closing USFM bytes are + * `ImmutableTypedTextNode` decorators, the shape Standard view renders. */ + async function figureEnvironment(): Promise<{ editor: LexicalEditor }> { + return baseTestEnvironment( + () => { + const figure = $createUnknownNode("figure", "fig").append( + $createImmutableTypedTextNode("marker", "\\fig "), + $createTextNode("cap"), + $createImmutableTypedTextNode("marker", "\\fig*"), + ); + $getRoot().append($createParaNode("p").append($createTextNode("before "), figure)); + }, + , + ); + } + + it("is left to Lexical, whose own answer for that form the caret guards depend on", async () => { + // The plugin's job is the form Lexical REFUSES to resolve — a point strictly INSIDE a + // decorator's element. A point that IS the element resolves by a different route: Lexical makes + // an element point beside the decorator out of it, and then deliberately nulls a selection whose + // BOTH ends came from decorator DOM, which is the arrival `TrailingNoteCaretGuardPlugin` reads. + // Answering it here would take that arrival away from the rules that can already see it. + const { editor } = await figureEnvironment(); + const openerDom = editor.getRootElement()?.querySelector('[data-text-type="marker"]'); + if (!openerDom) throw new Error("the opener glyph did not render"); + + await putDomCaret(openerDom, 0); + + editor.getEditorState().read(() => { + expect($getSelection()).toBeNull(); + }); + }); +}); + +describe("a landing on a collapsed note's caller", () => { + /** `\p before |note|` — a collapsed note nothing renders past, the shape + * `TrailingNoteCaretGuardPlugin` hosts a caret for. */ + async function noteEnvironment(children: ReactNode): Promise<{ editor: LexicalEditor }> { + return baseTestEnvironment(() => { + const note = $createNoteNode("f", "+").append( + $createMarkerNode("f", "opening"), + $createImmutableNoteCallerNode("+", "note preview"), + $createMarkerTrailingSeparator(), + $createCharNode("ft").append( + $createMarkerNode("ft", "opening"), + $createTextNode("note body"), + ), + $createMarkerNode("f", "closing"), + ); + $getRoot().append($createParaNode("p").append($createTextNode("before "), note)); + }, children); + } + + /** The caller's rendered `