Repository navigation
PT-4911 + PT-4921: Hidden-view clipboard filters and lossless paste - #43
tjcouch-sil wants to merge 157 commits into
Conversation
b79ccc6 to
5acf0ed
Compare
6e17d10 to
dab2ba4
Compare
e87f905 to
36bf341
Compare
d6fb939 to
1f01570
Compare
…constructs A copy or cut in a hidden-marker view (formatted, paragraph-structure, block-verse, Preview) now writes the text the view shows: - HiddenViewCopyPlugin (COPY/CUT at NORMAL, mounted right after CopyLimitPlugin for markerMode "hidden") writes text/plain without a paragraph's gutter marker glyph or the text of an UnknownNode construct (a figure's caption), text/html from Lexical's export, and the internal lexical flavor. Collapsed note text stays in the plain text, which a plain-text paste needs. - ImmutableTypedTextNode exports no element for a gutter glyph, so the HTML flavor leaves it out. Lexical consults excludeFromCopy only on element nodes, so a decorator has to leave itself out through exportDOM. - CopyLimitPlugin's over-limit plain writer and its fit leave the same nodes out in hidden views. - The clipboard corpus's gutter-glyph known-failure rule is removed: its 43 rows now pass (the 44th still fails, under the html-flavor rule). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add UnstorableTextPlugin, which removes every `\` and every `//` (pairing slashes from the left, and counting a pasted `/` that touches a document `/`) from each paste and drop route: node copies and html through SELECTION_INSERT_CLIPBOARD_NODES_COMMAND, plain text through PASTE_COMMAND and insertFromDrop. It reports each removal once through onRemoved and logs it at info. Under protected structure, a plain-text paste or drop joins its lines with a space instead of splitting the paragraph. StructureKeyboardPlugin takes a $filterPastedNodes prop for the html it inserts itself under protected structure. HiddenContentDeleteGuardPlugin treats a plain paste as inserting nothing only when the filter leaves it empty, and asks about html and node copies. EditorOptions gains onUnstorableTextRemoved (wired into the Editor separately). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The protected-structure sanitizer lists a paragraph's children without detaching them, so a text node the unstorable-text filter emptied was detached but stayed listed and was inserted with its original text. The filter now drops an emptied node from the list whenever it is listed, and both the plugin and the sanitizer use one `$insertsNothing` predicate, so a paste that leaves only an emptied character span no longer deletes the selection it was pasted over. Plain-text insertion goes through `$insertDataTransferForRichText` again (shared-react now depends on @lexical/clipboard, already in the lockfile). Under protected structure, each run of line breaks becomes one space and a trailing line break is dropped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hidden-marker views mount UnstorableTextPlugin in place of CommandMenuPlugin, so a paste or drop holding `\` or `//` goes in without them instead of being refused, and EditorOptions.onUnstorableTextRemoved is told once per paste or drop. Under protected structure the html sanitizer filters the same way through StructureKeyboardPlugin's $filterPastedNodes, given only in hidden-marker views so Standard view's paste keeps its markers. The Markers view keeps CommandMenuPlugin. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hidden-marker views now take a paste holding `\` or `//` without those, so the corpus oracle filters the pasted text (filterFragment: strings only, never a stray-marker node) instead of expecting the paste refused, and checks that the host is told once whenever the filter removed something. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t stray \ and // The paste filter strips `\` and `//` from a hidden-marker view's pasted text, so a copy that leaked a gutter glyph or marker into text/plain would pass the paste checks with a matching notice. Each hidden-view copy row now checks that the text/plain holds exactly as many `\` and `/` as the copied content does, a stray marker counted as its own text. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hidden-marker views refuse a typed `\`, and a typed `/` that would touch a `/` beside it (judged outside any selection the keystroke replaces); a lone `/` goes in. Each refused keystroke is reported once through `onUnstorableTextRemoved` with source "typing". The refusal happens at KEY_DOWN (shortcut chords such as Ctrl+/ pass, AltGr does not), with backstops for typed text that arrives without a keydown: a CRITICAL CONTROLLED_TEXT_INSERTION_COMMAND handler (string payloads and insertReplacementText events) and a capture-phase root beforeinput listener for native insertText. Multi-character typed text (autocorrect, replacements) loses only `\` and `//` and goes in, with one notice. IME composition is not filtered. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…guarded Simple mode's paragraph-structure view now runs every clipboard corpus row under each structure protection (off, guarded, protected — Simple mode's default); the other views run off. - Oracle: protected html pastes (all flavors included, since the sanitizer takes text/html first) unwrap paragraphs joined by one space and drop verse/chapter markers (protectedFragment); protected plain multi-line pastes join lines; a protected cut across a paragraph or over a verse is refused (document unchanged, nothing on the clipboard). The locked html route carries a stray marker as text, so its `\` is filtered with one notice (TJ ruling: accepted behavior). - Cut-paste-back expectations are built from the filtered fragment, so the document and the notice count agree; the filter models the characters beside the insertion point. - New rows: a range holding 2SA-1's figures, "cut, paste html and plain", and plain pastes of and/or, 1/2 and https://example.org. Every hidden-view copy checks text/plain for no gutter glyph, no figure caption, and the collapsed notes' text. - PT-4921 rule covers the html route under protection and a caption-less figure left out of the editor's own clipboard format (TJ ruling); PT-4388 rule narrowed to the rows that fail. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ally The plain-text paste rows now state what a hidden-marker view inserts and what its notice names as removed (https://example.org -> https:example.org with one "//" notice; and/or and 1/2 unchanged, no notice; the USFM-looking texts lose their "\"), instead of deriving both from the filter under test. A precondition asserts no "/" sits beside the paste point, which the literals assume. Also widens the PT-4921 reason to name an optbreak alongside the caption-less figure, and fixes two comment nits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rewrite the clipboard semantics doc's S5 for the hidden-marker views: the copy leaves out gutter markers and hidden constructs but keeps a collapsed note's text; paste, drop and typing remove `\` and `//` (with adjacency to a document `/`) and tell the host once through EditorOptions.onUnstorableTextRemoved; locked plain multi-line paste joins lines; the Markers view still refuses `\` and `/`. Close the CommandMenuPlugin paste-gate deferral, restate the collapsed-note copy as intended, point the remaining html-route and caption-less-figure gaps at PT-4921, and map the new tests. Record the user-visible changes and the new public API in the platform package's changelog. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dden-marker views Outside structure protection, html pasted or dropped into a hidden-marker view brought a marker glyph span back as text (ImmutableTypedTextNode's html import never claims one), and the unstorable-text filter then left it as text the user never wrote (`\q1 ` -> `q1 `) with a notice. A drag-move across a paragraph break in the paragraph-structure view hits this, since Lexical's DRAGSTART writes no data and the browser serializes the next paragraph's gutter glyph; so do html pastes of a Markers-view copy or an older copy. UnstorableTextPlugin now removes every `[data-text-type="marker"]` element from the html route of a paste or drop before Lexical parses it, through removeMarkerGlyphElements, which StructureKeyboardPlugin's protected sanitizer now shares. The node-copy and plain-text routes are untouched; html left with nothing to insert changes nothing. Standard view's handling of the same html is pinned unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- CopyLimitPlugin: the over-limit writer runs ahead of the hidden-marker views' HiddenViewCopyPlugin (NORMAL, registered after it), not only of HIGH handlers. - MarkerEditPlugin: the CUT arm's list of removers names HiddenViewCopyPlugin. - clipboard-semantics S5 and CHANGELOG: under protection a tab becomes a space only in a multi-line plain-text paste. - UnstorableTextPlugin joinLines: a trailing line break is dropped because under protection it would only split the paragraph. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e rule HiddenContentDeleteGuardPlugin skips asking about a paste that will insert nothing. It judged that by UnstorableTextPlugin's rule (only `\` and `//` are removed), but the guard also runs in the Markers view, where CommandMenuPlugin still refuses any paste whose plain text holds a `\` or `/`. In a Markers view made editable, a paste of "and/or" over hidden content would be asked about, and a "delete" answer would remove the hidden content while nothing was pasted. (The guard does nothing in a read-only editor, so the shipped read-only Markers view is not affected.) A new refusesCommandCharacters prop (set by Editor for markerMode "visible") restores the command-character answer there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…or precedence Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A comment or annotation mark saves as nothing, so text either side of its edge ends up adjacent in the file. The filter read only direct siblings, so a / typed or pasted right beside a / across a mark's edge saved as //. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nput alone insertFromYank (macOS Ctrl+Y) reaches CONTROLLED_TEXT_INSERTION_COMMAND as a replacement does, but the filter only read the replacement's event, so a yanked \ or // went in unfiltered. The beforeinput check called editor.read on every typed character, which commits pending updates; it now tests for a \ or / first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
$deletionJoinsSlashes reads what the file stores either side of what a delete removes (a range, one character from a caret, or a paragraph join) and says whether a / would end up against a / it was not already touching. It reads through annotation marks, gutter glyphs and cursor placeholders, through a character span the delete empties, and through two pieces of one span a join makes one again; a span edge that stays, a verse, a note or a hidden construct keeps the two apart, and a // already in the text is not counted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5J1wq8kko2sACPaBFZ
…idden-marker views Deleting b from a/b/c saved a//c, which USFM reads as an optional line break, so the next load showed ac. UnstorableTextPlugin now refuses any delete whose result would put a / against a / it was not already touching, and tells the host once through onUnstorableTextRemoved with the new source "deletion". It judges every delete route: Backspace and Delete with their word and line chords at the keystroke (CRITICAL, ahead of guarded structure's arming press, the hidden-content question and the paragraph-marker merge), the same deletes without a keystroke (DELETE_*_COMMAND), a selected paragraph marker, a cut and a drag-move's removal of its source (HIGH, after the guards that narrow them). Under protected structure a join is left to that structure's own refusal. Word and line extents are measured on a copy of the selection, as HiddenContentDeleteGuardPlugin measures them; its deleteChordOf is exported for the keystroke mapping. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5J1wq8kko2sACPaBFZ
An optional line break is a childless unknown node in the views that hide markers, so the filter and the delete check read it as a wall. The file stores it as a bare `//`, though: deleting the space from `a/ `, break, `/b` saved `a////b`, which reads back as two breaks, and the view lost both `/`. Typing or pasting a `/` against a break had the same effect when a `/` stood on its other side. $characterBeside now reads a break beside a point as `/`, and $readPieces gives a break's atom the text `//`, so $joinsSlashes sees a `/` brought against it. A delete still takes the break whole. Also in unstorableText.utils.ts: say where $insertsNothing and $textAroundSelection may be called, return undefined rather than null from $neighbourThroughMarks, build the grapheme segmenter once, and name the deletion's start, end and caret a Boundary rather than a Marker, which in this file otherwise means a USFM marker. New $deletionJoinsSlashes rows: a verse or a note between two `/` (staying and going), a range starting or ending inside a note, and a delete inside an expanded note's text. Docs: the CHANGELOG's protected-paste entry loses a stray line break, and Deferred item 9 states its open half instead of narrating it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5J1wq8kko2sACPaBFZ
…s filtered Pins two rules UnstorableTextPlugin documents but no test held: a keydown an input method is composing is neither refused nor reported, and a drop carrying only `text/uri-list` (as Lexical reads it when there is no `text/plain`) loses its `//` with one notice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5J1wq8kko2sACPaBFZ
…where markers hide It was mounted only when markerMode was "hidden", so an editor that switched into a hidden-marker view registered its CRITICAL KEY_DOWN and delete handlers behind ParaMarkerSelectionPlugin's: deleting a selected paragraph marker joined `a/` and `/c` into `a//c`. The plugin now takes isEnabled, held in a ref and asked by every handler and its beforeinput listener, and Editor mounts it always with isEnabled set for hidden markers, so its handlers keep their place across a view switch. mountInView gains rerenderWithView for the test. (Carried down from the PT-4921 branch, where it was 717333f, so that this branch holds it on its own.) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5J1wq8kko2sACPaBFZ
# Conflicts: # packages/platform/dist/index.js # packages/platform/dist/index.js.map # packages/platform/src/editor/Editor.tsx
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
katherinejensen00
left a comment
There was a problem hiding this comment.
Just two things blocking the merge, one new and one left over from item 1. I included some optional smaller things at the end, but you can merge without them so it is totally up to you
- High: the node-copy check is by type only, so a crafted copy can still break the editor or save corrupt USFM
nodeCopy.utils.ts:378 (typeOutsideHiddenView) · Confirmed (probe)
The allowlist stops root and marker entries. But it doesn't check where an allowed type sits, or what its fields hold, so any web page's copied html can still:
- Leave the editor unusable. An immutable-note-caller outside a note: the paste goes through, then every later update throws "noteNode.getIsCollapsed is not a function" until a reload. Probe: [para[immutable-note-caller]] in the html attribute, under off and protected.
- Save corrupt USFM. A char with marker "a b" is saved to USJ as is, and createDOM's classList.add throws. A verse number of "2\v 3" is saved and would write \v 2\v 3. An unknown entry with tag "chapter" or "book" saves a chapter or book node inside a paragraph, where nothing shows.
- Show what isn't saved, or the reverse. A text entry's style is applied to the live span, but the style isn't saved: display:none hides text that is still saved, and background-image:url(https://…) makes a request the web view's CSP allows. immutable-typed-text holding "\v 5" shows but isn't saved.
- Overflow the stack. Depth isn't capped: about 1000 levels pastes, and then getUsj throws RangeError, so the document can't be saved until the paste is undone. Deeper nesting throws out of the paste.
Suggested fix, all checked before any node is made, in typeOutsideHiddenView:
- allowed parents per type (a note caller only directly in a note);
- marker validated with the node class's isValidMarker;
- verse numbers run through the \ and // filter;
- text style and format rejected;
- unknown tags limited to the content-less constructs a hidden view holds (figure, optbreak, …);
- a depth cap of about 32.
A crafted-payload test for each would pin it. Another option is to round-trip the made nodes through USJ and the normal loader, which validates all of this already.
- Medium: under protected structure, a paste can now delete hidden content without asking
hiddenViewInsertion.utils.ts:160 ($predictInsertion) · Confirmed (probe + trace)
This comes from the item 4 fix. Under protected, StructureKeyboardPlugin's sanitizer puts a " " between top-level pasted paragraphs (structureKeyboard.utils.ts:839), but $predictInsertion judges the nodes before sanitizing. So when every pasted paragraph holds only \ or //, the guard decides the paste inserts nothing and doesn't ask. The sanitizer then inserts the lone space over the selection.
Probe: formatted view, protected, a selection across a \fig, html "
</p>
</p>". onHiddenContentDelete is never called, and the figure is replaced by a space. Off and guarded leave the document unchanged.
Simple mode uses protected whenever structure is locked, so this ships, though the trigger is unusual and Undo restores the figure. Suggested fix: predict on the sanitized nodes when isProtected, plus a two-paragraph test.
Smaller things, optional:
- Item 25: the selection-start change has no backward-selection test, so going back to anchor fails nothing.
- docs/clipboard-semantics.md:185 still names $unstorableInDrop, which is now $predictInsertion. Line 133 puts htmlWithNodeCopy in clipboard.utils.ts; it's in nodeCopy.utils.ts now.
- nodeCopy.utils.ts:99: "platformEditor" isn't Lexical's default namespace; Editor.tsx sets it. The {@link MARKER_TEXT_HTML_ATTRIBUTE} at :101 won't resolve.
- htmlClipboardRoute.test.tsx:278 narrates the old bug ("a root entry crashed the editor"); say what the check prevents instead.
- Withdrawn-marker leftovers from item 3: \cp B pastes "B" as its own line, and \ca 3\ca* pastes " 3" as text.
On the CodeQL alerts: #23–#27 are false positives like the earlier ones. Each is an inert DOMParser parse whose result only becomes Lexical nodes or an html string for another inert parse, and nothing reaches the live DOM. The editor's importDOM converters copy no href, src, on* or style. Dismissing them is right. #27 would go quiet if htmlWithNodeCopy built the carrier inside a DOMParser document. Please don't quiet it with carrier.innerHTML = html, which would be a real sink.
|
Found while testing PT-4820 (Model Text panel with gutter markers): in a hidden-marker view, double-click a paragraph's gutter glyph, say \q1, and press Ctrl+C. The clipboard gets "\q1 ". The double-click selects only the glyph's "". HiddenMarkersViewCopyPlugin filters glyph text out, gets "", and returns undefined (if (plain === "") return undefined;), so Lexical's own copy runs and writes the glyph text. Could the writer claim a copy whose filtered text is empty instead of falling through? paranext-core has a test.fails tripwire for this in model-text-gutter.real-editor.test.tsx that will flip once it's handled. |
…the whole note A range with both ends inside one collapsed note (its content hidden, only the caller showing) copied part of the note's shell, e.g. `\f - \cat things\cat*`, while the cut removed the whole note through the damaged-glyph-pair rule, so a cut and paste back lost everything after the category. Found in hand QA of PT-4921 (Power mode, Standard view). `$takeCollapsedNotesIntoRange` already widened a range with one end inside a collapsed note on selection change, and deliberately leaves a range wholly inside one note alone for the marker menu. Copy and cut now widen that range too, at CRITICAL priority ahead of every copy writer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
… dropped With structure locked, `StructureKeyboardPlugin` took every drop and inserted it at the editor's selection, which at a drop is still what was dragged. A drag-move within the editor so put the text back where it was (nothing moved), and a copy-drop or a drop from another program replaced the selection instead of landing at the drop point. Found in hand QA. It now inserts at the drop point (`$selectionAtDropPoint`), refuses a drop whose drop point would replace structure before anything is removed, and leaves a move within the editor to the move the unlocked views make (`UnstorableTextPlugin`, catalog CB-23/CB-24). A drag that holds structure is still refused at its dragstart. `isCopyDrop` moves to `dropPoint.utils.ts` for both plugins. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
The paste check took any entry of a type the view's document holds, so a crafted node copy in any web page's html could still put a note caller outside its note (every later update threw), save a char marker "a b" or a verse number "2\v 3" as corrupt USFM, show styled or typed text the save drops, or nest deep enough to overflow getUsj. nodeCopyProblem now checks each entry's placement, marker, number, attribute and text fields and caps nesting at 32 before any node is made; such a copy goes in by its html or plain text. immutable-typed-text and typed-mark leave the allowlist (a copy never holds them). The html carrier is built in an inert document. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
…s sanitized $predictInsertion judged a paste's nodes before StructureKeyboardPlugin's sanitizer puts a space between top-level paragraphs, so a paste of paragraphs holding only `\` over a figure was judged to insert nothing: HiddenContentDeleteGuardPlugin did not ask, and the space replaced the figure. Under protected structure the prediction now sanitizes first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
In a hidden-marker view, double-clicking a paragraph's gutter glyph and copying fell through to Lexical's copy, which wrote the glyph's text (`\q1 `). The view's copy writer now claims a copy holding nothing but gutter glyphs and writes nothing; a cut is still left to the paragraph-marker rules. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
…p out of a hidden-view paste The chapter token went, but `\ca 4\ca*` pasted `4` as text and `\cp B` pasted `B` as a line of its own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
…r this round A backward selection is judged where it starts (its focus). The clipboard semantics doc names $predictInsertion and nodeCopy.utils.ts where it named a removed helper and the wrong module, and describes the new node-copy checks, the protected-structure prediction and the \ca/\cp strip; CHANGELOG entries for each. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
Smaller items:
|
|
Fixed in |
…o its paragraph spaces inserts nothing The sanitizer puts a space between pasted paragraphs. Once a filter took out all of their text (paragraphs holding only `\`), that space was still inserted over the selection, so the paste replaced a figure with a space, where an unprotected paste of the same html changes nothing. The spaces it adds are now known ($insertsOnlyParagraphSeparators), and a paste left with nothing else inserts nothing; $predictInsertion judges it the same way, so nothing is asked either. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
…o a hidden view This reverts 12fa0a4. \ca and \cp are markers of their own in USFM and stand alone in every data format, so a paste that withdraws the chapter line keeps them, as it keeps any other marker's content. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
|
Two follow-ups to my reply above:
|
JSON.stringify of the crafted 1000-level payload overflowed the stack of CI's Windows test worker before the paste ran. The test now builds that payload's JSON by repetition; the code under test parses and refuses it under the same reduced stack. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
katherinejensen00
left a comment
There was a problem hiding this comment.
Thanks for making those changes, TJ! Three things block the merge. Optional items are at the end.
- High: crafted node copies still get past the check
nodeCopy.utils.ts:652 · Confirmed (probe + code)
- Attributes overwrite real fields. editor-usj.adaptor.ts spreads ...unknownAttributes after type, marker and number (:251, :267), so an attribute named marker, number or type replaces that field on save. Probes saved a char marked "a b", verse "2\v 3", and a chapter or book inside a paragraph, the same results the round-3 probes got.
- Two fields still break the editor. If a milestone's attributeOrder is a string, every later update throws "attributeOrder.forEach is not a function". Deeply nested $ node state makes every later update overflow the stack, because the depth cap counts only children.
- The * in UNMATCHED_MARKER (:415) is optional, so an unmatched "v" followed by " 5 X" saves as \v 5 X, a new verse.
Fix: reject structural keys as attribute names (or spread unknownAttributes first), require attributeOrder to be an array of names, allow only known $ keys, and make the * required. No test reaches fieldsProblem: with it forced to accept everything, all 1670 clipboard tests pass. One crafted case per check would pin them.
- Medium: Ctrl+C inside a collapsed note sets up its deletion
TrailingNoteCaretGuardPlugin.tsx:342 · Confirmed (probe + code)
The new COPY handler widens the live selection to the whole note and leaves it there. Select inside a collapsed footnote, press Ctrl+C, type X: the footnote is replaced without asking, in Standard view too. Restore the selection after the copy (keep the widening for cut), and add a test that types after copying.
- Medium: the glyph-copy fix only works when a verse follows the glyph
HiddenMarkersViewCopyPlugin.tsx:73 · Confirmed (probe)
On a line that starts with text (\q2 continuation, \m), getNodes() returns [glyph, text], so $selectsOnlyGutterMarkers fails and "\q2 " is copied again. The test's \q1 line starts with a verse. Letting text nodes through the every() fixes it, since the plain text is already "". The PT-4820 tripwire may not flip until then.
On \ca and \cp: I'm fine with your call. One sentence near clipboard-semantics.md:156 saying their content is kept would stop someone "fixing" it later.
Optional:
- HiddenMarkersViewCopyPlugin.tsx:64: the two new declarations split the component from its doc comment.
- StructureKeyboardPlugin.tsx:135: doc still names $insertsNothing.
- No CHANGELOG entry for the protected drop (85c447a) or the collapsed-note copy/cut (ee572e4). clipboard-semantics.md:182/:185 and the PR body still describe the old protected drop. :141 misses the glyph-only copy.
- A cut of a glyph still writes "\q1 ". Should it write nothing, like the copy?
- A text entry with text: null passes the check and then the paste throws. Require a string.
- collapsedNoteClipboard.test.tsx:3 narrates the old bug.
- The 1000-level depth case adds nothing over the 40-level one, and if the cap regresses it times out and breaks the tests after it.
…n where it was The copy widened the live selection to the whole note and left it there, so typing next replaced the note without asking. The selection now goes back once every copy writer has run; a cut still takes the whole note. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
… any line On a line led by text, a glyph-only selection also holds the text beside it, so the check failed and `\q2 ` was copied. Text in a selection whose plain text is empty contributes nothing, so it now passes. A glyph-only cut wrote `\q1 ` too while removing nothing; it is now claimed the same way, and still removes nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
…d cut, \ca/\cp kept clipboard-semantics.md and the CHANGELOG describe the drop that lands at its drop point under protected structure and the whole-note copy and cut, which had no entries, and the glyph-only copy and cut; the doc says why a chapter's \ca and \cp keep their content. StructureKeyboardPlugin's doc names $insertsOnlyParagraphSeparators. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
…or save A crafted node copy could still get past the check: an attribute named marker, number, type (or another field the save writes) replaced that field on save; a milestone's attributeOrder as a string, or deeply nested node state, made every later update throw; an unmatched marker without its `*` saved as a new marker; a text of null threw on paste. Every field each type's importJSON reads or the save writes is now checked: attribute names may not be saved fields, attributeOrder is a list of names, node state only the editor's own keys with values their own parse keeps, unmatched markers end in `*`, text is a string, and element format fields are their defaults. A unit test pins each check (each fails if its check accepts everything); the 1000-level case goes, since the 40-level one and a unit test at the cap cover depth. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
On Optional:
|
katherinejensen00
left a comment
There was a problem hiding this comment.
So sorry to put you through one more round of review. I am hoping this is the last one.
Blocking:
- Ctrl+C inside a collapsed note in Standard view drops the closing \f* (caused by this round). TrailingNoteCaretGuardPlugin.tsx:345
- Cause: the selection restore is queued with editor.update. That only runs after the copy writers when the copy is dispatched inside an update, which is how the tests do it. A real Ctrl+C dispatches outside one, so the restore lands before Standard view's copy writer runs.
- Probe: a real copy event gives text ending …\ft B with no \f*. Pasting that back saves an unclosed footnote. With the restore removed, the whole note copies correctly. Simple mode wasn't affected.
- Fix: run the restore after the whole copy has finished, and have the test fire a real copy event.
- A style attribute still replaces the marker (left over from blocker 1). nodeCopy.utils.ts:435
- Cause: when the document is saved, the USJ-to-USX converter sets style from the marker and then writes every other attribute over it. paranext-core saves through that converter.
- Probe: a pasted w char with style: "v" passes the check and saves as .
- Fix: add style to the list of field names an attribute can't use, plus one test.
Worth fixing, not blocking:
- Deep objects in node state: a deep object under a true/false node state (isAuthoredNbsp, isCaretHost) still gets through. Later edits are fine now, but saving the editor state to JSON throws a stack-overflow error, so copying that text would fail.
- Possible crash instead of fallback: the check builds its warning messages before the try starts. From reading the code (I didn't run this one), a deeply nested paste would throw instead of falling back to the HTML.
TL;DR
This PR combines PT-4911 and PT-4921, the two clipboard tickets for the views that hide USFM markers (Simple mode's paragraph-structure view, the formatted view, block-verse). It replaces #41 (closed as superseded). Companion: paranext/paranext-core#2976, which likewise absorbs #2969. In these views:
\q1) and no hidden figure or unknown-construct text; a stored no-break space is written as~in the plain text.\or//. Each\and each//is removed from the inserted text and the host is told throughonUnstorableTextRemoved. A single/("and/or", 1/2) is kept. A delete, cut, transpose or drag-move that would join two/into//is refused instead.\cat), milestones, figures and other unknown constructs survive, because the copy carries the editor's own format in an invisibledata-*attribute of itstext/html. A Select All copy keeps one book line and one chapter line, and a copy from Standard view or the Footnotes pane loses its marker tokens instead of pasting marker names as words./.Standard and Markers views keep their own
\and//handling.Bundled on purpose (review item 13): three fixes that are not clipboard changes ride in this PR: the loader fix (
usxStringToUsjreads USX by XML 1.0 line ends, so a stored U+2028, U+2029 or U+0085 is no longer lost), entered control characters becoming spaces in every view, and the U+000B settle break (a Standard-view marker settle no longer drops a U+2028). Each is small, has its own tests and its own commit; they came out of the U+2028 work. A squash-merge revert of the clipboard work would take them with it, which is accepted."Catalog" labels (CB-6, T-18, EV-5, F-9, S5, ...) are row ids in paranext-core's
.context/standards/Scripture-Data-Fidelity-Catalog.md(CB = clipboard, T = typing, EV = entered values, F = file/load fidelity) anddocs/clipboard-semantics.md(S5).Public API
@eten-tech-foundation/platform-editor:EditorOptions.onUnstorableTextRemoved: new optional handler, called synchronously from the editor command or update that made the edit, once per paste, drop or blocked keystroke that removed something (and once per refused delete or drag-move).UnstorableTextRemoved: the payload.sourceis"paste" | "drop" | "typing" | "deletion" | "move";removedis areadonlyarray ofUnstorableCharacter.UnstorableTextSource: new exported alias for thesourceunion, named in the CHANGELOG's Hosts note.UnstorableCharacter("\\" | "//") andUnstorableTextRemovedHandler: the other two types.ClipboardPlugin.pastesAsClipboardRead/logger,ContextMenuPlugin.logger,StructureKeyboardPlugin.nodeCopyNamespace/removesMarkerTokens/logger.Host action and merge order
npm run move-platform-yalc.The host shows the notices (paranext-core#2976).
Decisions for the reviewer
PT-4911:
protected(Simple's default), the locked html path's DOMPurify drops the<unmatched>tag, so a copied stray\*pastes as*(with the notice). Accepted by TJ for now. A two-line DOMPurify allowlist would keep it; unlocked and Ctrl+V-through-the-editor-format routes already keep it.\*) gives*with the notice, the same rule as\v 9 x→v 9 x.\or//is not filtered (documented in the plugin's TSDoc)./:HiddenContentDeleteGuardPlugincopies before it asks, so such a cut used to leave the copy on the clipboard and remove nothing. Since3e0000d8the guard does not ask before a cut thatUnstorableTextPluginrefuses, so that cut is refused without a question and copies nothing (CHANGELOG and S5 updated in37b7a491).protected, a glyph-only html paste over a selection removes the selection (StructureKeyboardPlugininserts an empty list).PT-4921:
\p) goes too, matching Standard view's own paste (stripPastedChapterAndBookId, shared by both views); a verse's number still stays as text, and so does a space the user wrote. A\inside an attribute value counts as a marker (src="c:\pics\a.jpg"→src="c:.jpg"): rare, and fixing it needs real tokenizing.source: "move"), since nothing was removed.Beyond the clipboard:
adr-editor-control-characters-entered-as-spaces).Known gaps
Selection.modify(word and line deletes) andcaretRangeFromPoint(the drop point).beforeinputtarget-range handling (insertClaimedBeforeInputTextapplying the event's first target range) is unverified in a real browser.p), which saves as\p …\p*. The paste check tests a marker's shape, not its kind, since real copies hold markers outside the node classes' lists (\cpand\zunkparagraphs, a\catchar in a note).ee572e4d; TJ re-tested it), and with structure locked a drag-move did nothing, because the drop went in at the selection rather than the drop point (CB-26,85c447a4; unit tests and the drag-move e2e, not yet re-tested by hand).Testing at HEAD
After merging
main(#55):nxtypecheck and lint (9 projects): green.in-editor-paste,hidden-view-clipboardandhidden-view-drag-moveagainst this tip staged: passed (3/3).node scripts/verify-committed-dist.mjs).After the two hand-QA fixes (
ee572e4d,85c447a4):nxtypecheck and lint: green.hidden-view-drag-moveagainst this tip staged: passed.After review round 5 (
62736de4..e159a5cb):nxtypecheck and lint (9 projects): green.hidden-view-clipboard,in-editor-paste,hidden-view-drag-moveandclipboard-usfm-round-tripagainst this tip staged: passed (4/4; the cold first test needed one retry).After review round 4 (
3f126398..87b1befb):nxtypecheck and lint (9 projects): green.hidden-view-clipboard,in-editor-paste,hidden-view-drag-moveandclipboard-usfm-round-tripagainst this tip staged: passed (4/4; the cold first test needed one retry).Not done:
AI Involvement
AI-assisted (Claude Code): implemented task by task by AI subagents from written plans (including Electron transport probes on Linux and Windows), each task spec- and quality-reviewed by a separate AI reviewer, then a whole-branch AI review and fix wave per ticket, an automated
/review-paratextrun per ticket and a/code-review highround per ticket. TJ ruled on the open product and scope questions.Risk Level
Medium — changes every clipboard route, merges this branch's copy writer into PT-4126's, and changes the html of every copy in Simple mode's editable view and the formatted view, plus drag-move and line-break handling. The fixes beyond the clipboard change what an entered control character becomes in every view, and how
usxStringToUsjreads USX for every consumer (narrowly: only line-end normalization). Covered by the corpus suite in all three structure-protection modes, and falls back to today's html path when the attribute is missing or unreadable.History
Review round 5 (Katherine, 2026-11-19)
11979192. Every field each allowed type reads or saves is checked: attribute names may not be saved fields,attributeOrderis a list of names, node state only the editor's own keys, unmatched markers end in*, text is a string, element format fields are defaults. One unit test per check (nodeCopy.utils.test.ts), each mutation-verified.62736de4. The selection goes back after the copy.488cae84; a glyph-only cut now writes and removes nothing too.dbdded4a,62736de4);text: nullrefused and the 1000-level case removed (11979192).Review round 4 (Katherine, 2026-11-17)
3f126398.nodeCopyProblemchecks each entry's placement, marker, numbers, attributes, text fields and nesting depth (32 levels) before any node is made.immutable-typed-textandtyped-markleave the allowlist. Each probe is a crafted-payload test, with a positive control.d18947d6,acefb5a9. Under protected structure,$predictInsertionjudges the sanitized nodes, and a paste whose paragraphs are all filtered out inserts nothing (the sanitizer's separator spaces don't count), as off and guarded do, so nothing is asked.cd5331ea); doc names and module (cd5331ea); namespace doc and test comment (3f126398);\ca/\cpleft out with their chapter (12fa0a4c), then reverted (7bdd023b): they are markers of their own and keep their content; reconsideration asked of the reviewer. CodeQL PT-4730: Limit how much text can be copied from the editor #23–PT-4703: Ask before an edit removes content the view does not show #27 are dismissed as false positives; the node-copy carrier is built in an inert document.ce419fb6. A copy of only gutter glyphs writes nothing.Review round 3 (Katherine, 2026-11-13)
Findings 1-26, all answered in the PR thread. Where the code changed:
33e9459f.$nodesFromNodeCopychecks every entry's type, at any depth, against the types a hidden-marker view's document holds before it makes any node (RootNode.importJSONreturns the live root, so a later check is too late); an already-attached made node is rejected too. Either case falls back to the html with awarn. Six newhtmlClipboardRoute.test.tsxcases.//refusal across a chapter line):7d68e042. The judge uses the ranges the removal uses ($rangesRemovedKeepingBookAndChapterLines).\id/\c):6f126f49.withoutMarkerTokensdrops book lines and chapter tokens with their payload and a line it leaves empty.cfa219ff. The guard asks$predictInsertion(hiddenViewInsertion.utils.ts), the prediction a drop's filter uses, so it judges every route.$runEnterfakes a keydown): documented in2a3ba42e; no shipping path reaches the wrong case.UnstorableTextPlugin): moved to PT-4984, which rewrites this routing for USFM on the clipboard. The route decision and the insertion prediction already live inhiddenViewInsertion.utils.ts.cfa219ff,d923189b.$canSplitAtandjoiningTextare shared byLineBreakPluginand the plain-text paste.2b5e1e2f. Both claims go throughinsertClaimedBeforeInputText.af96d243,2b5e1e2f.af96d243,2a3ba42e(UnstorableTextSourceexported).d0f64479; writer, readers and namespaces innodeCopy.utils.ts.f860044d(a dead selection restore removed;transferWithexported and aplainTextTransferadded for the test transfers). 24:72a13793. 25:7d68e042.Behavior details by ticket
PT-4911: Copy prose, and filter
\and//instead of refusing\q1) or hidden figure/unknown-construct text. A collapsed footnote's text stays in the plain-text copy, by design (catalog CB-6). A stored no-break space is written as~in the plain text unlessallowInvisibleCharactersis on (hiddenViewCopySpelling, the one spelling path for these views); the html and node copies keep U+00A0 (catalog EV-5). The copy is written bymain'sHiddenMarkersViewCopyPlugin, which this branch's own copy writer was merged into (see "Merged with PT-4126").\, and every//scanning left to right (///leaves/), is removed from the inserted text, on every insertion route (Lexical's node route, its plain-text route, drops, and the locked-structure html path). A/next to an existing/counts as forming//. A single/("and/or", 1/2) is kept. Stray-marker (unmatched) nodes are left alone. Sincemain'sStructureKeyboardPlugininserts a protected paste throughSELECTION_INSERT_CLIPBOARD_NODES_COMMAND,UnstorableTextPluginfilters that paste even without the sanitizer's own filter; the editor still passes the sanitizer's filter so a drop is reported as a drop./works;\, and a/beside a/, are blocked. A/beside a/is judged across a comment or annotation mark's edge too.//the file stores for it. The view shows nothing there, but a/against one saves as///, and with a/on its other side as////, two breaks, which hides both/after the next load. So a/typed or pasted against a break is refused or removed, and a delete that would bring a/against one is refused, as for any other/. TJ confirmed this rule: a lone/typed right beside a hidden break is refused even with no/on the break's other side./into//. Chromium raises it as abeforeinputinsertTranspose, which Lexical applies past the filters, so/x/could become//x.UnstorableTextPluginrefuses a transpose that would put a/against a/(an optional line break counts as//), judged as the Backspace and Delete at that caret ($transposeJoinsSlashes). It keeps the caret in place and reportssource: "typing". Modelled in jsdom only; unverified on a real Mac./into//. Deletingbfroma/b/csaveda//c, which the next load reads as an optional line break (the view then showedac). A delete whose result would put a/against a/it was not already touching is refused, and reported once throughonUnstorableTextRemovedwith the newsource: "deletion". It covers Backspace and Delete with their word and line chords, keydown-less deletes, selection deletes, a selected paragraph marker, a paragraph join (guarded structure's first press), cut and a drag-move's removal. Under protected structure a join stays protection's own refusal. An authored zero-width space between two/counts as text (b7647eca), and a range running out of an expanded note is judged in its paragraph (85ed24d9), so neither is wrongly refused. Known gaps (S5): IME composition, and Android Chrome's browser-applied Backspace.3e0000d8).HiddenContentDeleteGuardPluginno longer asks before a cut thatUnstorableTextPluginthen refuses for joining two/, nor before a paste that inserts nothing;pasteInsertsNothingnow judges the paste against the characters around the selection.EditorOptions.onUnstorableTextRemoved, called once per paste, drop or blocked keystroke that removed something (and, with PT-4921, once per refused drag-move). Hosts show the notice (paranext-core#2976).UnstorableTextPluginis mounted in every view and acts only where markers are hidden (isEnabled), so an editor that switches into a hidden-marker view keeps its delete handlers ahead of the other plugins' at the same priority. Mounted only in hidden views, it registered behindParaMarkerSelectionPlugin's after a switch, and deleting a selected paragraph marker could joina/and/c. Standard and Markers rows now pin that every handler stands aside there when disabled.PT-4921: Lossless Ctrl+V in hidden-marker views
Copy and paste within the editor is now lossless in these views: character attributes (incl. the "not closed" state), footnote details (
\cat, attributes), milestones, alternate and published verse numbers, figures and other unknown constructs all survive Ctrl+V, the context menu's Paste and the Edit menu's Paste.data-*attribute oftext/html, written during the copy event, and paste reads it back. A native paste (the macOS app menu's Paste) sees the flavor as well. No host command is needed. Standard and Markers views keep their route.platformEditor/hidden-markers, without gutter glyphs and keeping content-less constructs (a caption-less figure), and paste it losslessly. A Standard-view copy pasted here takes its html/plain text, and a hidden-view copy pasted into Standard view is foreign to it, so neither view family rebuilds the other's node shapes. The locked-structure path flattens the tagged copy like html and keeps everything else.1 \ft TR and→1 TR and). Verse numbers, a note's+caller and its reference stay as text; book lines and chapter tokens go whole, and so does a line left empty.off/guarded, Select All includes the\idand chapter lines (and in chapter 1,\h/\mt1/the introduction between them); the copy used to drop its node copy for them, so Ctrl+V read the html and added a second chapter (every later save fails), or lost the chapter line, the title and every note's\cat. Now the copy writes its node copy; no paste route (node copy, html, the locked sanitizer) inserts a book or chapter line; a copy holding a chapter line splits around the document's (front matter before it, Scripture text after) only when pasted over a selection holding it or from a caret right beside it (the start of the line after it, where a cut leaves the caret, or the end of the line before it), and anywhere else goes in at the caret like any paste; a cut, or a paste over a selection, keeps each book and chapter line, drops each run of lines it covers whole without leaving an empty paragraph, and leaves the caret in the Scripture text. Select All → cut → paste back restores chapter 1 exactly, and pasting a Select All copy over Select All changes nothing. A whole chapter-1 copy pasted at the end goes in there, front matter and all; pasted into any other chapter, its front matter goes after that chapter's line with the rest (only chapter 1 has lines before its chapter line). Plain text of several lines pasted over Select All keeps its lines (b8687c24); only a paste that lands in the\idline joins them. Protected structure still refuses the cut and the paste-over. A delete (Backspace, Delete, word/line chords), typed text, Enter or a drag-move over such a selection used to remove both lines and type into the\idline (older than this PR); the newBookAndChapterLinesPlugin(hidden-marker views, LOW priority, after every guard) removes the selection the way the cut does, so typing over Select All goes in after the chapter line. Pinned byhiddenViewSelectAllClipboard.test.tsxandhiddenViewSelectAllEdit.test.tsx(each case on a chapter with front matter and one without, plus a chapter-1 copy pasted into chapter 2, a drop, a line delete and a line break, and carets near the chapter line that must not split).\idline.startsMidParagraph, carried in the node copy and so in the html route too. When it is pasted around the chapter line, that paragraph joins the line that a paste without the split would have joined. So a mid-line cut pasted back restores\ip An introduction to the book.exactly, instead of splitting it in two. Whole-line copies, older copies without the mark, and Standard view are unchanged. The mark is stripped on paste, so no node in the document keeps it.$getCharacterOffsets, which answers 0 for an element point short of its element's end, so a selection ending on the root between two paragraphs copied none of the last one's text (and a stray line break), while the cut removed it. It now places each end by position. No gesture in Standard view is known to make that selection; fixed as low effort and low risk. Pinned bystandardCopyElementPoints.test.tsx.application/x-lexical-editorflavor, and Standard view and the Footnotes pane's row editor rebuilt nodes from it: a cut collapsed footnote pasted back threw, a paste over a note's\ftlanded outside the note, and a pasted paragraph marker showed but wasn't saved (PT-4872 cases 2, 4, 5, 8).ClipboardPlugin's newpastesAsClipboardRead(set byEditorfor everymarkerModebut"hidden") claims such a paste ahead of every other paste handler and re-dispatches it without that flavor, so it takes exactly what Ctrl+V delivers; Standard keeps its tokenizing paste. Trade-off: aref's wrapper andlocare lost on a native paste, as on Ctrl+V. In Paratext 10 that changes no saved byte: its data provider never makes arefnode from a project's USFM, and Paratext does not write arefwrapper to USFM either (paranext-core catalog F-2). Pins:nativePasteFlavor.test.tsx,ClipboardPlugin.test.tsx;unknownClipboardFidelity.test.tsx'sfullshape now pastes asplain+html(changed on purpose); the corpus runs native-paste rows in Standard and the pane row./is left out. The file stores a break as a bare//, so a break pasted aftera/, before/b, or between a pastedx/andysaved///.$removeUnstorableTextFromNodesremoves a pasted break that a lone/touches, in the pasted text or in the document's text at the insertion point, and reports it as//throughonUnstorableTextRemoved. A break beside text or another break (////) goes in as before. Covers the node copy, Ctrl+V's html, drops and locked structure's sanitizer;$textAroundSelectionnow reads a neighbouring break as//./, and lands where it was dropped (catalog CB-23, CB-24). With structure off or guarded, the browser removes the dragged text before it inserts it, and the drop's filter then left out a/that would land against a/(or an optional line break), after its source was already gone: dragging the lone/out ofa/bto just afterc/savedab c/.UnstorableTextPluginnow judges the drop atDROP_COMMAND, before anything is removed, and refuses a move that would leave anything out, reported once with the newsource: "move"(12ff8fce). Separately, a move within one line went in at the start of the line, because Lexical rewrites a changed text node whole (setTextContent), which moved the browser's drop point to the node's start. So the editor now performs every within-editor drag-move in these views itself, in one update and one undo step, through the sameREMOVE_TEXT_COMMANDandCONTROLLED_TEXT_INSERTION_COMMANDthe browser's move dispatches, so every guard and filter still applies (9ff56908). A drop at or inside what was dragged changes nothing. Copy-drops and drops from other programs are unchanged. Under protected structure a drop goes in at its drop point too (85c447a4):StructureKeyboardPlugininserted it at the selection, still the dragged text, and now leaves a move within the editor to this move.main; catalog CB-21). Plain text of several lines pasted, dropped or typed with Enter into the blank line after a bare chapter line, or into text with no paragraph marker, used to save run together asonetwo.\pparagraph. This covers the blank line after a bare chapter line, and unmarked text in Standard view too. Enter used to make a second unmarked line, which saved run together (\c 2 onetwo).ImpliedParaNode.insertNewAfternow creates a\p, and a paste's line breaks use the same split. The Enter menu's marker choice now applies there too. Protected structure still refuses Enter.beforeinputinsertLineBreak, and a typed newline. Pasted or dropped<br>and a node copy's line breaks split the paragraph. They become a space instead under protected structure, in a note, or on the\idline. A lone pasted\rcounts as a line break. NewLineBreakPluginis mounted in every view.\v 1 first x yverse text). Now each\n,\r\nand\rsplits the paragraph, or becomes a space, exactly as a pasted line break does. In Standard it goes in as Standard's plain-text paste, whose\id-line paste now joins lines with a space instead of running them together. PT-4911's filter still strips\and//from the whole string, with one notice. IME composition is left alone.f2857c60, catalog T-18). As in Paratext 9, the line and paragraph separators stay in the text as characters wherever they are typed, pasted or dropped. One the loaded text holds is kept even when typed text holding a line break lands in the same text node; that used to split it (load\p a<U+2028>b, typex\nyafterb: saveda/bx/y). Only\n,\r\nand\rsplit. A U+2028 also now survives Standard view's marker settle and loading from disk; see "Data-fidelity fixes beyond the clipboard".\idline in any mode. Each run of line breaks now becomes one space, or nothing next to whitespace. Before, a trailing line break joined the paste to the next word (x\nbeforeversesavedxverse), and a line break next to a space doubled it. A trailing line break at a paragraph's very end still leaves a trailing space, as typed text does.EditorRef.paste/pastePlainTextnow log a failed read to the editor's logger instead of leaving an unhandled rejection. They paste nothing from an empty clipboard, and nothing into an editor that became read-only during the read. The permission-deniedalert(), which could never show, is removed, and Paste as Plain Text reads the clipboard once.ClipboardPluginandContextMenuPlugintake an optionallogger.ClipboardPluginregisters it); plain prose pasted under protected structure stays inside the char span it lands in (main'sSELECTION_INSERT_CLIPBOARD_NODES_COMMANDroute does this; this branch pins it with a test).StructureKeyboardPluginreads its paste options through refs, so a view switch no longer re-registers its handlers behind others at the same priority (3ad9745c).Commits and their tickets (oldest first)
db86453fthrough100e8bbe. That is everything PT-4911: Hidden-marker views copy prose and filter \ and // on paste #41 held (up toe7a9373c), plus its/code-review highfixes3e0000d8,b7647eca,85ed24d9and adistrebuild.25c7e77athrough36789846. The first commits add a host-driven native paste, which the html-attribute route replaces;ae302ac3removes it, so it leaves no net diff. This range also holds fixes where PT-4921's paste meets PT-4911's filter (a pasted optional line break against a/, the drag-move fixes).main:eca22e85mergesorigin/main, bringing in PT-4126 (PT-4126: Copy hidden-marker views' notes as prose, without layout spacers #49, "Copy hidden-marker views' notes as prose, without layout spacers");37b7a491(CHANGELOG) andf191b099(dist) follow it. The branch is up to date withmain. See "Merged with PT-4126" below for how the two copy writers became one.4e389fe7(entered control characters go in as spaces),f46ffc9f(a marker settle keeps a U+2028),4c144a28(dist),0c42ac47(USX read keeps U+2028, U+2029 and U+0085) and36789846(utilitiesdist). They grew out of the U+2028 work in PT-4921 and touch typing and loading, not only the clipboard; see "Data-fidelity fixes beyond the clipboard" below.Since the
/code-review highroundsOne round ran on each ticket's pair of PRs. Its fixes are folded into the bullets above; for reviewers who read the earlier descriptions, they are:
3e0000d8(no hidden-content question before a refused cut or an empty paste),b7647eca(an authored zero-width space counts as text when judging a/join;$isCaretHostinstead of$isCursorPlaceholderOnlyText),85ed24d9(a range running out of an expanded note is judged in its paragraph).b8687c24(multi-line plain text pasted over Select All keeps its lines),3ad9745c(StructureKeyboardPluginpaste options through refs),88f8e72a(removedBookAndChapterLinesPlugin's unreachable line-break handler:LineBreakPluginclaims it first and runs Enter),100ee439(doc fixes).12ff8fceand9ff56908(with2f0eb805moving the drop point's DOM range intodropPoint.utils), and U+2028/U+2029 kept as charactersf2857c60.37b7a491: the CHANGELOG now records the drag-move changes, including the"move"source as a public API change, and that a cut over hidden content that would join two/is not asked about and copies nothing.Merged with PT-4126 (#49)
main's PT-4126 copies hidden-marker views' notes as prose, without layout spacers, through its ownHiddenMarkersViewCopyPlugin. This branch'sHiddenViewCopyPlugindid the same job, so the merge (eca22e85) keeps one plugin,main'sHiddenMarkersViewCopyPlugin(atCOMMAND_PRIORITY_LOW), and deletesHiddenViewCopyPlugin.tsx; its tests moved tohiddenViewCopy.test.tsx. A hidden-marker view's copy now writes:text/html: the view's html, carrying PT-4921's node-copy attribute.application/x-lexical-editor: PT-4921's tagged node copy (platformEditor/hidden-markers).text/plain: PT-4126: Copy hidden-marker views' notes as prose, without layout spacers #49's$selectionPlainText(no note spacers or empty-span placeholders, one space at a note's edges,~spelling), filtered by$isLeftOutOfHiddenViewCopy(no gutter glyphs, no hidden captions).Other parts of the resolution:
registerViewCopyWritergained an optional$removeRange, so a cut keeps the book and chapter lines.InvisibleCharactersPlugin's copy handler is removed (as onmain);hiddenViewCopySpellingis the one spelling path.CopyLimitPlugincombinesmain's$overLimitTextand fit with this branch's hidden-text predicate.$selectionShownTextread an element point at a range's end as offset 0, so after the merge a range ending right before an optional break copied the wrong text.$offsetInTextand$liesBeforeElementPointmoved tocopyLimit.utils.tsand are shared. This is the same class of bug as CB-19.Data-fidelity fixes beyond the clipboard
These came out of the U+2028 work above. They are not clipboard changes: they fix how characters are typed in and loaded, in every view.
f46ffc9f, paranext-core catalog T-19). The tier-2 rebuild spelled its display-only line break as U+2028 in the fragment it re-reads, and removed every U+2028 before tokenizing, so a real one in a rebuilt paragraph was lost (in Standard view, typingaa<U+2028>bbon a chapter line savedaabb, and a typed marker settled a paragraph holding one without it). The display-only break (DISPLAY_LINE_BREAKintier2Rebuild.utils.ts) is now U+000B. No text can hold it: ParatextData reads it as a space, and the editor takes an entered one in as a space (next item). It is still whitespace to every fragment reader that counts positions, and it never leaves the fragment. TJ ruled this on 2026-11-11. The formerit.failsinchapterLineEditing.test.tsxnow passes, and a new test checks that a stored U+2028 and U+2029 survive a typed marker's settle in Standard and Unformatted view, with no U+000B reaching the text nodes, the page or the saved document.4e389fe7, paranext-core catalog EV-6). That covers typing,CONTROLLED_TEXT_INSERTION_COMMANDandbeforeinputinsertText, plain-text, html and node-copy paste, and drop. Every C0 control character but tab, line feed and carriage return (U+0000–U+001F) becomes a space, as ParatextData's reader treats them.InvisibleCharactersPlugin, which already dropped an entered U+FEFF, now does both throughnormalizeEnteredText, and Standard view's typing and paste use the same rule. Standard view already did this, except for U+0000; the other views took these characters in as they were. A space rather than nothing keeps the text on either side apart. Stored and loaded text is never changed, following the U+FEFF precedent. This also keeps the editor's own uses collision-free: U+001A is Standard view's no-break-space stand-in, and U+000B is the settle's display-only break. 51 new tests failed before the fix.0c42ac47, pre-existing onmain; paranext-core catalog F-9, T-18 updated).usxStringToUsj(packages/utilities) read USX with xmldom's default XML 1.1 line-end rules, which turn these characters into\nin text and a space in attribute values; a text node holding only one was dropped as formatting whitespace. The next save then wrote a space. It now uses XML 1.0 rules (only\r\nand\rbecome\n), matching how ParatextData writes USX, and Paratext 9 keeps these characters. This fixes the editor, the finder and every otherusxStringToUsjconsumer. Entry inpackages/utilities/CHANGELOG.md.docs/standard-view-invariants.mddocuments the reserved U+000B. It also corrects an old claim that ParatextData keeps U+000B and U+000C: its reader turns every C0 control character but tab, LF and CR into a space. The platform CHANGELOG covers the first two items.Second
/code-review highroundfff9ee32: a paste the hidden-marker views insert themselves (plain text filtered for\or//, joined into one line under protected structure, or a Markers view copy without its marker tokens) over a range across a span's edge no longer moves the text after the range into the span.UnstorableTextPlugindispatches the newBEFORE_PLAIN_TEXT_INSERTION_COMMANDbefore inserting plain text itself, andInlineRangeEditPluginremoves the range there, as it already did before Lexical's own plain-text paste. 10 newpasteOverInlineRangetests failed before the fix.72884b27: doc fixes.LineBreakPlugin's clipboard rule applies in a view without spacing too, and a refused"move"can be for a\as well as a/.2be13cda:dist.944437ec:htmlWithNodeCopybuilds its carrier span with the DOM (setAttribute, thenouterHTML) instead of escaping the node copy by hand; the value read back is the same, and the round-trip tests (quotes,&,<, an attribute-closing attack string) still pass and fail when the escaping is removed. CodeQL did not recognize the hand-written escaping.7259f000:dist./check, a large paste parsed more than once, the deniedclipboard-readalert replaced by a log, and a line break over a range on a chapter line (it runs as Enter, which does the same).Why review this
Part of the Standard-view epic's data-fidelity work: catalog rows CB-1 and CB-5 (PT-4911) and CB-3 / EV-1 (PT-4921), found by the PT-4197 audit. In Simple view, copying across a paragraph break couldn't be pasted, a cut text was lost until Undo, and any paste with
\or/(a URL, "and/or") was silently refused. Ctrl+V in Simple and formatted views lost attributes, footnote details and figures from the pasted copy. The fixes beyond the clipboard (catalog T-19, EV-6, F-9) stop U+2028, U+2029 and U+0085 from being lost on load or in a marker settle, and keep entered control characters out of the text.Changes
PT-4911:
hiddenViewCopy.utils.ts+ the copy/cut writer for hidden-marker views (plain text without gutter glyphs or unknown-construct text; html without glyph spans; the editor's own flavor, which PT-4921 replaces with the tagged node copy). After the merge ofmain, the writer ismain'sHiddenMarkersViewCopyPlugin(see "Merged with PT-4126"), and its tests are inhiddenViewCopy.test.tsx.CopyLimitPlugin's over-limit copy combinesmain's$overLimitTextand fit with the same hidden-text predicate.UnstorableTextPlugin+unstorableText.utils.ts+unstorable-text.model.ts(shared-react): the filter, its paste/drop/typing routes, the delete check and the notice. Mounted in every view, acting only in hidden-marker ones, where it replacesCommandMenuPlugin(the Markers view keeps that). Word and line deletes are measured by$probeDeleteExtent(hiddenContentDelete.utils.ts), shared withHiddenContentDeleteGuardPlugin.StructureKeyboardPlugintakes$filterPastedNodesfor its locked html path (hidden views only).HiddenContentDeleteGuardPluginasks before a paste only when it will insert something, and not before a cutUnstorableTextPluginrefuses.shared-reactnow depends on@lexical/clipboard(same version aspackages/platform).clipboardViewsCorpus.test.tsx): paragraph-structure rows runoff,guardedandprotected; rows for "and/or", 1/2, a URL, ranges holding a figure; hidden-view copies checked for no gutter text, no caption, and the collapsed note's text. The 44 PT-4911 pins are gone.PT-4921:
clipboard.utils.ts(shared-react): node-copy readers and writers, the html attribute (data-platform-editor-node-copy) and the marker-showing mark (data-platform-editor-marker-text),removeMarkerTokens/withoutMarkerTokens,registerPasteAsClipboardRead,registerTabsAsSpaces;LEXICAL_CLIPBOARD_TYPEexported there.bookAndChapterLines.utils.ts+BookAndChapterLinesPlugin(Select All keeps the book and chapter lines),LineBreakPlugin+lineBreakText.utils.ts(Shift+Enter and inserted line breaks),dropPoint.utils.ts(where a drop lands, shared withParaMarkerSelectionPlugin),ImpliedParaNode.insertNewAfter(shared).ClipboardPlugin.pastesAsClipboardRead/logger,ContextMenuPlugin.logger,StructureKeyboardPlugin.nodeCopyNamespace/removesMarkerTokens/logger.ENGINE_MARKER_NAME_BYTESmoved tosharedand is re-exported under its old name.hiddenViewSelectAllClipboard,hiddenViewSelectAllEdit,bookAndChapterLines.utils(47),nativePasteFlavor,standardCopyElementPoints,pastedOptionalBreak(91),lineBreak,hiddenViewDragMove; the corpus models Ctrl+V through the html attribute and turns its PT-4921 pins into passing rows.Beyond the clipboard:
tier2Rebuild.utils.ts(DISPLAY_LINE_BREAKis U+000B),InvisibleCharactersPlugin.tsxandwhitespaceDisplay.utils.ts/whitespaceDisplay.plugin.utils.ts(normalizeEnteredText),packages/utilitiesusx-to-usj.ts(XML 1.0 line ends), with tests ininvisibleCharacters.test.tsx,whitespaceDisplay.utils.test.ts,lineBreak.test.tsx,chapterLineEditing.test.tsxandusx-to-usj.test.ts.Both:
docs/clipboard-semantics.md(S5 rewritten; the html route; Select All; drag-move);docs/standard-view-invariants.md; the platform and utilities CHANGELOGs; committeddist(platform and utilities) andetc/platform-editor.api.mdrebuilt.@eten-tech-foundation/platform-editor):EditorOptions.onUnstorableTextRemovedand three types,UnstorableCharacter("\\" | "//"),UnstorableTextRemovedandUnstorableTextRemovedHandler.UnstorableTextRemoved.sourceis"paste" | "drop" | "typing" | "deletion" | "move", andremovedis areadonlyarray. The handler is called synchronously from the editor command or update that made the edit. Everything else changes behavior only, with the same signatures; the fixes beyond the clipboard change no public type.Earlier testing rounds
At
2be13cda(after the second/code-review highround):nxtypecheck and lint: clean.pasteOverInlineRangewithSE_TEST_LIBRARY_VIEWS=1: 96 passed.node scripts/verify-committed-dist.mjs: current.hidden-view-drag-move,hidden-view-clipboardandin-editor-pastepassed.7259f000(the DOM-built carrier): shared-react 2717, packages/platform 7921 passed (99 expected-fail); e2ehidden-view-clipboardandin-editor-pastepassed,clipboard-usfm-round-trippassed on retry (first attempt: Electron "execution context was destroyed").Earlier, at
36789846(after the fixes beyond the clipboard):nxtypecheck and lint: clean.SE_TEST_LIBRARY_VIEWS=1, the touched and related suites: 4172 passed.node scripts/verify-committed-dist.mjs: current.no-unasked-edits-invisible-characters-*specs (which now seed, type and paste U+2028, U+2029 and U+0085, each saved as itself),no-unasked-edits-power-standardandno-unasked-edits-simple-paragraph-structurepassed. Against4c144a28, earlier in the same round,hidden-view-clipboardandin-editor-pastepassed too.Earlier rounds, kept for the coverage they show:
f191b099, after the merge ofmainwith PT-4126:nxtypecheck and lint 0 errors; shared-react 2717 passed; packages/platform 7874 passed, 100 expected-fail; withSE_TEST_LIBRARY_VIEWS=1, the clipboard corpus plus the copy, gate,invisibleCharactersandcopyLimitsuites 2327 passed, 127 expected-fail; e2ehidden-view-drag-move,hidden-view-clipboard,in-editor-pasteandclipboard-usfm-round-tripall passed on the first try.93f6f465, before the merge (after the/code-review highrounds, the drag-move fixes and the U+2028 change): shared-react 2717 passed; packages/platform 7849 passed, 100 expected-fail;lineBreakandchapterLineEditingwithSE_TEST_LIBRARY_VIEWS=1703 passed, 1 expected-fail (the U+2028 settle gap, since fixed); the same four e2e specs passed (hidden-view-drag-moveon retry after the known cold-start timeout;in-editor-pasteflaked once at the Edit flyout's Paste step, then passed on retry and in two more clean runs).unstorableText.utils.test.ts($deletionJoinsSlashes,$transposeJoinsSlashesrows),unstorableTextDeletion(101),unstorableTextTypingwith transpose rows (39),pastedOptionalBreak(91),bookAndChapterLines.utils(47),lineBreak(149 when added, 111 red before the fix). shared: 951 passed.main, both branches were rebased ontomainafter the Scribe removal (scripture-editors#2) and PT-4982 (#2973); no Scribe-specific content is left. The PT-4911 work was also re-run againstmainbefore that rebase.no-unasked-edits-*specs,no-unasked-edits-footnotes-pane-simpleandsimple-view-popupspassed.🤖 Generated with Claude Code
https://claude.ai/code/session_018WCZ5GVaw3B7KJ9xc7TC6c
This change is