Skip to content

Add and edit annotations in the Scene Editor (#426, #427-#438) - #509

Open
mayokunl wants to merge 2 commits into
mainfrom
426-add-edit-annotations
Open

mayokunl wants to merge 2 commits into
mainfrom
426-add-edit-annotations

Conversation

@mayokunl

Copy link
Copy Markdown
Collaborator

Pull Request Summary

Closes #426
Closes #427
Closes #428
Closes #429
Closes #430
Closes #431
Closes #432
Closes #433
Closes #434
Closes #435
Closes #436
Closes #437
Closes #438

What: Editors can now add and edit annotations in the Scene Editor: text labels, straight lines, pointer lines that connect a label to a feature, arrows, and see-through highlight regions. Each one can be selected and moved, reshaped, styled, reordered, or deleted.

Why: Until now, scenes could only hold images, plus annotations imported from existing bones that couldn't be edited. Editors need to mark up bone images themselves to identify anatomical structures.

How: A new Add annotation row picks a tool, and a toolbar shows controls for whatever is selected. Every change edits the scene's working copy and is stored with the next Save, the same as image edits (#451). No server changes were needed.

Testing: Two new test files cover each sub-issue end to end, including saving and reopening the scene. Full suite: 742 passed, 5 skipped. ESLint passes.

Screenshots

image

PR Checklist

  • Project builds and runs
  • Tests and linters pass
  • Any related documentation has been updated, including JSDoc comments or docstrings

Detailed Description

Features by issue

Issue What it does
#427 Create text label Click Text label, then click the canvas. The label is placed there, selected, with its text ready to type over. Multi-line text works.
#428 Edit label text Edit in the toolbar's Text box, or double-click the label. An empty label isn't applied: an error shows and the last text is kept.
#429 Move and resize labels Drag the label or use the arrow keys (Shift moves 10px). Resize with the corner handle or the Text size field (8–200px).
#430 / #437 Delete Delete annotation button or the Delete key. Other annotations, and other scenes, are unaffected.
#431 Straight line Drag to draw. Each end has a handle, and the whole line can be dragged.
#432 Pointer line Select a label, click Add pointer line, then click the feature. Moving the label moves the line's label end, and deleting the label deletes its pointer lines. Labels imported from existing bones work the same way.
#433 Arrow Drag from the tail to where it points. The head is drawn at the end.
#434 Highlight region Click each corner, then finish with Finish shape, Enter, double-click, or a click on the first corner. Escape cancels; Backspace removes the last corner.
#435 Edit a region Drag a corner handle, or drag the whole shape.
#436 Style a region Color picker and fill opacity. Opacity is capped at 0.8 so the bone stays visible. Labels, lines, and arrows have a color picker too.
#438 Layer order Send to back, backward, forward, or bring to front. Annotations always draw above images, so image order never changes.

Editors can now add labels, lines, pointer lines, arrows, and
highlight regions to a scene, then select them to move, reshape,
style, reorder, or delete them. Every change edits the working copy
and is stored with the next Save, like image edits (#451).

- Text labels (#427-#430): click to place, edit the text in the
  toolbar or by double-clicking, multi-line text, move by dragging
  or arrow keys, resize from the corner handle or the size field
  (8-200px). An empty label is not applied; the last text is kept.
- Lines and arrows (#431, #433): drag to draw; each end has a handle.
- Pointer lines (#432): drawn from a selected label to a feature and
  linked by labelId. Moving the label moves the line's label end,
  resizing re-anchors it on the label's edge, and deleting the label
  deletes its pointer lines. Imported legacy labels and their
  pointer lines behave the same way.
- Highlight regions (#434-#437): click each corner, then finish with
  the button, Enter, double-click, or the first corner. Corners and
  the whole shape can be dragged. Color and fill opacity are
  editable; opacity is capped at 0.8 so the bone stays visible.
- Layer order (#438): send to back, backward, forward, or to front.
  Annotations always draw above images, so image order is unchanged.

All tools share the scene coordinate system. Edits are made on the
annotation as drawn and mapped back into its stored frame, so
annotations imported with a bone keep following their image when it
is moved, resized, flipped, or rotated. Stored formats are preserved
(x1/y1/x2/y2 vs points, [x, y] vs {x, y}). No server changes: the API
already stores annotations as opaque objects.

Also: a new mouse press now clears the flag that skips the click
after a drag, so a drag whose click never arrives can no longer
swallow the user's next click.

Tests: sceneAnnotations.test.js (geometry and storing linked edits)
and sceneAnnotationEditor.test.js (each sub-issue end to end,
including save and reopen).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread templates/tests/sceneAnnotationEditor.test.js Fixed
Comment thread templates/tests/sceneAnnotationEditor.test.js Fixed
CodeQL (js/incomplete-multi-character-sanitization) flagged the regex
that removed <script> tags from boneset.html in the annotation editor
tests. Parse the page and remove its script elements instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@mudabs mudabs left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you Mayo for the substantial work on the annotation editor. The new label, line, arrow, pointer-line, and highlight functionality is a meaningful addition, and the accompanying tests make the behavior much easier to review.

I’m requesting one local behavior fix before approval. This is independent of the database/object-storage discussion; that can be handled in a separate follow-up issue.

File: templates/js/scenes.js

The issue involves:

  • cancelTool() around line 1404
  • setTool() around line 1387
  • startLineDraw() around line 1475
  • handleAnnotationKey() around line 1776

startLineDraw() registers document-level mousemove and mouseup listeners. However, cancelTool() and setTool() clear state.draft without removing those listeners.

This creates a reproducible failure:

  1. Open a scene and select the Line or Arrow tool.
  2. Begin dragging on the canvas.
  3. Press Escape before releasing the mouse.
  4. Move the mouse again.

Escape calls cancelTool(), which sets state.draft to null. The existing mousemove listener remains active and then executes state.draft.points[1] = ..., causing a TypeError because state.draft is null.

Switching tools while a line or arrow drag is active can leave the same kind of stale listener behind.

Please centralize the cleanup for an active line/arrow drag so that:

  • cancelTool() removes any active document-level drag listeners.
  • setTool() also cleans up an existing drag before replacing the current tool.
  • Normal mouseup remains safe and idempotent.
  • The draft state is cleared without allowing later mousemove events to access it.

Please add a regression test in templates/tests/sceneAnnotationEditor.test.js, near the existing line/arrow drawing tests. The test should start a line or arrow drag, press Escape, dispatch another mousemove/mouseup, and verify that no exception occurs and no annotation is created. It would also be useful to cover switching tools during an active drag.

The current local scene persistence and the existing base64 image approach can remain unchanged for this PR. Production image storage/database changes should be tracked separately.

Thank you again for the effort on this feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants