Fix image delivery and simplify the AI sidebar

[TerminallyLazy]

Problem and behavior

Sharing a study could inventory 34 frames while delivering none: the Codex session retained conflicting text-only instructions, and pixels depended entirely on the model requesting them. Scoped turns now capture and attach initial images before the model answers, explicitly authorize the declared tools, and record accepted image delivery. Capture failure cannot silently downgrade to text-only inference.

The reading-room sidebar replaces Prepare sharing with Images + Send with images, captures fresh images on subsequent turns, keeps delivery and Stop/Take over near the composer, moves voice setup behind a header control, and collapses completed native activity. Preview images are created only with actual sources; the obsolete detector suppression was removed.

Native adaptations of the requested OpenMed workflows add scoped technical DICOM metadata, literal source-anchored report structuring, and richer PubMed abstract metadata plus saved query receipts. Existing report-save review and explicit Jev confirmation remain intact. These adaptations do not install NER models or add SR/full-text parsing.

Validation

• Real signed-in gpt-6-astra through the normal backend and isolated Electron sidebar: 34/34 generated frames delivered, random pixel-only marker correctly counted, technical metadata queried, native slice/window actions and reading-view observation. No patient images or Realtime connection.
• Synthetic final UI flow: 34/34 frames, 38 image observations, two native actions, no saved image payloads; logo returns to local loader.
• Focused backend subscription/exploration, report/metadata and research checks passed; viewer type check/build and 14 targeted browser tests passed.
• Live public PubMed query returned original abstracts, publication metadata and a reproducible NCBI query receipt.
• Visual inspection at the narrow desktop rail; design scan has only a pre-existing local-loader grid advisory outside this change. No new suppression.

This verifies transport and bounded workflows, not clinical accuracy or exhaustive viewer-tool/modality parity. Updated runtime guidance and runbooks record those limits.

High-level PR Summary

This pull request fixes a critical image delivery bug in AI-powered study exploration and simplifies the AI sidebar interface. Previously, sharing a study could inventory frames without delivering any pixels to the model. The fix ensures that scoped turns capture and attach initial images before the model responds, explicitly authorize declared viewer tools, and properly track image delivery. The sidebar UI was streamlined: the Prepare sharing step was removed in favor of an explicit Images selector with Send with images, fresh captures happen automatically on each turn, and delivery counts plus Stop/Take over controls remain visible near the composer. Voice setup moved behind a header button, and completed native activity is now collapsed. Three native tool adaptations inspired by OpenMed workflows were added: scoped technical DICOM metadata extraction, literal report structuring with exact source offsets, and enhanced PubMed search with structured abstracts, publication metadata, and query receipts. These additions do not install NER models or add SR/full-text parsing.

⏱️ Estimated Review Time: 30-90 minutes

💡 Review Order Suggestion

Order	File Path
1	.impeccable/config.json
2	AGENTS.md
3	CLAUDE.md
4	README.md
5	WARP.md
6	backend/clinical/AGENTS.md
7	backend/clinical/ai_codex_tools.py
8	backend/clinical/ai_codex.py
9	backend/clinical/ai_radiology.py
10	backend/clinical/ai_research_worker.py
11	backend/clinical/ai_fixture_server.py
12	backend/tests/AGENTS.md
13	backend/tests/test_ai_radiology.py
14	backend/tests/test_ai_codex_exploration.py
15	backend/tests/test_ai_research.py
16	roadmap/AGENTS.md
17	roadmap/ai-backend/CODEX_STUDY_EXPLORATION.md
18	roadmap/ai-backend/OPENMED_ADAPTATIONS.md
19	viewer/assets/live/controller.ts
20	viewer/assets/live/exploration.ts
21	viewer/assets/live/exploration-panel.ts
22	viewer/assets/live/panel.ts
23	viewer/assets/radsysx-viewer.css
24	desktop/AGENTS.md
25	desktop/scripts/ui-import-smoke.mjs
26	viewer/assets/live/AGENTS.md
27	viewer/scripts/test-live.mjs

[qodo-code-review]

ⓘ Qodo reviews are paused because the subscription is no longer active. Ask your workspace admin to reactivate the subscription to resume reviews. Manage billing

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 465cae36-d125-443b-a1ed-27430fc3101d

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}