Skip to content

PRACTICE scenes + headless MCP: post-review fixes and refactors #41

Closed

Description

@NiclasDev63

Follow-ups from a full functional review of the PRACTICE scene type + the headless/MCP authoring path. The feature works end to end — publish, learner render, grading, persistence and resume were all verified against a real published snapshot (v3), and the notebook compact 4-tab editor (App / Lösung / Erklärung / Testlauf) self-tests to full marks. tsc, vitest (2776 passed) and next build are all green.

These are the leftovers. Paths are relative to apps/app/.


1. Learner-visible: practice work is lost on reload

Repro: complete a practice scene → get graded → reload the page → navigate back to that scene.

Observed: the player still shows the green "Super!" banner, but the app underneath is fully reset — counters at zero, all state cleared, Submit re-enabled. Inconsistent, and it invites a pointless re-submission.

Cause: the grade is server-backed, the work isn't.

  • src/features/learning/course-player/ui/player/lesson-player/progression/lesson-progression.view-model.ts:191 — scenePracticeWork: pendingSubmission?.practiceWork ?? null, i.e. only the in-memory pending-submission map.
  • src/features/learning/course-player/ui/player/lesson-player/components/scene-content.tsx:51 — previous: { work: practiceWork ?? null, grade }.
  • There is no practiceWork column on scene_analytics; only gradedBlocks is persisted.

Fix: persist the submitted practiceWork alongside gradedBlocks and feed it back into previous.work so review mode can rehydrate the app. Until then, at minimum the scene should not present itself as re-submittable.


2. Headless blocker: the auto-created Introduction scene can never be published

Every createLesson plants a default Introduction scene that a pure-MCP agent cannot get past.

  • src/features/course-authoring/lessons/server/ensure-default-introduction-scene.ts:9 writes raw-HTML bytes <div><p></p></div>.
  • src/shared/content/editor/documents/author-document.ts:16 — isBlankHtmlState only matches <p></p>.

The two don't agree, so publish fails at src/shared/content/editor/assessment/publish/publish-validation.ts:116 with UNREADABLE_SCENE and the message "Open the scene and re-save it, then publish again" — an instruction an agent has no way to act on.

Fix at the source: make isBlankHtmlState recognise what the seeder actually writes, or classify the seeded state as EMPTY_SCENE instead.


3. getSceneContent is silently wrong for PRACTICE scenes

src/features/course-authoring/scenes/server/scene-queries.ts:72 has no kind branch — it always reads the collab room. A PRACTICE scene has documentState: null, so a 200-line app comes back as html: "", indistinguishable from an empty DOCUMENT scene.

insertContent already gets this right and tells the caller to use writePractice; the read side doesn't. Branch on kind and either return the practice payload or point the caller at getPractice.


4. Draft SP reads 0 for practice scenes

src/features/course-authoring/scenes/editor/hooks/use-scene-tab-controller.ts:22 computes available SP by summing document question blocks. A PRACTICE scene has none, so the sidebar shows 0 SP while the published maxSp is 10 (summarizePublishedPracticeManifest sums field.points).

Same helper, two different inputs. Draft and published should agree.


5. MCP tool descriptions and schema

  • src/features/course-authoring/tools/scene-tools.ts:26 — createScene's description ("strictly for scene structure and metadata creation. To write content, call insertContent afterwards") contradicts its own kind: PRACTICE field, where insertContent is the wrong next call.
  • src/features/mcp/server/publishing-tools.ts:30 — publishCourse lists only three refusals (no lessons / no scenes / nothing changed). It omits both unvalidated practice and unreadable scene, which are the two an agent actually hits.
  • src/features/course-authoring/scenes/api/scene.schema.ts:132 — writePractice's explanation is nullable() but not optional(), so it is a required key that must be explicitly null. Either make it optional or say so in the description.
  • src/shared/content/practice/grade-practice-submission.ts:91 — validatePractice returns correctAnswer for every field. Author-side only, so low severity, but it means the answer key is readable by any agent holding an authoring token.

6. Polish

  • Publish-refusal toast is English inside an otherwise German UI — the message is built server-side in publish-validation.ts and never goes through i18n.
  • PRACTICE scenes are indistinguishable in the scene list — same icon, same "Interactive canvas" subtitle as DOCUMENT scenes.
  • Reverting an html edit doesn't clear the stale "Geprüft" note until reload. Errs safe (the hash gate is correct server-side), but the UI lies briefly.

7. Product question: an agent cannot go zero-to-published over MCP alone

Verified via the real handleMcpRequest in-process (13 integration tests, all passing, in src/features/mcp/server/practice-scenes.integration.test.ts). An agent can reach a correct, publish-ready draft and then stops: validatePractice deliberately never stamps practiceValidated — selfTest lives on validatePracticeInputSchema (editor/tRPC) and not on validatePracticeSchema (MCP-facing), because only an in-editor self-test proves the app's Submit button is wired.

That is by construction and documented in three places. The consequence worth deciding on: a 12-scene practice course needs 12 manual editor clicks before it can ship. Either accept it as the human-in-the-loop gate, or find a way for an agent to prove Submit-wiring without a human.

Activity

  1. added
    bugSomething isn't working
    ready-for-agentFully specified, ready for an AFK agent
    on Sep 4, 2026
  2. NiclasDev63 commented on Sep 4, 2026

    @NiclasDev63
    ContributorAuthor

    Fixed in #43 — §1–§4, §6 and §7 addressed; §5's correctAnswer deliberately left as is (an authoring token already reads the whole solution via getPractice). §7 converged on one pure checkPracticeScene, so publish, MCP validatePractice and the editor's readiness line all run the same check and no editor visit is needed.

  3. added a commit that references this issue on Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingready-for-agentFully specified, ready for an AFK agent

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions