Skip to content

Keep each turn's outcome through engine finalization and use it for workflow and cron completion - #481

Open
groeneai wants to merge 1 commit into
ClickHouse:mainfrom
groeneai:groeneai/turn-outcome
Open

groeneai wants to merge 1 commit into
ClickHouse:mainfrom
groeneai:groeneai/turn-outcome

Conversation

@groeneai

@groeneai groeneai commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Closes #292 (the remaining scope; #428 fixed the Claude translator).

What breaks today

AgentEngine.run() returns normally for a turn that hit max turns, got an API error, was aborted or crashed after streaming. Cron then logs success, and a workflow run ends done because it reads sessions.status, which such a turn leaves (or restores to) active. An aborted turn gets no inline notice, and abnormal backend results are not logged.

Change

  • The engine records a TurnOutcome (completed / failed / interrupted + reason) where each turn ends. A normally finished turn publishes it only after it is saved; a failed save publishes failed.
  • run_cron / run_persistent_cron return TurnResult(text, outcome) for their own turn; cron logs error with the reason. run() still returns str.
  • engine.watch_turns(session_id) reports the latest outcome of a session's turns, background continuations included. A workflow run watches its own session through the existing quiescence wait and ends failed on an abnormal outcome (a missing one fails closed). The session-status check is gone, so turn failure stays separate from session health.
  • A failed or interrupted turn reported by the backend (max turns, an API error, an abort) gets an inline Turn <status>[: reason] notice and one WARNING line. The engine's own crash, stop and timeout paths keep their existing notices and log lines.

Scope is option A as agreed in #292 (#292 (comment)). Unchanged limits: a parked isolated cron run is logged from its launching turn; a workflow follow-up turn that ends without content does not change the result; the pre-query drain desync is untouched.

Behaviour changes

  • Failed or interrupted cron runs (a user stop included) are logged error, so they do not anchor the interval and are caught up once at restart. I can give interrupted runs their own status if you prefer.
  • A workflow run whose last turn was interrupted ends failed; a stop through Nerve is still recorded killed by the stop listener.
  • A review-loop leg that hits max turns is retried, then escalated, instead of going to the verifier.

Validation

tests/test_turn_outcomes.py: 26 integration tests (real engine and database) over the engine, cron and workflow paths; each of 13 mutations of the change fails them. Full suite: 3952 passed (main: 3926).

🤖 Generated with Claude Code

…orkflow and cron completion

AgentEngine.run() returns normally for every turn that does not raise out
of the engine, so its callers could not tell how a turn ended. Cron logged
"success" for any normal return, and workflow runs inferred failure from
sessions.status, which a failed turn deliberately leaves (or restores to)
"active". A max-turns, API-error, aborted or crashed turn was therefore
reported as successful automation.

The engine now records a TurnOutcome (completed / failed / interrupted,
plus the reason) where each persisted turn ends:

- run_cron / run_persistent_cron return TurnResult(text, outcome) for the
  invocation's own turn (via _run_turn, which holds run()'s old body;
  run() still returns str), and cron logs "error" with the reason when
  the outcome is not completed.
- watch_turns(session_id) reports the latest outcome of a session's
  persisted turns, including autonomous background-continuation turns.
  A workflow run watches its own session over the launch and the existing
  quiescence loop and ends "failed" when that outcome is not completed;
  a missing outcome fails closed. The session-status inference is gone,
  so turn failure no longer depends on session health.
- The outcome is published only after the turn is persisted
  (_finalize_turn wraps the old body, now _persist_turn); a persistence
  failure or a cancel while saving publishes failed / interrupted.
- The autonomous drain exits that persist a partial turn (idle timeout,
  stream end, cancel) set the matching status instead of "completed".
- Every abnormal backend end gets an inline "Turn <status>[: reason]"
  notice (previously only failed turns with an error did) and one
  WARNING log line.

Scope follows the owner's choice of option A on ClickHouse#292: a parked isolated
cron run is logged from its launching turn, content-less autonomous turns
are not tracked, and the pre-query drain desync is left alone.

Closes ClickHouse#292

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@groeneai

groeneai commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author
Internal second-model review: adjudication log (click to expand)

Pre-publication review by an independent model (engine: codex; 3 findings) plus my own review.

# Sev Finding Verdict Evidence / action
1 ⚠️ Crash, hard-cancel and background stream-end paths do not get the new Turn <status> notice and WARNING DISAGREE #292 asked for the notice and logging on backend results; those paths already show Agent error: ... / [Stopped by user] / [Background turn interrupted ...] and log at ERROR / INFO / WARNING, and keep doing so. The PR text now says the notice is for backend-reported results.
2 ⚠️ A forced cancel publishes interrupted before the best-effort save of the partial text DISAGREE interrupted is how the turn ended whether or not the partial row is saved; cron and workflow treat any non-completed outcome the same and only read the outcome after the turn has returned.
3 ⚠️ Worker onboarding still logs "setup session completed" whatever the outcome DISAGREE Unchanged from main, and it writes no cron log or workflow row, so it is outside cron/workflow completion. I can add the branch here if you want it.

Severity: ❌ blocker / ⚠️ major / 💡 nit. DISAGREE verdicts carry recorded evidence and are terminal per finding.

Session id: cron:clickhouse-review-slot-8:20261001-135100

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Claude backend silently swallows abnormal turn terminations (max_turns, API errors, aborts)

1 participant