Repository navigation
fix(provider): DeepSeek thinking/action imbalance - reasoning round-trip + real effort ladder - #242
Merged
Merged
Conversation
…rip + real effort ladder
Real-world report: DeepSeek V4.1 Flash spends tens of minutes thinking,
re-reads dozens of files, never acts - the thinking:action ratio explodes.
Diagnosis: model trait (thinking ON at HIGH effort by default, documented)
AMPLIFIED by two genuine harness defects, both fixed here.
Defect 1 (the amplifier): reasoning_content was never round-tripped -
violating DeepSeek's documented tool-call contract. In tool-carrying
conversations the field is the model's CROSS-ROUND WORKING MEMORY
('let the model continue its previous thinking'); AIO dropped it on both
paths (in-turn append_assistant_message built role/content/tool_calls
only; history finish_message_for_api never inserted it though the DB
stores it). Every round started thinking from scratch and re-read files
to re-establish understanding - the re-read loop behind the report.
- New deepseek provider plugin (claims api.deepseek.com; URL/auth/models
delegate to the fallback), with trait additions:
- roundtrip_reasoning_content (default false - OpenAI's strict schema
rejects the unknown field, so the contract must stay provider-scoped)
- adjust_chat_body (default no-op), applied unconditionally in
stream_one_round after generic body assembly
- In-turn: append_assistant_message attaches the round's reasoning as
reasoning_content when flagged (empty stays absent); main loop computes
the flag once per turn, subagent loops wired identically
- History: finish_message_for_api inserts it (assistant role, non-empty,
gated by the same flag); the FE now passes message.reasoning through
buildApiMessages; chat-mode (no tools) stays unflagged per the contract
(server ignores the field without tools)
Defect 2: the reasoning slider was a no-op for DeepSeek. Thinking is ON
at HIGH effort by default and the only documented off-switch is
{'thinking':{'type':'disabled'}} - which AIO never sent. The deepseek
adjust_chat_body override now: Off -> thinking disabled (the slider's
'off' is finally real; BEHAVIOR CHANGE: DeepSeek users at Off no longer
get chain-of-thought output - move the slider to Low+ when thinking is
wanted); Medium -> Low (DeepSeek's real ladder is low/high/max and
upstream collapses medium to high - Low keeps three settings
distinguishable); Low/High pass through. Subagent rounds (which never
inherit the slider) default to thinking disabled - executor role.
Tests: provider +4 (host claim/guards, effort ladder mapping, URL
delegation), round_artifacts +1 (round-trip three branches), messages +1
(history round-trip role/flag/empty gating); full suite 469 green;
npm run verify fully green (87 FE tests, gates rgba=173 any=77 held).
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Real-world report: DeepSeek V4.1 Flash via AIO thinks for tens of minutes, re-reads dozens of files, never starts acting — the thinking:action ratio explodes. Diagnosis: a documented model trait (thinking ON at HIGH effort by default) amplified by two genuine harness defects, both fixed here.
Defect 1 (the amplifier):
reasoning_contentnever round-trippedDeepSeek's thinking-mode docs make it a documented tool-call contract: in tool-carrying conversations every assistant message must carry its
reasoning_contentback — it is the model's cross-round working memory ("让模型继续之前的思考"). AIO dropped it on both paths:append_assistant_messagebuilt{role, content, tool_calls}only — each round's thinking was discarded (RoundResult.reasoning: "kept only for debugging")finish_message_for_apinever inserted it, though the DB has areasoningcolumn per messageEvery round started thinking from scratch → re-read files to re-establish understanding → the re-read loop behind the report.
Fix: a new
deepseekprovider plugin (claimsapi.deepseek.com; everything else delegates to the fallback) + two trait additions:roundtrip_reasoning_content(default false — OpenAI's strict message schema rejects the unknown field, so the contract must stay provider-scoped)adjust_chat_body(default no-op), applied unconditionally instream_one_roundafter generic body assemblyThe flag gates both paths: in-turn rounds (main loop computes it once per turn; subagent loops wired identically) and history replay (the FE now passes
message.reasoningthroughbuildApiMessages; chat-mode without tools stays unflagged — the server ignores the field there per the docs).Defect 2: the reasoning slider was a no-op for DeepSeek
Thinking is ON at HIGH effort by default; the only documented off-switch is
{"thinking":{"type":"disabled"}}— which AIO never sent. The plugin'sadjust_chat_bodyoverride now:thinking: disabled— the slider's "off" is finally real (behavior change: no more chain-of-thought at Off; move to Low+ when thinking is wanted)reasoning_effort: low(DeepSeek's real ladder is low/high/max; upstream collapses medium→high — Low keeps three settings distinguishable)reasoning_effort: highSubagent rounds (which never inherit the slider) default to thinking disabled — executor role, not high-effort reasoning.
Model-side facts (not "fixed", documented)
Verification
npm run verifyfully green (87 FE tests, gates rgba=173 / any=77 ratchets held — file-size included, lesson from fix(harness): pwsh object-output P0s - formatter loss on exit, rc-0 masking, CLIXML stderr #241 applied)