Skip to content

fix(llm): keep malformed streamed tool arguments from Claude and Bedrock - #269

Open
wolfy-j wants to merge 1 commit into
masterfrom
fix/llm-streamed-tool-arguments
Open

wolfy-j wants to merge 1 commit into
masterfrom
fix/llm-streamed-tool-arguments

Conversation

@wolfy-j

@wolfy-j wolfy-j commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

Problem

Streaming Claude and Bedrock responses replace malformed tool input JSON with an empty object and drop the decoder error. A tool call whose streamed input does not parse (for example a truncated {"query": "x") reaches the host as a valid call with {} arguments and is executed with missing parameters.

The OpenAI and OpenAI-compatible drivers already keep the original string in arguments and raw_arguments and report the decoder diagnostic in arguments_error (#196); the README documents that contract for all tool calls, and hosts reject calls carrying arguments_error.

Cause

claude/client.lua and bedrock/client.lua decode the accumulated input_json_delta / toolUse.input text at block stop and fall back to {} on a decode error. claude/mapper.lua and bedrock/mapper.lua map_tool_calls copy only id, name and arguments, so the diagnostic fields could not reach the response either way.

Fix

  • New wippy.llm.util:tool_arguments library holds the single decode rule for provider argument text: valid JSON yields the decoded value, malformed text stays in arguments and raw_arguments with the decoder error in arguments_error.
  • The OpenAI, OpenAI-compatible, Claude and Bedrock drivers use it; the two existing copies of the rule in the OpenAI mappers are removed.
  • Claude and Bedrock mappers carry raw_arguments and arguments_error onto the contract tool call.
  • A tool block with no input deltas still yields {}, which is how both APIs stream a call with empty input.

Non-streaming Claude and Bedrock responses deliver tool input as a decoded object and are unchanged. Google delivers function call arguments as objects and has no decode step.

Tests

  • claude/client_test: malformed streamed input keeps the raw text and decoder error; empty input stays {} without diagnostics.
  • bedrock/client_test: malformed streamed input keeps the raw text and decoder error; valid input split across deltas decodes without diagnostics.
  • claude/mapper_test, bedrock/mapper_test: streaming response formatting carries both fields onto the contract tool call.

The malformed-input and mapper cases fail before the change. make test in src/llm/test: 2920 passed, 9 skipped. make lint: no issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant