Skip to content

fix(llm): keep null tool call arguments and send json.raw tool schemas as written - #272

Open
wolfy-j wants to merge 1 commit into
masterfrom
fix/llm-tool-json-null-order
Open

wolfy-j wants to merge 1 commit into
masterfrom
fix/llm-tool-json-null-order

Conversation

@wolfy-j

@wolfy-j wolfy-j commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

Depends on wippyai/runtime#1088 (json.null, json.decode_with_nulls, json.raw). Do not merge before a runtime release that includes it: on an older runtime json.decode_with_nulls does not exist and every tool call decode fails.

Defect

  • A model's tool call {"q": null, "r": 1} reaches the tool as {r = 1}. The tool cannot tell "the model set q to null" from "the model left q out".
  • A tool schema cannot reach a provider with its property order. Tables encode with sorted keys, and the Google mapper indexes the schema, so a schema passed as a JSON value fails there.

Cause

  • Every site that reads tool call arguments uses json.decode, which drops null members: OpenAI and OpenAI-compatible arguments strings, streamed Claude and Bedrock input JSON, and the history mappers that decode a stored function_call.arguments string before resending it (Claude, Bedrock, Google).
  • Claude (non-streamed), Bedrock (non-streamed) and Google (both) return arguments as JSON objects inside the response, so the response decode drops their nulls too.
  • google_mapper.map_tools passes every schema through filter_tool_schema, which mutates a table.

Fix

  • The string decode sites use json.decode_with_nulls. A null argument reaches the tool as json.null, and a resent assistant tool call carries null again.
  • For object arguments, the client keeps the response decode as it is and, when the response has tool calls, reads their inputs (content[i].input, output.message.content[i].toolUse.input, candidates[c].content.parts[p].functionCall.args) from a json.decode_with_nulls decode of the same body or stream event. Decoding the whole response with nulls is not an option: envelope fields such as Claude's stop_sequence and text block citations are null in normal responses, and as truthy json.null they would break every if field then and field or default read. The second decode runs only for responses that contain tool calls.
  • Tool schemas: a json.raw schema is a JSON value the encoder writes as is, so the OpenAI, OpenAI-compatible, Claude and Bedrock mappers send it with its property order without change. Google sends a json.raw schema as parametersJsonSchema (JSON Schema, accepted as written) and keeps the filtered parameters path for table schemas, since a raw value cannot be filtered without decoding it and losing the order.
  • structured_failure_contract_test pins the source line of the Google mapper's "Invalid Google response structure" error; the expected line moves from 395 to 400 with the map_tools change.

Tests

New cases in the existing driver suites (16):

  • openai/mapper_test, openai_compat/mapper_test: "keeps null tool call arguments as json.null", "sends a json.raw tool schema with its property order".
  • claude/mapper_test, bedrock/mapper_test: "resends null tool call arguments as null", "sends a json.raw tool schema with its property order".
  • google/mapper_test: "resends null tool call arguments as null", "sends a json.raw tool schema as parametersJsonSchema with its property order".
  • claude/client_test: "keeps null members of a streamed tool input as json.null", "keeps null members of a tool input as json.null and leaves envelope nulls absent".
  • bedrock/client_test: "keeps null members of a streamed toolUse input as json.null", "keeps null members of a toolUse input as json.null".
  • google/client_test: "keeps null members of streamed functionCall args as json.null", "keeps null members of functionCall args as json.null".

All tests ran with a wippy binary built from the runtime PR branch:

  • master with the new tests: 12 fail (every null case, and the Google schema case, which errors indexing the raw value). The four "sends a json.raw tool schema with its property order" cases for OpenAI, OpenAI-compatible, Claude and Bedrock pass on master, because those mappers already embed the schema value; they fail on a runtime without json.raw, and they pin that those drivers need no change.
  • This branch: the llm suite passes (2930 tests, 9 skipped), and wippy lint --level error reports no issues.

Note

Callers get ordered schemas by passing schema = json.raw(text). Tool schemas that the agent module reads from registry input_schema text are still decoded into tables before they reach llm, so their order is still lost there. That is a separate change in the agent module.

…s as written

Tool call arguments decode with json.decode_with_nulls, so a null argument
reaches the tool as json.null. Providers that return arguments as objects
take them from a null-preserving decode of the response. Google sends a
json.raw tool schema through parametersJsonSchema.
@wolfy-j
wolfy-j requested a review from skhaz November 15, 2026 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants