Repository navigation
fix(ai): keep Anthropic stream tool arguments after server tool blocks - #984
Conversation
dustinbyrne
left a comment
There was a problem hiding this comment.
thanks @breken-ai
this looks correct to me (and if it is, we could benefit from the same strategy in posthog-js) - worth @PostHog/team-ai-observability signing off
Stream delta and stop events address content blocks by their index in the
message. The accumulator looked them up in content_blocks, which skips block
types it does not format (server_tool_use, web_search_tool_result, ...). After
a web search, a client tool call's input_json_delta events pointed past the
end of the list (arguments captured as {}) or at a different tool block.
Keep a list that lines up with the stream index and use it for tool deltas
and finalization. content_blocks and the captured output are unchanged.
9e2eb58 to
98efd59
Compare
|
@dustinbyrne thanks for the merge. I checked
The test #5113 added covers the thinking-block case. To check the server-tool cases, I ran a local test (not committed) through the wrapped client with the same block layouts as
No other file in |
💡 Motivation and Context
Anthropic stream events address content blocks by their
indexin the message._AnthropicStreamAccumulatorpassedcontent_blockstohandle_anthropic_tool_deltaandfinalize_anthropic_tool_input, which look the block up withcontent_blocks[event.index]. Butcontent_blocksonly holds the block types the wrapper formats (text, thinking, redacted_thinking, tool_use). Aserver_tool_useorweb_search_tool_resultblock still takes an index in the stream and is skipped in the list.So once a response uses a server tool (web search, web fetch, code execution) before a client tool call, the list and the stream index no longer line up:
input_json_deltaevents point past the end of the list, and$ai_output_choicesrecords the call witharguments: {};The fix keeps a second list that lines up with the stream index (an empty placeholder for block types that are not formatted) and uses it only for tool deltas and finalization.
content_blocksand the captured output are unchanged.💚 How did you test it?
test_streaming_tool_call_after_server_tool_blocksbuilds the stream from the realanthropic.typesevents:server_tool_use(0),web_search_tool_result(1), text (2),tool_usewith twoinput_json_deltachunks (3). It goes through the wrappedAnthropicclient.main(4a138e6) it fails: the tool call is captured with'arguments': {}.{'city': 'SF'}.pytest posthog4144 passed, 16 skipped.ruff format --check,ruff check, mypy throughmypy-baseline filterandpython -W error -c "import posthog"are clean.📝 Checklist
If releasing new changes
sampo addto generate a changeset file🤖 Agent context
Autonomy: Fully autonomous
mainbefore the fix.handle_anthropic_*helper signatures, so the non-streaming path and the formatted output stay as they are.Commits here are unsigned; this repo squash-merges, so GitHub signs the merged commit.