fix(lib): don't propagate ValidationError on truncated JSON in chat.completions.parse (#1763) - #3763
Conversation
…ompletions.parse The best-effort parsing boundary in parse_chat_completion called maybe_parse_content and parse_function_tool_arguments without catching pydantic.ValidationError or json.JSONDecodeError. When the model returned truncated JSON (e.g. a stream cut off mid-token, intermittent in production per issue openai#1763), the exception escaped into user code instead of being treated as an unparseable response. Wrap both call sites so that: - parsed tool-call arguments become None on invalid JSON - message.parsed becomes None on invalid structured-output content - a debug-level log line records the underlying error Adds two regression tests that build a ChatCompletion with truncated and garbage JSON content and assert parse_chat_completion does not raise. Fixes openai#1763
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 130c02627c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # stream cut-off). Surface the call with parsed_arguments=None | ||
| # instead of letting the exception escape the best-effort parse | ||
| # boundary. See issue #1763. | ||
| log.debug("Failed to parse tool call arguments: %s", exc) |
There was a problem hiding this comment.
When openai.lib.parsing debug logging is enabled and Pydantic rejects structured content or tool arguments, interpolating exc renders the ValidationError, whose text can include the offending input_value; this can copy customer response data or credentials embedded in model output into application logs. Both new debug statements should log only sanitized metadata such as the exception type or error codes, not the exception string.
AGENTS.md reference: AGENTS.md:L26-L30
Useful? React with 👍 / 👎.
| ``parse_chat_completion`` never lets them escape. See issue #1763.""" | ||
| try: | ||
| return maybe_parse_content(response_format=response_format, message=message) | ||
| except (pydantic.ValidationError, json.JSONDecodeError) as exc: |
There was a problem hiding this comment.
When the response is valid JSON but fails the supplied model's schema or a custom Pydantic validator, this broad catch now returns parsed=None instead of preserving the prior ValidationError. Custom validator constraints are not necessarily represented in the JSON schema sent to the API, so this can silently discard otherwise complete responses and make a contract violation indistinguishable from absent or refused content; inspect the validation error and suppress only JSON decoding failures. The equivalent catch around tool argument parsing has the same problem.
Useful? React with 👍 / 👎.
2 participants
Summary
Fixes #1763.
The best-effort parsing boundary in
parse_chat_completioncalledmaybe_parse_contentandparse_function_tool_argumentswithout catchingpydantic.ValidationErrororjson.JSONDecodeError. When the model returned truncated JSON (e.g. a stream cut off mid-token — intermittent in production per the original report), the exception escaped into user code instead of being treated as an unparseable response.Both call sites are now wrapped so the parse helper degrades gracefully:
pydantic.ValidationErrorescapesmessage.parsed = None+ debug logjson.JSONDecodeErrororValidationErrorescapestool.function.parsed_arguments = None+ debug logmessage.parsed = None+ debug logReproduction (now passes)
Changes
src/openai/lib/_parsing/_completions.py— wrap both parse call sites; add_safe_maybe_parse_contenthelper that mirrorsmaybe_parse_contentbut catches the two exception types.tests/lib/chat/test_completions.py— two new unit tests that build aChatCompletionwith truncated and garbage JSON content and assertparse_chat_completiondoes not raise.Verification
Only files under
src/openai/lib/are touched (generator-safe area).