Skip to content

fix(llm): send OpenAI strict structured output only supported string keywords - #263

Closed
wolfy-j wants to merge 1 commit into
masterfrom
fix/openai-strict-schema-keywords
Closed

wolfy-j wants to merge 1 commit into
masterfrom
fix/openai-strict-schema-keywords

Conversation

@wolfy-j

@wolfy-j wolfy-j commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Problem

wippy.llm.openai:structured_output sends the caller's JSON schema verbatim as text.format.schema with strict: true. A portable schema that bounds strings with minLength/maxLength (honored by other backends, e.g. vLLM guided decoding) falls outside OpenAI's strict-mode subset, so the same schema works on one provider and not on OpenAI.

OpenAI documents pattern and format as the only supported string constraints, and states that calling with strict: true and an unsupported schema returns an error:
https://developers.openai.com/api/docs/guides/structured-outputs#supported-properties
https://developers.openai.com/api/docs/guides/structured-outputs#some-type-specific-keywords-are-not-yet-supported

Fix

The OpenAI adapter sends a copy of the schema with minLength/maxLength removed from every subschema position (properties, items, additionalProperties, anyOf/oneOf/allOf, prefixItems, not, $defs/definitions). Property names and literal values (enum, const, default) are copied untouched, and the caller's schema table is not mutated, so fallback routes and caller-side validation keep the original bounds.

Scope is the OpenAI adapter only; openai_compat and other drivers keep sending the schema as given. No new options.

Tests

src/llm/test/openai/structured_output_test.lua > Wire Schema:

  • unsupported string keywords are absent from the wire schema at nested/nullable/anyOf/$defs positions (red before the fix);
  • supported keywords (pattern, format, numeric bounds, minItems/maxItems, $ref) and a property literally named maxLength are kept;
  • the caller schema is unchanged.

make test in src/llm/test: 2910 passed, 9 skipped. make lint: no issues.

…keywords

OpenAI strict structured outputs support pattern and format as the only
string constraints. The OpenAI adapter forwarded the caller schema
verbatim, so a portable schema carrying minLength/maxLength (enforced by
other providers) left the strict-mode subset. The adapter now sends a
copy of the schema without those keywords; the caller schema is
unchanged and remains the contract callers validate against.
@wolfy-j wolfy-j closed this Nov 12, 2026
@wolfy-j
wolfy-j deleted the fix/openai-strict-schema-keywords branch November 12, 2026 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant