Skip to content

fix(docscout): batch extraction fields by their expected reply size - #63

Closed
wolfy-j wants to merge 1 commit into
fix/docscout-pool-provenancefrom
fix/docscout-output-batching
Closed

wolfy-j wants to merge 1 commit into
fix/docscout-pool-provenancefrom
fix/docscout-output-batching

Conversation

@wolfy-j

@wolfy-j wolfy-j commented Nov 10, 2026

Copy link
Copy Markdown
Contributor

Stacked on #59 (pool provenance, on #55): extraction_calls already builds passages per batch there. Review the last commit.

Cause

Extraction batched fields by count (max_fields_per_call, default 20) without regard to how much each field's reply needs. Every field carries a value plus up to five quotes inside one max_output_tokens (default 4096), so object and verbatim text fields overflow it, and one truncated reply fails every field of the call.

Fix

  • shapes.output_estimate(f): the expected reply of one field = envelope (30) + EVIDENCE_LIMIT quotes (5 x 20) + value leaves (text 40, other scalars 8, objects the sum of their properties, lists five items). The constants come from the per-call completion sizes the new call records report (feat(docscout): record model call outcomes and run summaries #53) on a stress run: a text field's reply is about 65 tokens at the median and 165 at the 90th percentile, so the estimate sits near the 90th percentile.
  • A call takes the fields of its group and region while their estimates fit max_output_tokens; max_fields_per_call stays the upper bound.
  • A field whose estimate alone exceeds max_output_tokens fails before any call with field_output_exceeds_budget (INVALID, not retryable), and the message names stages.extract.max_output_tokens.

A heuristic: it reduces collateral failures; a long verbatim value (measured up to about 1700 tokens) can still be cut, and then fails alone or with few neighbours. No configuration change.

Tests

  • shapes_test: estimates grow with text, objects and lists and include the quotes.
  • engine_test: a group splits by estimated output within the limit; a field whose estimate exceeds the limit fails with the code and the setting named while the other field extracts. Unit fixtures now carry a valid max_output_tokens (the schema minimum is 256).

Red before, green after. make lint clean. Suite on SQLite and PostgreSQL: 188 passed, 0 failed. test/check-package-bindings.py fails on main as well (unrelated).

Extraction batched fields by count only, so up to 20 fields with values and
quotes shared one max_output_tokens limit; a cut reply failed every field
of the call. Fields now fill a call while their estimated replies (value
leaves, lists at five items, five quotes each, set from measured reply
sizes) fit the output limit, with max_fields_per_call as an upper bound. A
field whose estimated reply alone exceeds the limit fails with
field_output_exceeds_budget, naming the setting to raise.
@wolfy-j

wolfy-j commented Nov 12, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #77 and #81, merged and released.

@wolfy-j wolfy-j closed this Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant