Repository navigation
feat(docscout): record model call outcomes and run summaries - #53
Merged
Merged
Conversation
added 10 commits
November 10, 2026 11:59
The support check asked for an array of verdicts bounded only above, so a constrained decoder could return an empty or partial list and every field without a verdict came back unverified. The reply is now an object with one required verdict per field path; a reply that omits a field fails schema validation as a retryable structured-output error.
A trace step reported only step totals, so a change in accuracy or cost could not be traced to the call, prompt sources or failure that caused it. Each step now lists its model calls with stage, model, prompt and completion tokens (absent when the provider reported none), latency, finish reason, error code, and the chunk ordinals and field paths the call carried. Run results and stored runs carry a summary of field coverage and of unverified fields per cause.
The support check asked for an array of verdicts bounded only above, so a constrained decoder could return an empty or partial list and every field without a verdict came back unverified. The reply is now an object with one required verdict per field path; a reply that omits a field fails schema validation as a retryable structured-output error.
These stages had no configured temperature by default, so new extractors sampled at the provider default and the same passages could read differently from run to run. New configurations now carry temperature 0 for each of them; an explicitly omitted temperature still leaves the provider default. The UI bundle is rebuilt with the new schema defaults.
…y item Scout evidence and notes, section summaries and highlighter passages were asked for as arrays a constrained decoder may legally leave empty or partial. An empty scout evidence list beside a drop list pruned the pool, missing summaries went unnoticed, and an empty highlighter reply dropped every passage of the batch. Each reply is now an object with one required key per field, section or passage. An explicitly empty list or note stays legal and means none; the scout step reports how many fields had no cited evidence. The support verdict schema uses the same keyed shape.
Deep analysis and agent scout replies were read without checking finish_reason, so a reply cut at the output limit was handed to extraction as if complete, and neither Deep nor the highlighter could set an output limit. ai.generate and the agent loop now fail a length-cut reply with model_reply_truncated. Deep accepts max_tokens next to its temperatures, and the highlighter accepts temperature and max_tokens.
A built outline lived only in the run's memory, so every run of an unheaded document paid for the outline calls again. Map now stores the outline it builds on the processed copy under an identity of the map model card, outline prompt version and outline window, and reuses it when the identity matches. Map outline "build" still builds on every run.
A declared expected_format reached the model only as prompt text, so a date in another format or an impossible date passed verification. Verify now parses a date value with the pattern its expected_format declares (YYYY, YY, MMMM, MMM, MM, M, DD, D and separators, any letter case), in lists and objects too, and reports a mismatch as a reason. A format written in prose stays a model hint. The README states that an extractor reads one document per input and that cross-field rules belong to the consumer.
A scout drop removed a passage for every field and blacklisted it, so a passage another field needed or a later targeted search found again could not return, and scout citations were discarded after pruning. The pool now records per passage the fields and queries that found it, its best rank, the fields a review cited it for and the fields it is excluded for. A drop excludes a passage only for the reviewed fields that did not cite it, a field's own later search re-admits it for that field, and each extraction call reads the passages still visible to its fields.
The same call stops at the same output limit again, so replaying it cannot succeed; truncation is not a transport failure.
added 4 commits
November 10, 2026 16:55
added 6 commits
November 10, 2026 17:11
added 25 commits
November 10, 2026 19:45
Merged
wolfy-j
added a commit
that referenced
this pull request
Nov 11, 2026
…ion (#64) * fix(docscout): meter failed replies and report length cuts as truncation A structured reply cut at the output limit failed in the provider as invalid JSON, without usage, so truncation looked like a model error and its tokens were missing from the trace. The llm module now carries a failed reply's usage and finish reason in the error details; model calls meter that usage, record the finish reason, and report a length cut as structured_reply_truncated. * test(docscout): cover failed generation and cumulative agent usage * chore(docscout): resolve current published harness modules * fix(docscout): retain reasons for unverified saved values (#66) * fix(docscout): require one support verdict per field The support check asked for an array of verdicts bounded only above, so a constrained decoder could return an empty or partial list and every field without a verdict came back unverified. The reply is now an object with one required verdict per field path; a reply that omits a field fails schema validation as a retryable structured-output error. * fix(docscout): require one support verdict per field The support check asked for an array of verdicts bounded only above, so a constrained decoder could return an empty or partial list and every field without a verdict came back unverified. The reply is now an object with one required verdict per field path; a reply that omits a field fails schema validation as a retryable structured-output error. * fix(docscout): default map, scout, extract and verify to temperature 0 These stages had no configured temperature by default, so new extractors sampled at the provider default and the same passages could read differently from run to run. New configurations now carry temperature 0 for each of them; an explicitly omitted temperature still leaves the provider default. The UI bundle is rebuilt with the new schema defaults. * fix(docscout): key scout evidence, section summaries and highlights by item Scout evidence and notes, section summaries and highlighter passages were asked for as arrays a constrained decoder may legally leave empty or partial. An empty scout evidence list beside a drop list pruned the pool, missing summaries went unnoticed, and an empty highlighter reply dropped every passage of the batch. Each reply is now an object with one required key per field, section or passage. An explicitly empty list or note stays legal and means none; the scout step reports how many fields had no cited evidence. The support verdict schema uses the same keyed shape. * fix(docscout): fail free-text replies cut by their output limit Deep analysis and agent scout replies were read without checking finish_reason, so a reply cut at the output limit was handed to extraction as if complete, and neither Deep nor the highlighter could set an output limit. ai.generate and the agent loop now fail a length-cut reply with model_reply_truncated. Deep accepts max_tokens next to its temperatures, and the highlighter accepts temperature and max_tokens. * fix(docscout): keep the built outline on the processed copy A built outline lived only in the run's memory, so every run of an unheaded document paid for the outline calls again. Map now stores the outline it builds on the processed copy under an identity of the map model card, outline prompt version and outline window, and reuses it when the identity matches. Map outline "build" still builds on every run. * fix(docscout): verify dates against their declared format A declared expected_format reached the model only as prompt text, so a date in another format or an impossible date passed verification. Verify now parses a date value with the pattern its expected_format declares (YYYY, YY, MMMM, MMM, MM, M, DD, D and separators, any letter case), in lists and objects too, and reports a mismatch as a reason. A format written in prose stays a model hint. The README states that an extractor reads one document per input and that cross-field rules belong to the consumer. * fix(docscout): keep per-field provenance in the evidence pool A scout drop removed a passage for every field and blacklisted it, so a passage another field needed or a later targeted search found again could not return, and scout citations were discarded after pruning. The pool now records per passage the fields and queries that found it, its best rank, the fields a review cited it for and the fields it is excluded for. A drop excludes a passage only for the reviewed fields that did not cite it, a field's own later search re-admits it for that field, and each extraction call reads the passages still visible to its fields. * fix(docscout): mark a length-cut free-text reply as not retryable The same call stops at the same output limit again, so replaying it cannot succeed; truncation is not a transport failure. * docs(docscout): scope truncation guarantee to the built-in scout * test(docscout): establish incomplete support reply failures * test(docscout): exercise keyed stage omission and empty replies * test(docscout): expose pruning transitions and unused highlights * fix(docscout): apply current evidence reviews and highlight usable passages * fix(docscout): leave month-name dates unverified without a language * test(docscout): preserve localized dates through enabled and disabled verification * fix(docscout): key cached outlines by sampling and model-card inputs * docs(docscout): define outline reuse and invalidation behavior * fix(docscout): preserve schema defaults in UI creation payloads * test(docscout): refresh harness dependencies from the registry * test(docscout): refresh harness dependencies from the registry * test(docscout): refresh harness dependencies from the registry * test(docscout): refresh harness dependencies from the registry * fix(docscout): retain reasons for unverified saved values * chore(docscout): refresh published harness dependencies --------- Co-authored-by: Wolfy-J <dev@kickside.ai> * feat(docscout): record model call outcomes and run summaries (#53) * fix(docscout): require one support verdict per field The support check asked for an array of verdicts bounded only above, so a constrained decoder could return an empty or partial list and every field without a verdict came back unverified. The reply is now an object with one required verdict per field path; a reply that omits a field fails schema validation as a retryable structured-output error. * feat(docscout): record every model call and a run summary in the trace A trace step reported only step totals, so a change in accuracy or cost could not be traced to the call, prompt sources or failure that caused it. Each step now lists its model calls with stage, model, prompt and completion tokens (absent when the provider reported none), latency, finish reason, error code, and the chunk ordinals and field paths the call carried. Run results and stored runs carry a summary of field coverage and of unverified fields per cause. * fix(docscout): require one support verdict per field The support check asked for an array of verdicts bounded only above, so a constrained decoder could return an empty or partial list and every field without a verdict came back unverified. The reply is now an object with one required verdict per field path; a reply that omits a field fails schema validation as a retryable structured-output error. * fix(docscout): default map, scout, extract and verify to temperature 0 These stages had no configured temperature by default, so new extractors sampled at the provider default and the same passages could read differently from run to run. New configurations now carry temperature 0 for each of them; an explicitly omitted temperature still leaves the provider default. The UI bundle is rebuilt with the new schema defaults. * fix(docscout): key scout evidence, section summaries and highlights by item Scout evidence and notes, section summaries and highlighter passages were asked for as arrays a constrained decoder may legally leave empty or partial. An empty scout evidence list beside a drop list pruned the pool, missing summaries went unnoticed, and an empty highlighter reply dropped every passage of the batch. Each reply is now an object with one required key per field, section or passage. An explicitly empty list or note stays legal and means none; the scout step reports how many fields had no cited evidence. The support verdict schema uses the same keyed shape. * fix(docscout): fail free-text replies cut by their output limit Deep analysis and agent scout replies were read without checking finish_reason, so a reply cut at the output limit was handed to extraction as if complete, and neither Deep nor the highlighter could set an output limit. ai.generate and the agent loop now fail a length-cut reply with model_reply_truncated. Deep accepts max_tokens next to its temperatures, and the highlighter accepts temperature and max_tokens. * fix(docscout): keep the built outline on the processed copy A built outline lived only in the run's memory, so every run of an unheaded document paid for the outline calls again. Map now stores the outline it builds on the processed copy under an identity of the map model card, outline prompt version and outline window, and reuses it when the identity matches. Map outline "build" still builds on every run. * fix(docscout): verify dates against their declared format A declared expected_format reached the model only as prompt text, so a date in another format or an impossible date passed verification. Verify now parses a date value with the pattern its expected_format declares (YYYY, YY, MMMM, MMM, MM, M, DD, D and separators, any letter case), in lists and objects too, and reports a mismatch as a reason. A format written in prose stays a model hint. The README states that an extractor reads one document per input and that cross-field rules belong to the consumer. * fix(docscout): keep per-field provenance in the evidence pool A scout drop removed a passage for every field and blacklisted it, so a passage another field needed or a later targeted search found again could not return, and scout citations were discarded after pruning. The pool now records per passage the fields and queries that found it, its best rank, the fields a review cited it for and the fields it is excluded for. A drop excludes a passage only for the reviewed fields that did not cite it, a field's own later search re-admits it for that field, and each extraction call reads the passages still visible to its fields. * fix(docscout): mark a length-cut free-text reply as not retryable The same call stops at the same output limit again, so replaying it cannot succeed; truncation is not a transport failure. * docs(docscout): scope truncation guarantee to the built-in scout * test(docscout): establish incomplete support reply failures * test(docscout): exercise keyed stage omission and empty replies * test(docscout): expose pruning transitions and unused highlights * fix(docscout): apply current evidence reviews and highlight usable passages * fix(docscout): leave month-name dates unverified without a language * test(docscout): preserve localized dates through enabled and disabled verification * fix(docscout): key cached outlines by sampling and model-card inputs * docs(docscout): define outline reuse and invalidation behavior * fix(docscout): preserve call outcomes and known request attribution * fix(docscout): preserve schema defaults in UI creation payloads * docs(docscout): identify native model references precisely * test(docscout): refresh harness dependencies from the registry * test(docscout): refresh harness dependencies from the registry * test(docscout): refresh harness dependencies from the registry * test(docscout): refresh harness dependencies from the registry * fix(docscout): retain reasons for unverified saved values * chore(docscout): refresh published harness dependencies * chore(docscout): refresh published harness dependencies --------- Co-authored-by: Wolfy-J <dev@kickside.ai> --------- Co-authored-by: KickSide <dev@kickside.ai>
wolfy-j
added a commit
that referenced
this pull request
Nov 11, 2026
…ion (#69) * fix(docscout): require one support verdict per field The support check asked for an array of verdicts bounded only above, so a constrained decoder could return an empty or partial list and every field without a verdict came back unverified. The reply is now an object with one required verdict per field path; a reply that omits a field fails schema validation as a retryable structured-output error. * feat(docscout): record every model call and a run summary in the trace A trace step reported only step totals, so a change in accuracy or cost could not be traced to the call, prompt sources or failure that caused it. Each step now lists its model calls with stage, model, prompt and completion tokens (absent when the provider reported none), latency, finish reason, error code, and the chunk ordinals and field paths the call carried. Run results and stored runs carry a summary of field coverage and of unverified fields per cause. * fix(docscout): require one support verdict per field The support check asked for an array of verdicts bounded only above, so a constrained decoder could return an empty or partial list and every field without a verdict came back unverified. The reply is now an object with one required verdict per field path; a reply that omits a field fails schema validation as a retryable structured-output error. * fix(docscout): default map, scout, extract and verify to temperature 0 These stages had no configured temperature by default, so new extractors sampled at the provider default and the same passages could read differently from run to run. New configurations now carry temperature 0 for each of them; an explicitly omitted temperature still leaves the provider default. The UI bundle is rebuilt with the new schema defaults. * fix(docscout): key scout evidence, section summaries and highlights by item Scout evidence and notes, section summaries and highlighter passages were asked for as arrays a constrained decoder may legally leave empty or partial. An empty scout evidence list beside a drop list pruned the pool, missing summaries went unnoticed, and an empty highlighter reply dropped every passage of the batch. Each reply is now an object with one required key per field, section or passage. An explicitly empty list or note stays legal and means none; the scout step reports how many fields had no cited evidence. The support verdict schema uses the same keyed shape. * fix(docscout): fail free-text replies cut by their output limit Deep analysis and agent scout replies were read without checking finish_reason, so a reply cut at the output limit was handed to extraction as if complete, and neither Deep nor the highlighter could set an output limit. ai.generate and the agent loop now fail a length-cut reply with model_reply_truncated. Deep accepts max_tokens next to its temperatures, and the highlighter accepts temperature and max_tokens. * fix(docscout): keep the built outline on the processed copy A built outline lived only in the run's memory, so every run of an unheaded document paid for the outline calls again. Map now stores the outline it builds on the processed copy under an identity of the map model card, outline prompt version and outline window, and reuses it when the identity matches. Map outline "build" still builds on every run. * fix(docscout): verify dates against their declared format A declared expected_format reached the model only as prompt text, so a date in another format or an impossible date passed verification. Verify now parses a date value with the pattern its expected_format declares (YYYY, YY, MMMM, MMM, MM, M, DD, D and separators, any letter case), in lists and objects too, and reports a mismatch as a reason. A format written in prose stays a model hint. The README states that an extractor reads one document per input and that cross-field rules belong to the consumer. * fix(docscout): keep per-field provenance in the evidence pool A scout drop removed a passage for every field and blacklisted it, so a passage another field needed or a later targeted search found again could not return, and scout citations were discarded after pruning. The pool now records per passage the fields and queries that found it, its best rank, the fields a review cited it for and the fields it is excluded for. A drop excludes a passage only for the reviewed fields that did not cite it, a field's own later search re-admits it for that field, and each extraction call reads the passages still visible to its fields. * fix(docscout): mark a length-cut free-text reply as not retryable The same call stops at the same output limit again, so replaying it cannot succeed; truncation is not a transport failure. * docs(docscout): scope truncation guarantee to the built-in scout * test(docscout): establish incomplete support reply failures * test(docscout): exercise keyed stage omission and empty replies * test(docscout): expose pruning transitions and unused highlights * fix(docscout): apply current evidence reviews and highlight usable passages * fix(docscout): leave month-name dates unverified without a language * test(docscout): preserve localized dates through enabled and disabled verification * fix(docscout): key cached outlines by sampling and model-card inputs * docs(docscout): define outline reuse and invalidation behavior * fix(docscout): preserve call outcomes and known request attribution * fix(docscout): preserve schema defaults in UI creation payloads * docs(docscout): identify native model references precisely * test(docscout): refresh harness dependencies from the registry * test(docscout): refresh harness dependencies from the registry * test(docscout): refresh harness dependencies from the registry * test(docscout): refresh harness dependencies from the registry * fix(docscout): retain reasons for unverified saved values * chore(docscout): refresh published harness dependencies * chore(docscout): refresh published harness dependencies * fix(docscout): meter failed replies and report length cuts as truncation (#64) * fix(docscout): meter failed replies and report length cuts as truncation A structured reply cut at the output limit failed in the provider as invalid JSON, without usage, so truncation looked like a model error and its tokens were missing from the trace. The llm module now carries a failed reply's usage and finish reason in the error details; model calls meter that usage, record the finish reason, and report a length cut as structured_reply_truncated. * test(docscout): cover failed generation and cumulative agent usage * chore(docscout): resolve current published harness modules * fix(docscout): retain reasons for unverified saved values (#66) * fix(docscout): require one support verdict per field The support check asked for an array of verdicts bounded only above, so a constrained decoder could return an empty or partial list and every field without a verdict came back unverified. The reply is now an object with one required verdict per field path; a reply that omits a field fails schema validation as a retryable structured-output error. * fix(docscout): require one support verdict per field The support check asked for an array of verdicts bounded only above, so a constrained decoder could return an empty or partial list and every field without a verdict came back unverified. The reply is now an object with one required verdict per field path; a reply that omits a field fails schema validation as a retryable structured-output error. * fix(docscout): default map, scout, extract and verify to temperature 0 These stages had no configured temperature by default, so new extractors sampled at the provider default and the same passages could read differently from run to run. New configurations now carry temperature 0 for each of them; an explicitly omitted temperature still leaves the provider default. The UI bundle is rebuilt with the new schema defaults. * fix(docscout): key scout evidence, section summaries and highlights by item Scout evidence and notes, section summaries and highlighter passages were asked for as arrays a constrained decoder may legally leave empty or partial. An empty scout evidence list beside a drop list pruned the pool, missing summaries went unnoticed, and an empty highlighter reply dropped every passage of the batch. Each reply is now an object with one required key per field, section or passage. An explicitly empty list or note stays legal and means none; the scout step reports how many fields had no cited evidence. The support verdict schema uses the same keyed shape. * fix(docscout): fail free-text replies cut by their output limit Deep analysis and agent scout replies were read without checking finish_reason, so a reply cut at the output limit was handed to extraction as if complete, and neither Deep nor the highlighter could set an output limit. ai.generate and the agent loop now fail a length-cut reply with model_reply_truncated. Deep accepts max_tokens next to its temperatures, and the highlighter accepts temperature and max_tokens. * fix(docscout): keep the built outline on the processed copy A built outline lived only in the run's memory, so every run of an unheaded document paid for the outline calls again. Map now stores the outline it builds on the processed copy under an identity of the map model card, outline prompt version and outline window, and reuses it when the identity matches. Map outline "build" still builds on every run. * fix(docscout): verify dates against their declared format A declared expected_format reached the model only as prompt text, so a date in another format or an impossible date passed verification. Verify now parses a date value with the pattern its expected_format declares (YYYY, YY, MMMM, MMM, MM, M, DD, D and separators, any letter case), in lists and objects too, and reports a mismatch as a reason. A format written in prose stays a model hint. The README states that an extractor reads one document per input and that cross-field rules belong to the consumer. * fix(docscout): keep per-field provenance in the evidence pool A scout drop removed a passage for every field and blacklisted it, so a passage another field needed or a later targeted search found again could not return, and scout citations were discarded after pruning. The pool now records per passage the fields and queries that found it, its best rank, the fields a review cited it for and the fields it is excluded for. A drop excludes a passage only for the reviewed fields that did not cite it, a field's own later search re-admits it for that field, and each extraction call reads the passages still visible to its fields. * fix(docscout): mark a length-cut free-text reply as not retryable The same call stops at the same output limit again, so replaying it cannot succeed; truncation is not a transport failure. * docs(docscout): scope truncation guarantee to the built-in scout * test(docscout): establish incomplete support reply failures * test(docscout): exercise keyed stage omission and empty replies * test(docscout): expose pruning transitions and unused highlights * fix(docscout): apply current evidence reviews and highlight usable passages * fix(docscout): leave month-name dates unverified without a language * test(docscout): preserve localized dates through enabled and disabled verification * fix(docscout): key cached outlines by sampling and model-card inputs * docs(docscout): define outline reuse and invalidation behavior * fix(docscout): preserve schema defaults in UI creation payloads * test(docscout): refresh harness dependencies from the registry * test(docscout): refresh harness dependencies from the registry * test(docscout): refresh harness dependencies from the registry * test(docscout): refresh harness dependencies from the registry * fix(docscout): retain reasons for unverified saved values * chore(docscout): refresh published harness dependencies --------- Co-authored-by: Wolfy-J <dev@kickside.ai> * feat(docscout): record model call outcomes and run summaries (#53) * fix(docscout): require one support verdict per field The support check asked for an array of verdicts bounded only above, so a constrained decoder could return an empty or partial list and every field without a verdict came back unverified. The reply is now an object with one required verdict per field path; a reply that omits a field fails schema validation as a retryable structured-output error. * feat(docscout): record every model call and a run summary in the trace A trace step reported only step totals, so a change in accuracy or cost could not be traced to the call, prompt sources or failure that caused it. Each step now lists its model calls with stage, model, prompt and completion tokens (absent when the provider reported none), latency, finish reason, error code, and the chunk ordinals and field paths the call carried. Run results and stored runs carry a summary of field coverage and of unverified fields per cause. * fix(docscout): require one support verdict per field The support check asked for an array of verdicts bounded only above, so a constrained decoder could return an empty or partial list and every field without a verdict came back unverified. The reply is now an object with one required verdict per field path; a reply that omits a field fails schema validation as a retryable structured-output error. * fix(docscout): default map, scout, extract and verify to temperature 0 These stages had no configured temperature by default, so new extractors sampled at the provider default and the same passages could read differently from run to run. New configurations now carry temperature 0 for each of them; an explicitly omitted temperature still leaves the provider default. The UI bundle is rebuilt with the new schema defaults. * fix(docscout): key scout evidence, section summaries and highlights by item Scout evidence and notes, section summaries and highlighter passages were asked for as arrays a constrained decoder may legally leave empty or partial. An empty scout evidence list beside a drop list pruned the pool, missing summaries went unnoticed, and an empty highlighter reply dropped every passage of the batch. Each reply is now an object with one required key per field, section or passage. An explicitly empty list or note stays legal and means none; the scout step reports how many fields had no cited evidence. The support verdict schema uses the same keyed shape. * fix(docscout): fail free-text replies cut by their output limit Deep analysis and agent scout replies were read without checking finish_reason, so a reply cut at the output limit was handed to extraction as if complete, and neither Deep nor the highlighter could set an output limit. ai.generate and the agent loop now fail a length-cut reply with model_reply_truncated. Deep accepts max_tokens next to its temperatures, and the highlighter accepts temperature and max_tokens. * fix(docscout): keep the built outline on the processed copy A built outline lived only in the run's memory, so every run of an unheaded document paid for the outline calls again. Map now stores the outline it builds on the processed copy under an identity of the map model card, outline prompt version and outline window, and reuses it when the identity matches. Map outline "build" still builds on every run. * fix(docscout): verify dates against their declared format A declared expected_format reached the model only as prompt text, so a date in another format or an impossible date passed verification. Verify now parses a date value with the pattern its expected_format declares (YYYY, YY, MMMM, MMM, MM, M, DD, D and separators, any letter case), in lists and objects too, and reports a mismatch as a reason. A format written in prose stays a model hint. The README states that an extractor reads one document per input and that cross-field rules belong to the consumer. * fix(docscout): keep per-field provenance in the evidence pool A scout drop removed a passage for every field and blacklisted it, so a passage another field needed or a later targeted search found again could not return, and scout citations were discarded after pruning. The pool now records per passage the fields and queries that found it, its best rank, the fields a review cited it for and the fields it is excluded for. A drop excludes a passage only for the reviewed fields that did not cite it, a field's own later search re-admits it for that field, and each extraction call reads the passages still visible to its fields. * fix(docscout): mark a length-cut free-text reply as not retryable The same call stops at the same output limit again, so replaying it cannot succeed; truncation is not a transport failure. * docs(docscout): scope truncation guarantee to the built-in scout * test(docscout): establish incomplete support reply failures * test(docscout): exercise keyed stage omission and empty replies * test(docscout): expose pruning transitions and unused highlights * fix(docscout): apply current evidence reviews and highlight usable passages * fix(docscout): leave month-name dates unverified without a language * test(docscout): preserve localized dates through enabled and disabled verification * fix(docscout): key cached outlines by sampling and model-card inputs * docs(docscout): define outline reuse and invalidation behavior * fix(docscout): preserve call outcomes and known request attribution * fix(docscout): preserve schema defaults in UI creation payloads * docs(docscout): identify native model references precisely * test(docscout): refresh harness dependencies from the registry * test(docscout): refresh harness dependencies from the registry * test(docscout): refresh harness dependencies from the registry * test(docscout): refresh harness dependencies from the registry * fix(docscout): retain reasons for unverified saved values * chore(docscout): refresh published harness dependencies * chore(docscout): refresh published harness dependencies --------- Co-authored-by: Wolfy-J <dev@kickside.ai> --------- Co-authored-by: KickSide <dev@kickside.ai> --------- Co-authored-by: Wolfy-J <dev@kickside.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Trace records retain model call outcomes and known request attribution without changing prompts or extraction policy. Selected-agent records consume published native Dataflow outcomes, including request failures, truncation, latency and field attribution. Outline, summary, highlight and support records carry known source ordinals and fields; malformed embeddings retain their validation failure and reported usage.
The branch merges current main and the saved-verification-reason fix without rewriting existing commits. The static bundle is rebuilt from the merged source; all 118 UI tests and typecheck pass. The harness resolves dependencies fresh, including Dataflow 0.7.37 and LLM 0.5.18, with no source replacement for either dependency.
Validation with official v0.3.59a:
For C reported requests, N document chunks and F <= 50 configured fields, additional metadata has at most C records, CN source references and CF field references plus scalar outcomes. Records contain no prompts, replies or vectors. Native agent-internal tool and memory requests remain outside primary-step telemetry; source ordinals stay absent when the native contract does not report them. Summaries count produced field outcomes rather than all requested fields.
Compatibility: call records and summaries are optional additions to existing JSON payloads. Older saved runs remain readable. There is no new setting, table, migration, retry or retention mechanism. At run admission rate r and effective retention T, retained run rows approach rT, 10rT and 100r*T at base, 10x and 100x load; metadata increases bytes per row within the documented input-dependent bound.