Repository navigation
Conversation
Only select native usage when its receipt supplies observed counters. Preserve the existing campaign subtotal and source when the attached receipt has no counters. Agent-assisted-by: Codex (author) Signed-off-by: Inference1 <3617237115@qq.com>
Dong1017
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base 9466bc8 and head 0d731b8 (full diff, first review). prior-reviews.md shows no formal reviews and no outstanding CHANGES_REQUESTED, so this Approve is the first independent review on this head.
Verified from the diff:
src/inferrsi/apps/delivery/benchmark_usage.py::attach_native_usage:usagenow starts asNone. Theselected_native_usageflag is gone.usageis only set when the policy is supported, the snapshot is workspace-bound, andcoverage.campaign_plan_digestequalscontrol["plan_digest"]. In that case it comes fromterminal_token_usage(...).- The native override of
result["usage"]["session_tokens"]and the source rewrite now run only whenusage is not None and usage["observed_tokens"] is not None. A terminal receipt with no counters therefore no longer replaces the campaign-levelbenchmark_metrics.token_usageandusage.session_tokenswith an all-null native record. This matches the PR title. - The missing-receipt case with observed tokens still takes the native path. The existing
test_single_benchmark_missing_terminal_receipt_preserves_observed_subtotalhasobserved_tokens == 1200, so it is not affected. - The new test
test_single_benchmark_terminal_receipt_without_counters_preserves_campaign_usagedeletesobservation["usage"]and projects with the agent job. It asserts these results:token_usageequals the original campaign projection.observed_tokens == 1200andtotal_tokens is None.session_tokensis unchanged.- A
native_agent_usagesource is still recorded, becausesnapshot.include(native, ...)still runs unconditionally.
This covers the regression path.
Not verified here:
- I did not run the test suite, and no CI status was seeded.
- The diff context ends a few lines after the changed block, so I could not see the rest of
attach_native_usage. I assume nothing after theifdereferencesusagewhen it isNone. The existing unsupported-policy test (test_native_job_preserves_unsupported_policy_reason) exercises that path and should catch aNonedereference. Please confirm it is green in CI.
Non-blocking: the repo ruleset still needs two eligible approvals. This is one vote.
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).
zengchuang-hw
left a comment
There was a problem hiding this comment.
PASS
首次独立检视:9466bc8d910c9fef55d4a9e23ee1a82950280c30…0d731b8efba7fb99859340e7b7484954256cbeb4,范围是 #536 的独立 Benchmark 导出。附加的原生回执仍被索引;缺少可用计数时,不再覆盖已有的 1,200 token 观察小计、来源或未知的完整总量。合法的 0 仍可选用,原有身份、范围和完整性门槛不变。
已阅读两个变更文件并在隔离环境对精确函数做了缺失、0、正数三种投影检查(使用快照替身),同时核对原生来源仍被索引。未跑完整导出套件或设备基准。本批准仅适用于这项导出修复,不代表 #536 的整体验收。
amy-why-3459
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base 9466bc8 and head 0d731b8 (full diff).
Verified:
attach_native_usagenow appliesterminal_token_usageonly when the receipt hasobserved_tokens is not None(zero still qualifies). A completed native receipt with nousagefield no longer overwrites the campaign's retained subtotal or its source;snapshot.includestill indexes the job for audit.- The new
test_single_benchmark_terminal_receipt_without_counters_preserves_campaign_usagecovers that regression (1,200 observed tokens,total_tokens is None, native source still present). - I read the rest of
attach_native_usage: whenusageisNoneit is not dereferenced after theif. Identity, interval and completeness checks interminal_token_usageare unchanged.
This Approve is for this delivery fix only, not an acceptance of #536 as a whole. Required checks were still running at review time; please keep them green before merge.
Governing Issue and delivery slice
Refs #536
Standalone delivery follow-up to the versioned metrics implementation in #539. The slice covers a native terminal receipt with an identity-bound coverage declaration and absent token counters.
Problem
When
export --agent-jobattaches a completed native observation with nousagefield, the Benchmark report replaces the campaign's retained caller-exported token subtotal with unknown values. The artifact index still contains both sources, but the reported subtotal and its provenance disappear.What
Keep the campaign's token projection when the selected native receipt has no usable counters. Continue indexing the attached native job and its receipts for audit.
Why needed
A user can attach a job to audit a benchmark even when its host did not export token counters. The report should continue to show the observed subtotal and mark complete session cost as unknown.
Design and implementation
Reuse
terminal_token_usagefor identity, interval and coverage checks. Apply its projection only when it contains an observed token total. An observed zero qualifies. The complete total still requires the existing full-scope checks, and the result and its rendered views continue to share one projection.Knowledge authorship declaration
Not applicable. The patch touches standalone Apps delivery and its tests.
Checks and dependencies
ruff check src tests toolsandgit diff --checkpassed.Remaining work and Issue closure
This software regression is ready for independent review. The governing Issue's broader acceptance stays with its owner.