Repository navigation
Trace CANNBot round knowledge in managed optimization - #523
Conversation
zengchuang-hw
left a comment
There was a problem hiding this comment.
PASS
首轮检视,c573f40a43ceef23ef08507fca662c95ffb65390 → 226f8d0f5fb114443ed76311879e78f24b4b6180,覆盖全部 14 个变更文件,按 #522 的软件交付范围核对。
实际追踪了 CLI → managed session → Harness → CANNBot 渠道与 worker。未开启时不调用外部渠道;搜索和读取分开,读取要求当前轮 locator 并校验 revision、digest 和 page。正文不写入留存 trace;候选只能引用已送达的本地条目或外部读取回执,搜索命中和未引用的读取均不算采用。Dashboard 读取同一记录,分别展示来源、状态、引用及 evaluator 分数,没有更改候选选择。
批准前必做:无。已静态阅读覆盖查找、读取、引用、不可用及留存最小化的用例;未运行代码、测试、浏览器或生产 grant/Ascend 集成,不据此宣称硬件验收或性能收益。
david6666666
left a comment
There was a problem hiding this comment.
Reviewed the managed retrieval and Dashboard attribution paths on this head. Two findings below.
| reference = {"path": path.relative_to(root).as_posix(), "sha256": fingerprint, | ||
| "kind": "delivered_context"} | ||
| if number is not None: | ||
| reference["round"] = number |
There was a problem hiding this comment.
This adds round to the returned reference even with CANNBot off. capture_delivery.managed_envelopes() still reconstructs the exact three-field {path, sha256, kind} response, so the trace verifier can no longer match these genuine local searches. Both Ready jobs fail four citation regressions. Please keep round on the retained state reference only, or version the envelope and verify both old and new traces.
| "status": facts, "evaluation": judged}, | ||
| usage=record["observations"]["usage"], | ||
| measured_score=_number(assessment.get("score")), | ||
| knowledge_uses=[citations[ref] for ref in record.get("knowledge_refs") or [] |
There was a problem hiding this comment.
This resolves mutable state.json citations against the current session-wide context map. _load() verifies the observation file's hash but doesn't compare its recorded citations and context snapshot with state. A completed round can therefore display a later receipt as cited while its verified observation stays unchanged. Please resolve completed-round citations from the verified observation and submitted context snapshot, or check that state matches both, with a regression for later receipts.
|
@zengchuang-hw @david6666666 麻烦重新评审当前提交 当前提交的 Dashboard 浏览器检查已通过,Python 3.10 和 3.11 的 full-check 测试仍有失败。 CI 结果 如果复核后认可当前改动,请在 GitHub 提交 Approve,正文第一行单独写 |
zengchuang-hw
left a comment
There was a problem hiding this comment.
PASS
增量检视 226f8d0f5fb114443ed76311879e78f24b4b6180 → 8bbc58ccbb05fee2b0ad2f44209a7badea9732b2,承接上轮检视。本轮只有一个提交,覆盖控制器及回归测试。读取已完成候选时,现在先核对原始 observation 摘要,再核对原始引用和当时的 context 快照;已引用的 receipt 不能被后续轮次悄悄换掉。留存时仍保存 round,本地调用恢复原先的精简响应字段。已核对新检查与真实 settle 写入一致,也确认两边都没有 knowledge_refs 的旧记录不会因此失败。
批准前必做:无。合入协调:独立于本轮检视,#526 若先合,旧 apps.cannbot 兼容层不导出 SCORE_SCALE。后合入的一方应把 round_knowledge.py 及相关导入迁到 apps.integrations.cannbot,核对安装包 worker 和 managed attribution smoke;未做组合验证。 本轮未执行代码、测试或实机验证。
david6666666
left a comment
There was a problem hiding this comment.
One remaining Dashboard metadata validation concern.
| # The digest checks file integrity, not whether the record's | ||
| # fields are safe to disclose. Expose only known view metadata. | ||
| search = { | ||
| **{key: record.get(key) for key in ( |
There was a problem hiding this comment.
The metadata field allowlist does not validate the values. public_value() preserves nested objects and lists, so a digest-valid retained record can still carry card content or diagnostics inside an allowed field. The existing regression only covers extra top-level fields. Please validate the expected scalar types and formats for search and read metadata before projection, and cover malformed values.
|
Posted by an agent on behalf of @Dong1017 (incremental review only; points already raised by others are not repeated). DO NOT PASS — reviewed head Blocking
Non-blocking
Scope checked: the |
|
#516 has been merged into |
|
PR review verdict — DO NOT PASS 已检视 base @david6666666 在旧 head 上的两点已经在 下面两项在当前 head 上仍然成立。
合入前还请处理两项协调(不是本轮代码结论的替代):#516 合入后需要再 rebase 到当前 |
|
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5). 维持 DO NOT PASS。@amy-why-3459 指出的两项我在完整 diff( 1. 示例页未再生 — diff 里确实没有那个文件。 2.
怎样才解除阻塞(三项,前面两项就在本 PR 内):
关于当前 head 上的 @zengchuang-hw Approve:我看到了,它没有覆盖 red CI 和上述投影校验缺口,PASS 要求无未解除阻塞,所以我不会翻转结论。 非阻塞、随作者下次 push 一起处理的协调项:#516 已进 推送新的 head 后我会增量重读 delta。本轮只读了完整 diff 与 #523 的 body / 线程 / 正式评审,没有执行测试、浏览器用例、生产 grant 或昇腾实机验证。 — Dong-bot(授权代理) |
Record opt-in CANNBot searches and pinned read receipts beside local captures. Show metadata and explicit citations in Dashboard, preserve native local capture verification, and reserve final evaluation time when the external worker times out. Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com> Agent-assisted-by: Codex / implement_523 (implementer)
8bbc58c to
24f5e08
Compare
|
PR review verdict — DO NOT PASS 增量检视上一轮 head 已修好的部分:
仍未解决:
本轮只读了相对上一 head 的增量与相对当前 main 的完整 20 个文件 diff,没有执行测试套件、生产 grant 或昇腾实机验证。 |
Use the contained regular-file reader and verify the retained SHA-256 before consuming managed knowledge. Cover escaping paths, symbolic links, the capture byte limit, and digest mismatch through the Dashboard. Signed-off-by: jiangkuaixue123 <jiangxiaozhou111@163.com> Agent-assisted-by: Codex / implement_523 (implementer)
|
@amy-why-3459 @david6666666 @Dong1017 @zengchuang-hw 麻烦复审 这版修复了 Amy 最新指出的本地 前一提交的 Python 3.10 / 3.11 全量检查、Chromium 和安装包检查均已通过。新提交的 GitHub CI 已触发。前一提交的 CI 如果复核通过,请在 GitHub 提交 Approve,正文第一行单独写 |
Dong1017
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base f0e92e8d4992746ac62e89f16347ecab6141de67 and head cc552e47bc5a707f715fcb36da91b7d42525e42f (full diff, diffComplete: true), plus the incremental step from the last formally reviewed head 24f5e08a, #523 body/thread, the seeded review record, and #522's goal and acceptance bullets.
All three blockers from my 8bbc58cc DO NOT PASS and the single remaining blocker in @amy-why-3459's 24f5e08a review are resolved on this head.
Fixed in cc552e47
- Local
delivered_contextnow uses the bounded reader. Amy's finding was thatmanaged_knowledge()still didjson.loads((root / path).read_text("utf-8"))for local captures. In this diff_read()delegates tobounded_record_bytes(root, relative, MAX_CONTEXT_CAPTURE_BYTES)(retrieval_projection.py), so the local path now gets the same treatment as external records: non-string/empty names,\and:rejected, absolute and..parts rejected, every path component checked for symlinks (so ancestor symlinks are covered, not only the final file),inside()containment, andfile_absent/too_large/unsafe_pathstates instead of silent parsing.managed_knowledge()now calls that one_read()for every kind and requiresdigest == reference["sha256"]before projecting, so a matching digest is necessary but no longer sufficient. - Harness first access is bounded too.
harness/optimization.pyadds_read_context(), which reads throughbounded_record_bytesand compares SHA-256 against the retained reference;_load()uses it for every retained context (replacing the unbounded(root / context["path"]).read_bytes()), andcapture_context()uses it for the returnedcontext. The diagnosis-binding check is correctly skipped forexternal_search/external_read, which are stored withoutdiagnosis_id. - Regression coverage matches the claim.
test_managed_knowledge_rejects_unverified_captureis parametrized overabsolute,outside(../),symlink,ancestor_symlink,oversized,hash_mismatch; it asserts the projection returns([], {}), assertsread_regularis never reached for unsafe paths, and assertstoo_largeis enforced at theMAX_CONTEXT_CAPTURE_BYTESboundary. That is the "context path outside the root" case Amy asked for, not a restatement of it.
Earlier findings, re-checked in this diff rather than taken from the review record
- Published example page:
docs/examples/dashboard-profile-review.htmlis in this diff and carries the samemanagedRetrieval()/knowledgeUses()additions and the samenumbers/contextIds/ results-branch edits assrc/inferrsi/apps/views/round_decision_view.html. The deterministic cause of the 3.10/3.11test_published_profile_example_matches_fixturefailure (run 44545533153) is therefore fixed by construction, not by re-running. - Unvalidated metadata pass-through: external projection now goes through
search_metadata()/read_metadata()with allowlisted types —countanintbounded below by retained items and at 200,revision40-hex,card_sha25664-hex,pagea positiveint,scorea finite number orNone,statefromCHANNEL_STATES,platformfromPLATFORMS, locator relative and regex-checked.external_readno longer bare-subscriptsrecord[...](myKeyErrorpoint), and a receipt is only linked when itssearch_refresolves to a search in the same round with matching revision, locator and digest. The new assertions on the malformed-record block check the exact surviving key set and thatcount/revision/score_scale/elapsed_msdegrade toNoneinstead of carrying nested objects. - Attribution:
_load()compares a completed round'sknowledge_refsandcontextswith the SHA-256-verified observation and re-checks each cited path against the submitted context snapshot; the tests exercise replacing an earlier candidate's refs, appending a later receipt, and rewriting a retained receipt with its state hash updated (Round knowledge attribution…/Cited knowledge context differs…). - Degradation check David asked for on #522 is present:
test_cannbot_timeout_keeps_local_ranking_and_final_budgetasserts the channel timeout budget is_knowledge_time − _WORKER_CLEANUP_SECONDS, the deadline is unchanged, local refs and ordering are identical to the local-only run, a candidate citing only the local item settles, andfinalstill completes. - Retention: the test asserts card text, worker diagnostics and the private locator path appear in neither the Dashboard payload nor any retained JSON under the session root, and the query is redacted before search and retention, matching #522's allow-list bullet.
Non-blocking follow-ups
managed_knowledge()still indexesrecord["contexts"],record["items"],record["request"]and item["ref"]/["resources"]directly fordelivered_context. A malformed local capture with a matching digest raises an uncaughtKeyErrorin the Dashboard projection instead of a typed rejection. It fails visibly rather than widening the view, so I do not treat it as blocking, but a guard would match the external path's style.- #516 integration is done in this head (
round_knowledge.pyand its imports live underapps.integrations.cannbot, anddocs/apps-module-migrations.jsonlists the new consumers). @zengchuang-hw's ordering note still stands if #526 lands after this. - #522 remains open for requester acceptance, and production CANNBot grant plus Ascend workload verification are still open. Both are outside this code verdict.
Not verified here: I did not execute the test suite, the Chromium/Playwright cases, a production grant or Ascend hardware; exact-head CI for cc552e47 was only reported as triggered, so I read the fixture/example consistency from the diff. The preceding head 24f5e08a had both full-check jobs, Chromium and the installed-package check green (run 44707929174). The 214-test and Ruff result on this head is the author's report, not my run.
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).
Dong1017
left a comment
There was a problem hiding this comment.
PR review verdict — DO NOT PASS
复审 #523:base f0e92e8d4992746ac62e89f16347ecab6141de67 → head cc552e47bc5a707f715fcb36da91b7d42525e42f,重点核对 24f5e08a..cc552e47 和 #522 的软件交付范围。当前 head 没有正式 APPROVED,也没有未解除的 CHANGES_REQUESTED;旧 head 的 Approve 已被撤销,不计入本轮。
阻塞:当前 head 的必需 CI 尚未有通过记录。 Python 3.10/3.11 full-check、Chromium 和安装包结果 属于 24f5e08a。cc552e47 修改的是 Dashboard 与 Harness 共用的记录读取路径。PR 正文报告当前 head 的 214 项定向测试和 Ruff 已通过(1 项旧 wheel 兼容测试跳过),但我将当前提交的必需 CI 通过作为本次 Approve 的前置条件,因此本轮只提交 COMMENT。请补上 cc552e47 的必需检查结果;如有新 push,我会按新 head 复查。
此前代码阻塞已解除。 Diff 中 managed_knowledge() 对本地 delivered_context 也走 _read(),并比较留存 SHA-256;Harness 的 _load()、首次捕获返回及条目引用查找共用 _read_context()。两侧共用的 bounded_record_bytes() 拒绝绝对或越界路径以及文件或祖先目录符号链接,并限制读取为 4 MiB 加一个用于判断超限的字节。新增参数化回归覆盖这些路径、超限和摘要不匹配。发布示例已再生,外部搜索和回执的标量校验及关联检查仍在,纯本地响应仍是三字段 reference,已完成引用仍与校验过的 observation 和提交时的上下文快照绑定。
本轮仅做静态核对,没有重跑测试、浏览器、生产 grant 或昇腾工作负载。生产验证和 #522 的请求方验收不属于本轮阻塞。
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model gpt-6.2-sol).
amy-why-3459
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
增量检视上一轮 head 24f5e08a0d600721e59f0734ffa1c66fb2cb0bca 到当前 head cc552e47bc5a707f715fcb36da91b7d42525e42f(base f0e92e8d4992746ac62e89f16347ecab6141de67)。
上一轮留下的阻塞已解除:managed_knowledge() 对本地 delivered_context 也走 _read(),并对照 reference["sha256"]。_read() 现在经 bounded_record_bytes() 拒绝绝对或越界路径、文件及祖先目录符号链接,并限制读取为 MAX_CONTEXT_CAPTURE_BYTES。Harness 的 _load()、首次捕获返回、外部搜索读取和条目引用查找共用 _read_context()。test_managed_knowledge_rejects_unverified_capture 覆盖了 absolute / outside / symlink / ancestor_symlink / oversized / hash_mismatch。
此前已修好的部分在本 head 仍然成立:发布示例已再生,外部检索/读取有标量类型与格式校验,纯本地响应仍是三字段 reference,已完成引用仍与校验过的 observation 和提交时的上下文快照绑定。
这是路径安全与记录读取的修复,不改变模型输出或性能路径,不需要准确率/性能对照实验。架构上复用已有的 bounded reader,测试是针对上一轮缺口的定向回归,没有重复堆叠。
提交前已重新拉取本 head 的 review 与评论:@Dong1017 已在 5990392804 对本 head Approve;5990416879 的 DO NOT PASS 仅因当时必需 CI 尚未出结果。当前 Ready CI 44709987715 上 full-check 3.10/3.11、installed-package、dashboard-browser、pr-references 以及 knowledge-model-policy 均为 success,该项已解除。无未解除的 CHANGES_REQUESTED。本轮只读了 24f5e08a..cc552e47 的 4 个文件增量,没有重跑测试套件、生产 grant 或昇腾实机验证。生产验证和 #522 的请求方验收不属于本轮阻塞。
|
Posted by an agent on behalf of @Dong1017 (incremental review only; items already raised by others are not repeated). Incremental review of Note the timing: #523 was merged at 09:40Z, before this review started. The head has not moved since Verified here
One non-blocking observation, not raised previously The 4 MiB bound is now applied on read while the write side has no matching bound. It is not reachable on this head, so I am not treating it as a defect: Scope: read the API diff |
…atrixLab#519/InferMatrixLab#521/InferMatrixLab#523 Rounds now require a retained bottleneck diagnosis (InferMatrixLab#519), so the explanation tests record one before candidate rounds. The combined declaration test cites both a delivered context and a InferMatrixLab#523 path#item-ref and checks that the round, its retained observation and the dashboard keep the citations, the diagnosis binding and the bound mechanism together through final. InferMatrixLab#523 labels a round's measured score with metric_key, which this change names after the measured metric. Under --decision paired the score is the median paired speedup, so managed rows carry score_key (the frozen selection metric) and the round view prefers it; legacy rows fall back to metric_key. Service-reuse profiling is optional since InferMatrixLab#521; the docs and the start-time comment say its receipts are not inputs to the paired/explanation switches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>
…atrixLab#519/InferMatrixLab#521/InferMatrixLab#523 Rounds now require a retained bottleneck diagnosis (InferMatrixLab#519), so the explanation tests record one before candidate rounds. The combined declaration test cites both a delivered context and a InferMatrixLab#523 path#item-ref and checks that the round, its retained observation and the dashboard keep the citations, the diagnosis binding and the bound mechanism together through final. InferMatrixLab#523 labels a round's measured score with metric_key, which this change names after the measured metric. Under --decision paired the score is the median paired speedup, so managed rows carry score_key (the frozen selection metric) and the round view prefers it; legacy rows fall back to metric_key. The published profile-review example is regenerated from its fixture. Service-reuse profiling is optional since InferMatrixLab#521; the docs and the start-time comment say its receipts are not inputs to the paired/explanation switches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>
) * harness: report paired AB/BA statistics alongside the point verdict Pair the k-th baseline and candidate samples that steps() already alternates, and report the median paired speedup with a seeded percentile bootstrap interval and its relation to the frozen gate. The field is report-only: verdict, eligibility and selection are unchanged, and any value that cannot be computed is reported as unavailable so records stay strict JSON. Fewer than five pairs report no interval because simulated coverage at three pairs is about 75%. tools/measurement_simulation.py reproduces the synthetic ablation. Agent-assisted-by: Claude Code / claude-opus-5-5 (author) Agent-assisted-by: Codex CLI / gpt-6.1-sol [effort=high] (review) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com> * harness: explain gains with profile evidence and opt-in decision switches Measurement scripts may add per-step region breakdowns and work counters. A round may declare its mechanism and expected region/counter directions. Regions declared unchanged act as an in-step clock, so ratios to them and their share of the step total cancel device-wide clock or thermal drift; counters such as launches or DRAM bytes show whether a small gain is reproducible. Directions come only from paired intervals, with no repository-wide tolerance. Two switches are frozen at campaign start and default to the existing behaviour: --require-explanation adopts only candidates whose mechanism is confirmed by the data, and --decision paired requires the paired interval to clear the gate, failing within-noise rounds without blocking the search. Agent-assisted-by: Claude Code / claude-opus-5-5 (author) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com> * harness: bind declared mechanisms before execution and harden profile evidence Bind the round's mechanism into control state before the candidate runs and assess against that copy, so a candidate cannot rewrite its own declaration. Require five pairs before any profile direction, treat zero-baseline ratios as undefined, report unavailable shares instead of raising, keep raw evidence for every region, and never normalize declared references against each other. Agent-assisted-by: Claude Code / claude-opus-5-5 (author) Agent-assisted-by: Codex CLI / gpt-6.1-sol [effort=xhigh] (review) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com> * harness: judge the in-memory declaration and seal its digest The candidate runs with the caller's privileges and can write state.json or the attempt copy. Pass the declaration held since submission to the assessor, record its digest, and treat a copy that no longer matches as an integrity failure. Agent-assisted-by: Claude Code / claude-opus-5-5 (author) Agent-assisted-by: Codex CLI / gpt-6.1-sol [effort=xhigh] (review) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com> * harness: do not trust a declaration recovered after interruption An interrupted attempt is settled by a new process that can only read files the candidate could write. Settle it without the declaration, mark the record, and leave require_explanation to keep the round unadopted. Agent-assisted-by: Claude Code / claude-opus-5-5 (author) Agent-assisted-by: Codex CLI / gpt-6.1-sol [effort=xhigh] (review) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com> * harness: document switch precedence and the paired-only decision With both switches on, --decision sets the performance verdict first and --require-explanation only downgrades a PASS. The paired decision reads the interval alone, so a candidate whose point estimate misses the gate but whose paired interval clears it passes, with agrees_with_point_gate recorded false. Regression tests pin both behaviors. Agent-assisted-by: Claude Code / claude-opus-5-5 (author) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com> * harness: exact paired intervals, Bonferroni checks and INCONCLUSIVE within noise The percentile bootstrap lower bound for a median of n pairs is an order statistic whose one-sided error exceeds 2.5% at small n (3.1% at 5 pairs, 6.25% at 7). Use the exact distribution-free order-statistic interval (the inverted sign test) instead: the rank keeps each tail at or below 2.5%, so the minimum is six pairs and A/A at the gate passes at most 2.5% of the time. Declared expectations are judged as a family: region and counter intervals use the Bonferroni confidence 1 - 5%/k, and too few pairs for it is unverifiable. A declared-unchanged item is reported as not contradicted, never confirmed. A paired interval that straddles the gate shows only that passing was not demonstrated, which is not evidence that the gate was missed, so the round is INCONCLUSIVE (within_noise) and still does not block the search. Records say report_only: false when the interval decides the verdict. Reported by linyueqian on #457. Agent-assisted-by: Claude Code / claude-opus-5-5 (author) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com> * harness: exact binomial arithmetic and report-only profile failures A float 2 ** pairs overflows near a thousand pairs; compare binomial tails as exact rationals instead. A failure while building profile evidence is reported as unavailable rather than raised, so it cannot change the default verdict. Found by Codex review of the exact-interval change. Agent-assisted-by: Claude Code / claude-opus-5-5 (author) Agent-assisted-by: Codex CLI / gpt-6.1-sol [effort=xhigh] (review) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com> * harness: keep raw/normalized reversals and missing evidence inconclusive When a region's raw and reference-normalized intervals point in opposite definite directions, an absolute increase can read as a relative decrease (two references can each cross 1 while their sum rises in every pair). No declaration predicts a relative effect, so record raw_normalized_conflict and leave that check unverifiable instead of letting the normalized sign win. Under --require-explanation, only contradicted evidence fails. An unsupported prediction is missing evidence, like a paired interval within noise, so it is INCONCLUSIVE; the candidate is still not adopted and the search continues. Reported by david6666666 on #454 and #457. Agent-assisted-by: Claude Code / claude-opus-5-5 (author) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com> * harness: refuse explanation gates that could never adopt a candidate With --require-explanation, fewer than six repetitions give no interval for any expectation, so every round would be unverifiable and nothing could be adopted; campaign start now rejects that. A round whose declaration needs more pairs than the frozen repetitions (Bonferroni over its expectations) is rejected before execution, so it spends no budget. Without the switch a declaration remains a report and is not restricted. Reported by linyueqian on #457. Agent-assisted-by: Claude Code / claude-opus-5-5 (author) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com> * harness: rank paired-decision candidates by paired speedup Under --decision paired the verdict read the paired interval, but adoption still compared independent candidate medians, so a paired PASS could be outranked and never reach final. The frozen selection is now the median paired speedup against the baseline (1.0), and that same speedup is the round's score, speedup and improvement. Point values stay in point_estimate. Each round records why it was or was not selected. The default point mode and benchmark sessions keep their selection. Agent-assisted-by: Claude Code / claude-opus-5-5 (author) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com> * harness: refuse paired and explanation switches in service-reuse campaigns Service-reuse campaigns (#480) record neither AB/BA pairs nor per-step profiles, so --decision paired, --require-explanation and --mechanism-file would be silently ignored there. start and round now reject them. Agent-assisted-by: Claude Code / claude-opus-5-5 (author) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com> * harness: keep scoreless settlement to the project paired decision A completed, ineligible round could settle without a score for any assessor, which changed the packaged-task contract outside #454. Only a project plan with the frozen paired decision may now omit the score (its paired statistics can be unavailable). Packaged tasks and point-mode projects still block on a missing score, as before. Agent-assisted-by: Claude Code / claude-opus-5-5 (author) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com> * docs: say which service-reuse inputs the paired switches lack Service reuse keeps its own profiling stage (#480); what it does not produce are the AB/BA pairs and per-measurement breakdown/counters that the paired decision and explanation checks read. Agent-assisted-by: Claude Code / claude-opus-5-5 (author) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com> * apps: deliver no baseline speedup for a final without a score The delivery projection (#485) reported 1.0 for a retained baseline whenever the final completed. Under the paired decision, a final whose statistics are unavailable completes without a score, and the controller summary already reports no speedup for it. The delivery bundle now agrees. Also covers #501's --strategy/--knowledge-ref alongside a bound mechanism on one round and its final. Agent-assisted-by: Claude Code / claude-opus-5-5 (author) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com> * harness: keep mechanism and knowledge citations together after #519/#521/#523 Rounds now require a retained bottleneck diagnosis (#519), so the explanation tests record one before candidate rounds. The combined declaration test cites both a delivered context and a #523 path#item-ref and checks that the round, its retained observation and the dashboard keep the citations, the diagnosis binding and the bound mechanism together through final. #523 labels a round's measured score with metric_key, which this change names after the measured metric. Under --decision paired the score is the median paired speedup, so managed rows carry score_key (the frozen selection metric) and the round view prefers it; legacy rows fall back to metric_key. The published profile-review example is regenerated from its fixture. Service-reuse profiling is optional since #521; the docs and the start-time comment say its receipts are not inputs to the paired/explanation switches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com> --------- Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Governing Issue and delivery slice
Refs #522
Add opt-in CANNBot retrieval and explicit knowledge citations to managed benchmark sessions and AB/BA campaigns. The implementation uses the existing per-project CANNBot grant and the current
apps.integrations.cannbotadapter.start --cannbot-platform <platform>freezes the Ascend platform. The flag defaults to off.knowledgereturns separate local and CANNBot results. The timeout regression checks stable local result IDs and rankings, local citation, and reserved final evaluation.knowledge-readverifies the selected search hit's revision and digest before returning the requested page. Retained receipts carry identity metadata. Dashboard validates scalar values and search/read linkage before display.round --knowledge-refcites an explicitly delivered local item or pinned read receipt. The settled observation binds citations to the submitted context snapshot. Round decisions shows these citations beside evaluator outcomes. Native Codex trace verification recognizes the exact local envelope in both local-only and joint tool returns.Review follow-up
reference. Return format reviewCurrent-head review follow-up
cc552e47routes local managed context projection through the same bounded read as external receipts. The Harness uses the protected reader from its first access through citation lookup. Both verify SHA-256 against the retained reference. Regression coverage includes absolute and escaping paths, file and ancestor symlinks, oversized records, and digest mismatch. Amy's reviewcc552e47againstf0e92e8d. Exact-head targeted checks passed 214 tests and Ruff, with one compatibility test skipped because it requires a pinned pre-migration wheel. The preceding head24f5e08apassed both full-check jobs, Chromium, and installed-package CI. Preceding head CIVerification on the preceding head
f0e92e8dand head24f5e08aafter rechecking service-reuse compatibility. GitHub maintainer review remains pending.Remaining work and Issue acceptance
Production CANNBot grant and Ascend workload verification remain open. Issue #522 remains open for requester acceptance.