Skip to content

Gate RSI search on measured bottleneck diagnosis - #519

Merged
Dong1017 merged 3 commits into
InferMatrixLab:mainfrom
Bounty-hunter:codex/rsi-diagnosis-first
Nov 20, 2026
Merged

Dong1017 merged 3 commits into
InferMatrixLab:mainfrom
Bounty-hunter:codex/rsi-diagnosis-first

Conversation

@Bounty-hunter

@Bounty-hunter Bounty-hunter commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Refs #337

问题

#501 的 on/off 对照已经冻结任务、评测协议和预算。通用 RSI loop 目前允许直接查知识或提交候选,瓶颈分析和 profiling 的顺序由 Agent 自行决定。这个 PR 把先诊断再搜索的规则接入共享 Harness。

改动

  • campaign diagnose 和 benchmarks session diagnose 复用同一控制器,记录当前瓶颈与 workload 对应的 UTF-8 测量摘要。摘要可来自当前 profiling 或已有的同条件测量。
  • Harness 默认给所有新会话启用诊断。受管 knowledge 和候选 round 都要求先登记诊断,并记录关联的诊断 ID。service-reuse 建立新的参考条件后,需要登记当前 epoch 的诊断。
  • benchmarks compare 的 on/off 两组、独立 benchmark 和用户直接发起的普通项目优化都走同一规则。on 组在有具体实现疑问时按需检索,off 组完成诊断后直接提交候选。回收时核对测量摘要、原始轮次引用和诊断时间。
  • start 沿用现有参数。既有会话仍按各自的冻结计划执行。

流程

  1. 对冻结基线进行 profiling,或读取同条件的既有测量。
  2. 用 diagnose 保存瓶颈及对应观测。
  3. 有实现疑问时按知识开关检索,再提交候选进行评测。
  4. 根据反馈更新诊断或继续使用仍相关的记录,最后独立复测。

Harness 核查诊断记录和顺序。测量口径及瓶颈推断由执行者根据当前任务确认。

验证

  • Ruff 和 diff 检查通过。
  • 本地核心回归 129 项通过、1 项跳过、43 项子检查通过。覆盖 benchmark on/off、普通项目、service-reuse 诊断入口和证据回收。

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NO PASS

首次独立检视:9c7b7b05042f54ca71d13b2897a7505d818d3429 → f473060feb5ab51ea4a1d3ae3cab75840e9449b4,范围 #337 的诊断顺序和对照回收。

批准前必做:报告需确认引用的诊断仍含测量凭据。 optimization._load 会核对保留的评估/交付证据,但不检查新增的诊断记录;benchmark_compare 回收时只检查 diagnosis_id 是否存在。若归档中诊断 ID 保留、测量摘要缺失或为空,报告仍不会在此处增加缺证据项,可能把无法审计的对照算入有效结果。这与本 PR 要求保留测量诊断不一致。建议在回收时校验引用诊断的必填字段与摘要;增加摘要缺失/空白时报告降为不可纳入的回归。不是要求 Harness 判断测量本身是否科学有效。

启动、诊断登记、knowledge/round 门槛及 service-reuse epoch 分流已静态核对。未运行代码或测试,未查 CI。

@Dong1017 Dong1017 added area:harness Harness module: profile, A/B loop, quality gates, evidence reports kind:task Bounded delivery task with a delegated slice labels Nov 19, 2026

@david6666666 david6666666 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comparison collector should verify the recorded activity-to-diagnosis binding, not only the existence of a diagnosis.

Comment thread src/inferrsi/apps/benchmark_compare.py Outdated
if plan.get("diagnosis_required"):
diagnoses = {record["id"] for record in control.get("diagnoses", [])}
if any(record.get("diagnosis_id") not in diagnoses
for record in (*control["rounds"], *control["contexts"])):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also accepts a valid diagnosis from the wrong point in the run. If a collected round's state.json reference is changed to a second, later diagnosis, the original observation.json hash still verifies and the membership check passes, even though the retained observation names the first diagnosis. Please check the round's reference against its retained observation (and the activity order), with a regression that swaps between two complete diagnoses.

@Bounty-hunter
Bounty-hunter marked this pull request as ready for review November 19, 2026 10:19
zengchuang-hw
zengchuang-hw previously approved these changes Nov 19, 2026

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PASS

独立增量检视:f473060feb5ab51ea4a1d3ae3cab75840e9449b4 → 680827c3fae2c5c780542179e6a53e7e0b7f54a5,上轮意见。

上轮缺项已修复:回收读取会检查诊断 ID 唯一、瓶颈与测量摘要非空、诊断时间有效,与对应轮次的绑定及 epoch 一致。不再只检验 ID 存在,轮次的诊断引用与已固定的 observation 对照;service-reuse 恢复也复用该校验。回归覆盖摘要缺失/空白、重新绑定较晚诊断和记录时间失真时不计入有效对照。既有会话仍按冻结计划读取。未发现新增阻塞。

静态核对了代码和回归;这是对诊断记录完整性及顺序约束的检视,不验证摘要中的测量结论。未运行代码、测试或查 CI。

@david6666666 david6666666 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Round binding is now checked against retained evidence. Knowledge capture still has a separate gap.

Comment thread src/inferrsi/harness/optimization.py Outdated
if not isinstance(reference, str) or reference not in diagnoses:
raise ValueError("Campaign activity lacks a retained bottleneck diagnosis")
diagnosis = diagnoses[reference]
if round_record and (

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both knowledge-capture read paths call this with round_record=False, and the diagnosis ID is stored only in the mutable context reference. As a result, rebinding a retained knowledge capture to a later or different-epoch diagnosis still passes as long as that ID exists. Please bind the diagnosis ID and the capture epoch/order to retained capture evidence and verify it on collection, with a regression for a knowledge capture rebound across epochs.

@david6666666

Copy link
Copy Markdown
Collaborator

#516 has been merged into main. Please rebase this branch onto the latest main and adjust the changes as needed. Please verify that the diagnosis gate still allows a managed knowledge lookup once the required diagnosis has been recorded.

Require measured bottleneck evidence before managed lookup and candidate search. Verify each capture against the diagnosis and reference conditions in effect at lookup, and keep independent final validation separate.

Agent-assisted-by: Codex (implementation, review fixes and regression tests)
Signed-off-by: dengyunyang <584797741@qq.com>
Agent-assisted-by: Codex (regression test)
Signed-off-by: dengyunyang <584797741@qq.com>
Report invalid baseline and parent evidence before asking for a new diagnosis. Candidate execution still requires a diagnosis for the current conditions.

Agent-assisted-by: Codex (implementation)
Signed-off-by: dengyunyang <584797741@qq.com>

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS
Reviewed base 1b826b5f9666a8b734000e1eb2990e582cdef876 and head 20530c5b3996964c10d5252f0c11d88ba93a28d2.

Independent third-party technical verify (cursor[bot]; collaborator permission none — not a write+ eligible Approver under docs/agent-collaboration.md; does not complete dual tip / merge-ready).

Leftover tip re-check after head moved past dismissed zengchuang-hw Approve on 680827c…. Verified against Refs #337 diagnosis-gate scope:

  • Managed diagnose → knowledge/round gating: diagnosis_required default, _diagnosis_reference / _retain_diagnosis, and round/context retain diagnosis_id.
  • Prior david knowledge-rebind concern: captures now store diagnosis_binding; _check_context_diagnosis requires binding ID to match the retained context reference and reuses activity epoch/order checks — addresses tip COMMENT on 680827c….
  • Compare recovery rejects incomplete/rebound diagnoses (parametrized tests including knowledge-rebound).
  • Local targeted pytest: 173 passed, 1 skipped (test_benchmark_compare, test_project_service_reuse, test_project_optimization, test_optimization_session) after PATH python→python3 shim (CI uses python).
  • Ready tip green: full-check 3.10/3.11, declarations, installed-package, dashboard-browser, pr-references (https://github.com/InferMatrixLab/InferRSI/actions/runs/44669404917).

No tip Dong1017 Approve yet on this head; zeng tip Approve dismissed by push. This is cursor first formal tip vote as independent verify.

Merge still needs write+ independent Approves / steward integration — not claimed here.

@zengchuang-hw zengchuang-hw left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PASS

增量 680827c3 → 20530c5b,上轮检视。本轮在知识查找前绑定当次诊断,恢复旧上下文时核对诊断 ID、epoch 与开始时间;service 先校验参考与条件,仍不会越过诊断提前建立候选。benchmark 路径保留原有哈希校验,service 不新增不适用的 fingerprint 要求。已核对 refresh、未诊断拒绝、时间/epoch 错配与恢复用例定义。

无批准前必修项。仅静态核对精确差异与 #337 范围;未执行测试、采集或查询 CI。

@Bounty-hunter

Copy link
Copy Markdown
Contributor Author

@Dong1017 @david6666666 @zengchuang-hw 麻烦帮忙确认 #519 的审批条件,满足后请合入。

当前提交为 20530c5b,已处理知识检索凭据绑定和 #516 适配的意见,验证了记录诊断后可以正常检索。Python 3.10 和 3.11 的完整 CI 均已通过,GitHub 当前显示无合并冲突。

CI 结果

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

PR review verdict — PASS
Reviewed base 1b826b5f9666a8b734000e1eb2990e582cdef876 and head 20530c5b3996964c10d5252f0c11d88ba93a28d2 (Dong-bot, authorized delegate). I checked the full diff against the earlier findings. I did not run tests. I did not run CI myself and rely on the CI link in the PTAL comment and cursor[bot]'s review of the same run.

Earlier blockers:

  • Diagnosis completeness (zeng, first round): _diagnosis_index rejects duplicate IDs, blank or missing bottleneck / measurement_summary, a missing epoch, and an invalid recorded_at. Both _load and _service_load call it.
  • Round bound to the wrong diagnosis (david, inline): _load compares record["diagnosis_id"] with the retained observation.json (whose hash is already verified). It then runs _check_activity_diagnosis, which checks that the diagnosis exists, precedes started_at, and has the same reference_epoch.
  • Knowledge capture rebound (david, latest tip comment): capture_context and the service knowledge path now write diagnosis_binding (ID, epoch, started_at) into the retained capture. _check_context_diagnosis requires the binding ID to equal the mutable state.json reference. It then reuses the epoch and order check. In the non-service path the capture bytes are covered by the existing context sha256. The parametrized knowledge-rebound test and the service test that rewrites contexts[0]["diagnosis_id"] cover this.
  • #516 adaptation and "diagnose, then knowledge works": The gate (_diagnosis_reference) raises before lookup if no diagnosis exists for the current epoch. After diagnose, lookup succeeds. This is asserted in test_benchmark_compare.finish(knowledge=True) and test_diagnosis_gate_applies_to_service_knowledge. That test also covers refresh requiring a new-epoch diagnosis.
  • Existing sessions: Gating is keyed on plan["diagnosis_required"]. Old sessions have no such field and are not gated. _control compares the field only for service campaigns. Compare adds a gap if a campaign did not freeze the requirement.

No blockers found.

Non-blocking:

  • The service-reuse capture is checked only through service.read and does not appear to be pinned by a state-side hash. The binding check is therefore weaker there than in the benchmark and project path. It is not a regression, and it is out of scope for this PR.
  • The checks cover record completeness and order only. They do not judge whether the measurement conclusion is correct, which the PR body states.

Merge conditions:

  • Current head has formal Approves from @zengchuang-hw (write+) and @cursor[bot]. cursor[bot]'s own review says it is not an eligible write+ approver.
  • This Approve adds a second tip approval from the Dong1017 account. Per the thread, David's last formal state is a COMMENT on 680827c3, and his knowledge-rebind concern is addressed in the code above. If the process wants his explicit tip sign-off, he should confirm. Otherwise the steward can integrate.

Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).

Copy link
Copy Markdown
Contributor

回应本轮 @:已复核当前仍是 20530c5b,我的正式代码检视批准 已覆盖该提交,无需重复批准。本次没有核验合入门槛,也没有执行合入,请由维护者确认并操作。

@Dong1017
Dong1017 merged commit b593008 into InferMatrixLab:main Nov 20, 2026
10 checks passed
BeatSeat added a commit to BeatSeat/InferRSI that referenced this pull request Nov 20, 2026
…atrixLab#519/InferMatrixLab#521/InferMatrixLab#523

Rounds now require a retained bottleneck diagnosis (InferMatrixLab#519), so the explanation
tests record one before candidate rounds. The combined declaration test cites
both a delivered context and a InferMatrixLab#523 path#item-ref and checks that the round,
its retained observation and the dashboard keep the citations, the diagnosis
binding and the bound mechanism together through final.

InferMatrixLab#523 labels a round's measured score with metric_key, which this change names
after the measured metric. Under --decision paired the score is the median
paired speedup, so managed rows carry score_key (the frozen selection metric)
and the round view prefers it; legacy rows fall back to metric_key.

Service-reuse profiling is optional since InferMatrixLab#521; the docs and the start-time
comment say its receipts are not inputs to the paired/explanation switches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>
BeatSeat added a commit to BeatSeat/InferRSI that referenced this pull request Nov 20, 2026
…atrixLab#519/InferMatrixLab#521/InferMatrixLab#523

Rounds now require a retained bottleneck diagnosis (InferMatrixLab#519), so the explanation
tests record one before candidate rounds. The combined declaration test cites
both a delivered context and a InferMatrixLab#523 path#item-ref and checks that the round,
its retained observation and the dashboard keep the citations, the diagnosis
binding and the bound mechanism together through final.

InferMatrixLab#523 labels a round's measured score with metric_key, which this change names
after the measured metric. Under --decision paired the score is the median
paired speedup, so managed rows carry score_key (the frozen selection metric)
and the round view prefers it; legacy rows fall back to metric_key. The
published profile-review example is regenerated from its fixture.

Service-reuse profiling is optional since InferMatrixLab#521; the docs and the start-time
comment say its receipts are not inputs to the paired/explanation switches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>
Dong1017 pushed a commit that referenced this pull request Nov 21, 2026
)

* harness: report paired AB/BA statistics alongside the point verdict

Pair the k-th baseline and candidate samples that steps() already
alternates, and report the median paired speedup with a seeded percentile
bootstrap interval and its relation to the frozen gate. The field is
report-only: verdict, eligibility and selection are unchanged, and any value
that cannot be computed is reported as unavailable so records stay strict
JSON. Fewer than five pairs report no interval because simulated coverage at
three pairs is about 75%. tools/measurement_simulation.py reproduces the
synthetic ablation.

Agent-assisted-by: Claude Code / claude-opus-5-5 (author)
Agent-assisted-by: Codex CLI / gpt-6.1-sol [effort=high] (review)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>

* harness: explain gains with profile evidence and opt-in decision switches

Measurement scripts may add per-step region breakdowns and work counters.
A round may declare its mechanism and expected region/counter directions.
Regions declared unchanged act as an in-step clock, so ratios to them and
their share of the step total cancel device-wide clock or thermal drift;
counters such as launches or DRAM bytes show whether a small gain is
reproducible. Directions come only from paired intervals, with no
repository-wide tolerance.

Two switches are frozen at campaign start and default to the existing
behaviour: --require-explanation adopts only candidates whose mechanism is
confirmed by the data, and --decision paired requires the paired interval
to clear the gate, failing within-noise rounds without blocking the search.

Agent-assisted-by: Claude Code / claude-opus-5-5 (author)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>

* harness: bind declared mechanisms before execution and harden profile evidence

Bind the round's mechanism into control state before the candidate runs and
assess against that copy, so a candidate cannot rewrite its own declaration.
Require five pairs before any profile direction, treat zero-baseline ratios as
undefined, report unavailable shares instead of raising, keep raw evidence for
every region, and never normalize declared references against each other.

Agent-assisted-by: Claude Code / claude-opus-5-5 (author)
Agent-assisted-by: Codex CLI / gpt-6.1-sol [effort=xhigh] (review)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>

* harness: judge the in-memory declaration and seal its digest

The candidate runs with the caller's privileges and can write state.json or
the attempt copy. Pass the declaration held since submission to the assessor,
record its digest, and treat a copy that no longer matches as an integrity
failure.

Agent-assisted-by: Claude Code / claude-opus-5-5 (author)
Agent-assisted-by: Codex CLI / gpt-6.1-sol [effort=xhigh] (review)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>

* harness: do not trust a declaration recovered after interruption

An interrupted attempt is settled by a new process that can only read files
the candidate could write. Settle it without the declaration, mark the record,
and leave require_explanation to keep the round unadopted.

Agent-assisted-by: Claude Code / claude-opus-5-5 (author)
Agent-assisted-by: Codex CLI / gpt-6.1-sol [effort=xhigh] (review)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>

* harness: document switch precedence and the paired-only decision

With both switches on, --decision sets the performance verdict first and
--require-explanation only downgrades a PASS. The paired decision reads the
interval alone, so a candidate whose point estimate misses the gate but whose
paired interval clears it passes, with agrees_with_point_gate recorded false.
Regression tests pin both behaviors.

Agent-assisted-by: Claude Code / claude-opus-5-5 (author)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>

* harness: exact paired intervals, Bonferroni checks and INCONCLUSIVE within noise

The percentile bootstrap lower bound for a median of n pairs is an order
statistic whose one-sided error exceeds 2.5% at small n (3.1% at 5 pairs,
6.25% at 7). Use the exact distribution-free order-statistic interval (the
inverted sign test) instead: the rank keeps each tail at or below 2.5%, so the
minimum is six pairs and A/A at the gate passes at most 2.5% of the time.

Declared expectations are judged as a family: region and counter intervals use
the Bonferroni confidence 1 - 5%/k, and too few pairs for it is unverifiable.
A declared-unchanged item is reported as not contradicted, never confirmed.

A paired interval that straddles the gate shows only that passing was not
demonstrated, which is not evidence that the gate was missed, so the round is
INCONCLUSIVE (within_noise) and still does not block the search. Records say
report_only: false when the interval decides the verdict.

Reported by linyueqian on #457.

Agent-assisted-by: Claude Code / claude-opus-5-5 (author)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>

* harness: exact binomial arithmetic and report-only profile failures

A float 2 ** pairs overflows near a thousand pairs; compare binomial tails as
exact rationals instead. A failure while building profile evidence is reported
as unavailable rather than raised, so it cannot change the default verdict.

Found by Codex review of the exact-interval change.

Agent-assisted-by: Claude Code / claude-opus-5-5 (author)
Agent-assisted-by: Codex CLI / gpt-6.1-sol [effort=xhigh] (review)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>

* harness: keep raw/normalized reversals and missing evidence inconclusive

When a region's raw and reference-normalized intervals point in opposite
definite directions, an absolute increase can read as a relative decrease
(two references can each cross 1 while their sum rises in every pair). No
declaration predicts a relative effect, so record raw_normalized_conflict and
leave that check unverifiable instead of letting the normalized sign win.

Under --require-explanation, only contradicted evidence fails. An unsupported
prediction is missing evidence, like a paired interval within noise, so it is
INCONCLUSIVE; the candidate is still not adopted and the search continues.

Reported by david6666666 on #454 and #457.

Agent-assisted-by: Claude Code / claude-opus-5-5 (author)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>

* harness: refuse explanation gates that could never adopt a candidate

With --require-explanation, fewer than six repetitions give no interval for
any expectation, so every round would be unverifiable and nothing could be
adopted; campaign start now rejects that. A round whose declaration needs more
pairs than the frozen repetitions (Bonferroni over its expectations) is
rejected before execution, so it spends no budget. Without the switch a
declaration remains a report and is not restricted.

Reported by linyueqian on #457.

Agent-assisted-by: Claude Code / claude-opus-5-5 (author)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>

* harness: rank paired-decision candidates by paired speedup

Under --decision paired the verdict read the paired interval, but adoption
still compared independent candidate medians, so a paired PASS could be
outranked and never reach final. The frozen selection is now the median
paired speedup against the baseline (1.0), and that same speedup is the
round's score, speedup and improvement. Point values stay in point_estimate.
Each round records why it was or was not selected. The default point mode
and benchmark sessions keep their selection.

Agent-assisted-by: Claude Code / claude-opus-5-5 (author)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>

* harness: refuse paired and explanation switches in service-reuse campaigns

Service-reuse campaigns (#480) record neither AB/BA pairs nor per-step
profiles, so --decision paired, --require-explanation and --mechanism-file
would be silently ignored there. start and round now reject them.

Agent-assisted-by: Claude Code / claude-opus-5-5 (author)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>

* harness: keep scoreless settlement to the project paired decision

A completed, ineligible round could settle without a score for any
assessor, which changed the packaged-task contract outside #454. Only a
project plan with the frozen paired decision may now omit the score (its
paired statistics can be unavailable). Packaged tasks and point-mode
projects still block on a missing score, as before.

Agent-assisted-by: Claude Code / claude-opus-5-5 (author)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>

* docs: say which service-reuse inputs the paired switches lack

Service reuse keeps its own profiling stage (#480); what it does not
produce are the AB/BA pairs and per-measurement breakdown/counters that
the paired decision and explanation checks read.

Agent-assisted-by: Claude Code / claude-opus-5-5 (author)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>

* apps: deliver no baseline speedup for a final without a score

The delivery projection (#485) reported 1.0 for a retained baseline
whenever the final completed. Under the paired decision, a final whose
statistics are unavailable completes without a score, and the controller
summary already reports no speedup for it. The delivery bundle now
agrees. Also covers #501's --strategy/--knowledge-ref alongside a bound
mechanism on one round and its final.

Agent-assisted-by: Claude Code / claude-opus-5-5 (author)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>

* harness: keep mechanism and knowledge citations together after #519/#521/#523

Rounds now require a retained bottleneck diagnosis (#519), so the explanation
tests record one before candidate rounds. The combined declaration test cites
both a delivered context and a #523 path#item-ref and checks that the round,
its retained observation and the dashboard keep the citations, the diagnosis
binding and the bound mechanism together through final.

#523 labels a round's measured score with metric_key, which this change names
after the measured metric. Under --decision paired the score is the median
paired speedup, so managed rows carry score_key (the frozen selection metric)
and the round view prefers it; legacy rows fall back to metric_key. The
published profile-review example is regenerated from its fixture.

Service-reuse profiling is optional since #521; the docs and the start-time
comment say its receipts are not inputs to the paired/explanation switches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>

---------

Signed-off-by: BeatSeat <137616995+BeatSeat@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:harness Harness module: profile, A/B loop, quality gates, evidence reports kind:task Bounded delivery task with a delegated slice

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants