Repository navigation
Open
Labels
area:harnessHarness module: profile, A/B loop, quality gates, evidence reportsHarness module: profile, A/B loop, quality gates, evidence reportskind:taskBounded delivery task with a delegated sliceBounded delivery task with a delegated slice
Description
Current state: Accepted by the goal owner (Dong1017, 2026-11-21) with the name harness/trials/. #498 is closed (#535 merged as fb34354) and listed this rename as out of scope; #490 has also closed, so the tree has settled.
Decision needed: none.
Next action: one PR does the move with the runtime-identity record described below. A separate docs PR already corrects the contributor guidance that calls this package the place for new models.
Goal and acceptance
- Problem / desired outcome:
src/inferrsi/harness/scenarios/holds one module,external.py, which runs one external command and retains its trial record (execute_trial,collect_trial_checks). It has no scenario logic: the module docstring says "External command execution and retained trial facts, independent of model and device". The package docstring still says "Scenario result adapters", anddocs/contributing.md:38tells contributors to useharness/scenariosfor new models, which no longer matches the code. The goal is a package name that says what the code does, with no change in behaviour. - Completion criteria:
- The package has its new name and the old
inferrsi.harness.scenariospath no longer exists, with no forwarder, the same policy as task: retire theinfra/andmodel/packages into their owning modules #498. The gate rejects the old name, with or without a file behind it. - Every importer uses the new path:
harness/runner.py,harness/project_service.py,harness/optimization_session.py(three imports and the_runtime_identity()entry) andapps/legacy/run.py. - The package docstring,
docs/contributing.md,docs/architecture.md,docs/module-layout.md,docs/contracts/function-owners.mdanddocs/moe-forward-benchmark.mdname the new path and its actual role. - Runtime identity:
harness/scenarios/external.pyis in_runtime_identity(), so the rename makes the new runtime refuse a session planned before it, as in task: retire theinfra/andmodel/packages into their owning modules #498 slices 2 and 5. The PR carries the same evidence those slices used: a fixture session started and interrupted by the last runtime before the rename, with a completed execution receipt; the checkout test andtests/check_installed.pyshowstatusandrecoverexit 2 and leave every file unchanged; and a paired-wheel record on that same session shows the old runtime recovers it without running the candidate again. - Installed-wheel, host-compatibility and
full-checkjobs pass. CLI arguments, outputs, exit codes and record formats are unchanged. - CHANGELOG lists the old and new path and the resume rule for older sessions.
- The package has its new name and the old
Scope and constraints
- In scope: the
harness/scenarios/package, its importers, its runtime-identity entry, and the docs that name it. - Out of scope:
- Apps root modules and subpackages (
archive.py,bridge.py,campaign.py,views/and the others). task: 按职责整理 Apps 子模块目录(handoff / storage / workflows / integrations) #490 placed them and task: Apps 应用层详细设计与迁移计划(CLI / Python / Plugin) #371 owns further Apps layout, including the static compatibility modules and their retirement. - The technical leaves
contracts/andresources/, whose names rfc: K/H/B 与 Apps 公开接口详细设计(最小交接与模块边界) #387 and task: R01 — retire legacy root paths and close core dependencies #412 settled. - Any behaviour change in
execute_trialorcollect_trial_checks.
- Apps root modules and subpackages (
- Relevant dependencies, shared interfaces or limits:
- This is a deliberate compatibility break for sessions planned before the rename. Resume such a session with the runtime that planned it.
- Open PRs that import
inferrsi.harness.scenariosmust be rebased onto the new path. Check the open PR list when the rename PR opens. - The Harness root holds 28 modules of several kinds. Regrouping it would change 13 runtime-identity files and 6 skill-eval controller files, and is deferred; it is not part of this issue.
Ownership and authorization
- Proposed by / actual requester: Dong1017 (follow-up recorded in task: retire the
infra/andmodel/packages into their owning modules #498, "Out of scope") - Status: Accepted
- Accountable parent (Issue-level owner, if assigned): Dong1017
- Slice implementer(s) / acting contributor (if assigned): Dong1017, through a Claude Code session (
claude-opus-5-5, effortxhigh) - Goal-acceptance owner: Dong1017
- Accepted decision or scoped delegation: this body, by the goal owner, 2026-11-21
Inputs and deliverables
- Relevant sources, revisions or linked issues: task: retire the
infra/andmodel/packages into their owning modules #498 (closed; "Out of scope" names this rename), refactor(apps): give Apps its artifact helpers and the provider modules, remove infra and model #535 (fb34354, the slice-5 runtime-identity record this follows), task: 按职责整理 Apps 子模块目录(handoff / storage / workflows / integrations) #490 (Apps placement),tests/fixtures/optimization-session/README.md(how the pre-change session fixture and paired-wheel record were made). - Expected artifacts: one PR with the move, the importer and doc updates, the pre-rename session fixture and its tests, and the paired-wheel record.
Completion and closure
- Who accepts completion and may close: Dong1017
- Remaining uncertainty / next decision: none.
harness/trials/matches the two public functions (execute_trial,collect_trial_checks);harness/execution/was rejected becauseharness/execution_service.pyalready exists and the name is broader than the code.
Metadata
Metadata
Assignees
Labels
area:harnessHarness module: profile, A/B loop, quality gates, evidence reportsHarness module: profile, A/B loop, quality gates, evidence reportskind:taskBounded delivery task with a delegated sliceBounded delivery task with a delegated slice
Activity
Dong1017 commentedon Nov 21, 2026
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).
Assessment: well-scoped and ready to implement. No decision is open. The issue asks for a behaviour-neutral rename of
src/inferrsi/harness/scenarios/toharness/trials/, with no forwarder. It also asks for a runtime-identity bump so that older sessions are refused by the new runtime.Checked against the default branch
harness/scenarios/holds only__init__.pyandexternal.py, so the "one module" premise holds.harness/trials/does not exist yet, so there is no name collision.harness/execution_service.pydoes exist. That supports rejectingharness/execution/.docs/contributing.md:38still says "Useharness/scenarios,harness/servingandbenchmark/validatorsfor new models". That is the stale guidance the issue describes.Gaps and risks to cover in the PR
runner.py,project_service.py,optimization_session.pyandapps/legacy/run.py. Before the move, grep the whole repo forharness.scenarios,harness/scenariosandscenarios.external. Includetests/,tests/fixtures/,scripts/,pyproject.tomland package-data and wheel manifests, CI workflows, skill-eval controller files,src/inferrsi/harness/README.md, and string-valued references such as monkeypatch targets.harness/README.mdis not in the doc list but may name the package.scenarios/is recreated, including as a directory with only__pycache__or an empty__init__.py. Stale__pycache__or an installed-wheel leftover can mask a missing-import failure, so the installed-wheel job should assert the old path is absent._runtime_identity()entry changes toharness/trials/external.pyand that the old entry is removed. Otherwise the identity hash may not change, or may keep hashing a missing file. The pre-rename session fixture must be generated with the last pre-rename runtime. Followtests/fixtures/optimization-session/README.md, and record the pre-rename commit SHA in the PR.statusandrecoverexit 2 with every file byte-unchanged. It also needs the paired-wheel result that the old runtime still recovers without re-running the candidate. This is the highest-risk item, so keep the raw logs or links in the PR rather than a summary.docs/contributing.mdedit in the rename PR, and that the two do not leave the line pointing at a path that does not exist in between.Clarifying questions (non-blocking)
trialsappear indocs/architecture.md#where-to-extendas an extension point, or is it just an internal execution leaf?docs/contributing.md:38currently lists the package as a place for new models, and the issue says that is wrong. The replacement wording should say where new models go instead.scenariosname as a string, such as receipts or run-store entries? If so, the "record formats unchanged" claim needs a test for it.Next step: open the single PR with the move, importer and doc updates, the fixture and tests, and the paired-wheel record. Run the grep in item 1 first and attach its output to the PR description. Dong-bot (authorized delegate): the issue records acceptance of the
harness/trials/name, and I have no objection to it.zengchuang-hw commentedon Nov 21, 2026
PASS(设计检视,不是实现验收)
源码支持改名:
external.py的职责是执行外部命令并保留试验事实,不是新增模型或场景的扩展入口。建议落地时明确三项:docs/contracts/function-owners.md中read_metric/validate_metric已不在当前external.py,不能只做路径替换。请把当前所有权行更新为execute_trial/collect_trial_checks,历史决策的原路径引用仍保留。status和recover返回 2,文件哈希不变;旧端恢复成功,候选执行次数不增加。退出码 2 是有意给旧会话的兼容边界,不与“兼容会话的 CLI 不变”混写。host-compatibility应按AGENTS.md标为非阻塞。记录宿主覆盖即可,不能把它设为检视或合入门槛。已核对当前模块、runtime identity 比对与既有会话 fixture 说明;尚无改名实现可测,未运行测试或查看 CI。
amy-why-3459 commentedon Nov 21, 2026
I've read the accepted scope: a behaviour-neutral rename of
src/inferrsi/harness/scenarios/toharness/trials/, with no forwarder, and a runtime-identity bump so sessions planned before the rename are refused.Dong-bot and zengchuang-hw already covered importer completeness, the gate versus installed-wheel proofs, paired-wheel exit-2 evidence, updating
docs/contracts/function-owners.mdtoexecute_trial/collect_trial_checks, and leaving host-compatibility advisory. I agree and will not repeat those points.Two items to pin down when the implementation PR opens:
Sequencing with already-open #559. That PR documents the current layout:
docs/contributing.mdand the architecture "Where to extend" table send new models toharness/servingandbenchmark/metrics.py/validators, and describeharness/scenarios/external.pyas the model-independent trial runner. If docs: correct stale layout text and complete the directory map #559 merges first, the rename PR must update those same sentences toharness/trials/. If the rename merges first, docs: correct stale layout text and complete the directory map #559 must be rebased, or it will reintroduce the old path.Open-PR impact. Besides docs: correct stale layout text and complete the directory map #559, please list in the rename PR description every open PR that still imports
inferrsi.harness.scenarios, and rebase or comment there when the rename lands.The
harness/trials/name and the deliberate resume break look consistent with the two public functions and with #498. I have no objection to a single implementation PR once the evidence plan above is followed.Dong1017 commentedon Nov 21, 2026
Implementation: #563 (stacked on #559). It carries the pre-rename session fixture, the paired-wheel record and an H200 A/B with no candidate-only failure; details in the PR body.
Dong1017 commentedon Nov 21, 2026
#563 merged as
af836963e73251622eefcf8a8a67f8ad3f4ecf12(squash), approved at head10bb12bby zengchuang-hw and amy-why-3459, with every required check green.Against the completion criteria:
inferrsi.harness.scenariosno longer exists, with no forwarder. The gate retires the name as a whole, with import and returning-file controls.harness.trials:harness/runner.py,harness/project_service.py,harness/optimization_session.py(three imports and the_runtime_identity()entry) andapps/legacy/run.py.execute_trialandcollect_trial_checksas Harness functions.tests/fixtures/optimization-session/pre-trials-rename-session/is a session that the previous wheel started and interrupted. The checkout test andtests/check_installed.pyshowstatusandrecoverexit 2 and leave every file unchanged. The paired-wheel record (old wheel recovers from the receipt without rerunning the candidate) is in the fixture README and the PR body.full-checkpassed on the merged head. The H200 A/B had no candidate-only failure.Draft PRs #488 and #326 still import the old path and need
harness.trialswhen they rebase. Closing is for the goal owner.