Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
/tests/fixtures/legacy/** -text
# A session started by an older runtime keeps its exact bytes; its plan pins file hashes.
/tests/fixtures/optimization-session/pre-infra-removal-session/** -text
/tests/fixtures/optimization-session/pre-trials-rename-session/** -text
# Reusable guidance is editable text, not a frozen evidence bundle.
/evidence/README.md text eol=lf
/evidence/_template/** text eol=lf
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/ci-ready.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ jobs:
"$RUNNER_TEMP/wheel-env/bin/python" -m pip install "${wheels[0]}[skills]"
cp tests/check_installed.py "$RUNNER_TEMP/check_installed.py"
cd "$RUNNER_TEMP"
env -u PYTHONPATH INFERRSI_CHECKOUT_RESOURCES="$GITHUB_WORKSPACE/src/inferrsi" INFERRSI_SESSION_LOCK_PROBE="$GITHUB_WORKSPACE/tests/session_lock_probe.py" INFERRSI_PRE_INFRA_SESSION="$GITHUB_WORKSPACE/tests/fixtures/optimization-session/pre-infra-removal-session" "$RUNNER_TEMP/wheel-env/bin/python" -I check_installed.py
env -u PYTHONPATH INFERRSI_CHECKOUT_RESOURCES="$GITHUB_WORKSPACE/src/inferrsi" INFERRSI_SESSION_LOCK_PROBE="$GITHUB_WORKSPACE/tests/session_lock_probe.py" INFERRSI_PRE_CHANGE_SESSIONS="$GITHUB_WORKSPACE/tests/fixtures/optimization-session" "$RUNNER_TEMP/wheel-env/bin/python" -I check_installed.py

full-check:
strategy:
Expand Down Expand Up @@ -249,8 +249,8 @@ jobs:
)
# #498 slice 5: a session that main planned before infra/artifacts.py left the
# runtime identity must be refused by the installed wheel without any change.
environment["INFERRSI_PRE_INFRA_SESSION"] = str(
workspace / "tests/fixtures/optimization-session/pre-infra-removal-session"
environment["INFERRSI_PRE_CHANGE_SESSIONS"] = str(
workspace / "tests/fixtures/optimization-session"
)

def run(*command, cwd=work):
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ jobs:
"$RUNNER_TEMP/wheel-env/bin/python" -m pip install "${wheels[0]}[skills]"
cp tests/check_installed.py tools/check_release.py "$RUNNER_TEMP/"
cd "$RUNNER_TEMP"
env -u PYTHONPATH INFERRSI_CHECKOUT_RESOURCES="$GITHUB_WORKSPACE/src/inferrsi" INFERRSI_SESSION_LOCK_PROBE="$GITHUB_WORKSPACE/tests/session_lock_probe.py" INFERRSI_PRE_INFRA_SESSION="$GITHUB_WORKSPACE/tests/fixtures/optimization-session/pre-infra-removal-session" "$RUNNER_TEMP/wheel-env/bin/python" -I check_installed.py
env -u PYTHONPATH INFERRSI_CHECKOUT_RESOURCES="$GITHUB_WORKSPACE/src/inferrsi" INFERRSI_SESSION_LOCK_PROBE="$GITHUB_WORKSPACE/tests/session_lock_probe.py" INFERRSI_PRE_CHANGE_SESSIONS="$GITHUB_WORKSPACE/tests/fixtures/optimization-session" "$RUNNER_TEMP/wheel-env/bin/python" -I check_installed.py
env -u PYTHONPATH "$RUNNER_TEMP/wheel-env/bin/python" -I check_release.py \
installed --tag "$RELEASE_TAG"
- name: Build and validate the installable host plugin ZIPs
Expand Down
13 changes: 13 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,19 @@

### Changed

- `inferrsi.harness.scenarios` is now `inferrsi.harness.trials` (Refs #558). Its one
module, `external.py`, runs an external command as one trial and retains its record
(`execute_trial`, `collect_trial_checks`); it holds no scenario logic and is the same
for every model. There is no forwarding module or alias, and the import gate fails any
import of the old package name, with or without a file behind it.
- The optimization session's runtime identity now hashes `harness/trials/external.py`
instead of `harness/scenarios/external.py`. A benchmark or AB/BA session planned before
this change, whose plan carries that identity, is refused before `status`, `recover` or
any other operation runs, and none of its files change. Resume such a session with the
runtime that planned it. Service-reuse campaigns are dispatched before this check, so
the rename does not change how they resume.
`tests/fixtures/optimization-session/pre-trials-rename-session/` is a real session of
that kind, and its README records the paired-wheel recovery.
- The `inferrsi.infra` and `inferrsi.model` packages are removed (Refs #498, slices 3b, 4
and 5). There is no forwarding module or alias, and the import gate fails any import of
either package name, with or without a file behind it.
Expand Down
8 changes: 4 additions & 4 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ tool and retain the native identity and returned artifacts. The child owns the
complete start/knowledge/round/final lifecycle; the parent observes and collects. This path records the handoff; it
does not implement a background Agent CLI driver or a native-host deadline.

The application injects `harness.scenarios.external.execute_trial` into the
The application injects `harness.trials.external.execute_trial` into the
[two-rank MoE entry](moe-forward-benchmark.md). The standalone image benchmark
and [TTS service protocol](tts-benchmark.md) expose their documented execution
boundaries. Apps and Harness callers assemble these operations into optimization
Expand Down Expand Up @@ -317,7 +317,7 @@ src/inferrsi/
artifact_sandbox.py Artifact capture and bubblewrap replay; no vectors or verdicts
atomic_json.py Harness-owned atomic JSON evidence writes
runner.py One AB/BA loop; execute/snapshot/assess/report ports injected
scenarios/
trials/
external.py External command trials; metric validation injected
serving/
server.py Supervised Linux serving workers and their group cleanup
Expand Down Expand Up @@ -352,7 +352,7 @@ the dependency checks.

| Add | Main extension locations | Required qualification |
| --- | --- | --- |
| Model or scenario | `harness/serving`, `benchmark/metrics.py`, `benchmark/validators`; trials run through the model-independent `harness/scenarios/external.py` | Frozen request and metric contract; model-specific quality coverage |
| Model or scenario | `harness/serving`, `benchmark/metrics.py`, `benchmark/validators`; trials run through the model-independent `harness/trials/external.py` | Frozen request and metric contract; model-specific quality coverage |
| Repository | `apps/host/targets`, a composed scenario, `harness/data/repo_packs` | Stable source identity; command/result contract; environment isolation |
| Hardware | `apps/host/platforms` | Device selection, process exclusivity and independent backend/model evidence; a backend is registered in `apps/host/platforms/__init__.py` and its probe command is defined in the implementing module under `apps/host/platforms/`, not in package data |

Expand Down Expand Up @@ -400,7 +400,7 @@ float; `benchmark.legacy_run_v0.trial_metric` then applies the one finite, posit
metric contract (`benchmark.metrics.validate_metric`) to whatever the reader
returns, so a reader that yields `0.0`, a `bool` or a non-finite value records a
metric error rather than entering the evaluation. The Harness executor
`harness.scenarios.external.execute_trial` holds no metric rule: a caller that
`harness.trials.external.execute_trial` holds no metric rule: a caller that
passes a metric key also passes `metric=`, its validation function.

Run state is written by the termination path, never assumed. `status.json`
Expand Down
2 changes: 1 addition & 1 deletion docs/contracts/function-owners.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Status: **Proposed baseline**; individually **adopted** entries are identified b
| `harness/improvement` `usage_totals`/`assessment`/`holdout_accepts`/`rerun_assessment` | B experiment validity/comparison/acceptance rules; budget admission/reservation/charging reconciliation remains H | read normalized facts only; provider parsing does not move because of import convenience |
| same file `choose` | H search progression; `SelectionPolicy` split into B threshold rules and H progression/stop policy | no shared business `Policy` class; Apps does not create a retry loop |
| `harness/knowledge_capture` `assemble_context_text`/`_item` | K pure knowledge text projection | Apps connects K/H; Apps gives H the fixed expected text/hash; H stores the actually delivered text/hash; `verify_capture` byte integrity belongs to H and does not call K assembly in reverse; it is not a task-benefit verdict |
| `harness/scenarios/external` `read_metric`/`validate_metric` | B semantic validation; `execute_command` and retention → H | no whole-file ownership; offline assess executes nothing |
| `harness/trials/external` `execute_trial`/`collect_trial_checks` | H: run one command through `execute_command`, retain its record, and collect the trial checks; semantic metric validation (`read_metric`/`validate_metric`) is B and lives in `benchmark/metrics.py` | `execute_trial` applies only the metric callable the caller injects and holds no metric rule; offline assess executes nothing |
| `campaign_records` `STAGES`/claim capture/reconciliation | H records; dashboard/report presentation → Apps | first remove the `campaign_records → dashboard_data` reverse dependency; #333 is Proposed and no claim is made that a unified event envelope is accepted |
| `catalog.repository_pack`/`REPO_TYPES` | D020 neutral raw-data technical leaf; target interpretation stays H | K does not expose the pack reader publicly; `infra` changes only its same-leaf import; seven K bodies/root entries/tests move together |
| `apps/views/dashboard_workflow` `command_state`/`run_workflow`/`validation_progress` (added with the OBS dissolve; **adopted owner: Apps views; RSI-master architecture decision 2026-11-14 Asia/Shanghai, recorded in the [D008 wave-2 ownership decision](d008-wave2-ownership-decision.md#dashboard-workflow); independent PR review remains separate**) | Apps views: read-only presentation projection of workflow and command state | The module imports only the standard library and reads records the caller already loaded; it has no H, K or Benchmark edge. A group `state` such as `completed` is a display state for the run workflow, not a verdict, and must not be read as the legacy `Verdict`. `validation_progress` only validates counters from a snapshot `dashboard_data` has already read; it opens no file. |
Expand Down
2 changes: 1 addition & 1 deletion docs/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Nightly checks cover Linux and Windows checkouts with `core.autocrlf` set to

Use `harness/serving` for a new model's server and request protocol and
`benchmark/metrics.py` with `benchmark/validators` for its metric and quality checks.
`harness/scenarios/external.py` runs any external command as one trial and is the
`harness/trials/external.py` runs any external command as one trial and is the
same for every model. Use `apps/host/targets` with `harness/data/repo_packs` for
repositories, and
`apps/host/platforms` for hardware. A backend is registered in
Expand Down
2 changes: 1 addition & 1 deletion docs/module-layout.md
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,7 @@ Several flat modules remain shared seams until their slice starts.
| `apps/views/`, `apps/workflows/campaign.py` | Dashboard views, templates and the Knowledge-to-records connection | Apps composition under #371; campaign composition is grouped under workflows by #490 |
| `apps/integrations/mindie/` (`project.py`, `context.py`, `mcp.py`, `read_worker.py`) | The local MindIE adapter: project grant, stdio MCP, context export and the read worker | M1 (#452) moved the four root MindIE files here; no Harness, Knowledge or Infra module names MindIE, and the initializer imports nothing |
| `harness/` | Mixed legacy tree, not target isolation | Not redefined |
| `harness/runner.py`, `harness/scenarios/external.py`, `harness/atomic_json.py` | Old run execution facts and Harness-owned JSON writes | Ports injected; no Model edge, and no Infra edge except the local process executor |
| `harness/runner.py`, `harness/trials/external.py`, `harness/atomic_json.py` | Old run execution facts and Harness-owned JSON writes | Ports injected; no Model edge, and no Infra edge except the local process executor |
| `benchmark/legacy_run_v0.py`, `benchmark/sampling.py` | Old run gates and metric validation; image/video sample collection | Imports only `benchmark/metrics.py`; owns its rule inputs and verdict labels |
| `apps/legacy/` | Old run composition, old `evaluate` entry and HTML report | Supplies the Git source snapshot from `apps/host/targets`; remains in Apps |
| `apps/inspection.py` | Prepared-environment inspection bound to this host | Binds the `apps/host/inspection.py` facts API to `apps.host.commands.probe_command` and `apps.host.targets.repository_identity`; replaces root `inspection.py`, which R1 deleted with the three old-run root forwarders |
Expand Down
2 changes: 1 addition & 1 deletion docs/moe-forward-benchmark.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ inferrsi benchmarks run moe-forward-npu \
`--timeout-seconds` 都必须显式指定;两组配对为每实现每性能 case 16 个计时样本,
四组为 32 个。超时包括入口准备和执行,之后最多有额外的有界进程清理等待。
入口先冻结题目、候选和 grader 的副本及 SHA256,再通过现有
由应用入口注入的 `harness.scenarios.external.execute_trial` 执行。
由应用入口注入的 `harness.trials.external.execute_trial` 执行。
该执行器不含 metric 规则,入口在传 metric key 的同时传入自己的校验函数
(`metric=read_metric`)。
Python 调用 `evaluate` 时须显式传入 `execute_trial`。它额外清理本次唯一环境标记的
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ target-version = "py310"
extend-exclude = [
"tests/fixtures/legacy",
"tests/fixtures/optimization-session/pre-infra-removal-session",
"tests/fixtures/optimization-session/pre-trials-rename-session",
]

[tool.ruff.lint]
Expand Down
2 changes: 1 addition & 1 deletion src/inferrsi/apps/legacy/run.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
from inferrsi.benchmark.legacy_run_v0 import read_metric, trial_metric
from inferrsi.contracts import RunSpec
from inferrsi.harness.runner import run_experiment as _run_experiment
from inferrsi.harness.scenarios.external import execute_trial as _execute_trial
from inferrsi.harness.trials.external import execute_trial as _execute_trial

from .evaluation import run_rules
from .reporting import write_report
Expand Down
8 changes: 4 additions & 4 deletions src/inferrsi/harness/optimization_session.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ def runtime_environment() -> dict:


def _session_execute(root, directory, plan, deadline):
from inferrsi.harness.scenarios.external import execute_trial
from inferrsi.harness.task_protocol import run as run_protocol
from inferrsi.harness.trials.external import execute_trial

if plan["task_source"] == "project":
return _project_execute(root, directory, plan, deadline)
Expand Down Expand Up @@ -63,7 +63,7 @@ def _project_execute(root, directory, plan, deadline):
from inferrsi.harness.atomic_json import atomic_write_json
from inferrsi.harness.candidates import identity, members, retain
from inferrsi.harness.project import inspect_step, steps
from inferrsi.harness.scenarios.external import execute_trial
from inferrsi.harness.trials.external import execute_trial

output = directory / "execution"
output.mkdir()
Expand Down Expand Up @@ -117,7 +117,7 @@ def _runtime_identity():
root = Path(str(files("inferrsi")))
names = ("harness/optimization_session.py", "harness/optimization.py",
"harness/local_process.py",
"harness/serving/server.py", "harness/scenarios/external.py",
"harness/serving/server.py", "harness/trials/external.py",
"harness/atomic_json.py", "harness/artifact_primitives.py",
"harness/usage.py", "harness/agent_sessions.py", "harness/task_protocol.py",
"harness/protocol_child.py",
Expand Down Expand Up @@ -375,7 +375,7 @@ def _run_entry(entry: str, arguments: list[str]) -> None:
# resolve generic names such as project or candidates from Harness.
runner = Path(__file__).resolve().parent
sys.path[:] = [path for path in sys.path if Path(path).resolve() != runner]
from inferrsi.harness.scenarios.external import execute_trial
from inferrsi.harness.trials.external import execute_trial

sys.argv = [entry, *arguments]
sys.path.insert(0, str(Path(entry).parent))
Expand Down
2 changes: 1 addition & 1 deletion src/inferrsi/harness/project_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -661,7 +661,7 @@ def execute(root, directory, plan, pending, deadline):
"""One retained attempt, one owned service, ordered client stages, no silent retry."""
from .local_process import supervised_service
from .optimization_session import runtime_environment
from .scenarios.external import execute_trial
from .trials.external import execute_trial

settings = plan["evaluation_settings"]
conditions = pending["conditions"]
Expand Down
2 changes: 1 addition & 1 deletion src/inferrsi/harness/runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

from ..contracts import RunSpec
from .atomic_json import atomic_write_json as _write_json
from .scenarios.external import collect_trial_checks
from .trials.external import collect_trial_checks


def run_experiment(
Expand Down
4 changes: 3 additions & 1 deletion tests/_gate_policy_support.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,9 @@
# Slices 3b, 4 and 5 moved the artifact helpers to ``inferrsi.apps.artifacts`` and the
# provider modules to ``inferrsi.apps.integrations.anthropic``, and deleted both packages,
# so the two package names are retired as a whole.
RETIRED_MODULES = frozenset({"inferrsi.infra", "inferrsi.model"})
# #558 moved ``inferrsi.harness.scenarios`` to ``inferrsi.harness.trials``, with no
# forwarder, so the old package name is retired as a whole too.
RETIRED_MODULES = frozenset({"inferrsi.infra", "inferrsi.model", "inferrsi.harness.scenarios"})
# #412 R1, permanent: the one importer of the neutral data leaf outside
# ``inferrsi.knowledge``, with the only name it may bind (D020: Infra reads one pack
# description and reaches no Knowledge or Harness). This is not a manifest
Expand Down
2 changes: 1 addition & 1 deletion tests/architecture_real_ports.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@

from inferrsi.apps.legacy.evaluation import evaluate
from inferrsi.contracts import RunSpec
from inferrsi.harness.scenarios.external import execute_trial
from inferrsi.harness.trials.external import execute_trial
from inferrsi.knowledge.knowledge_operations import KnowledgeReader
from tests.architecture_closed_loop import _canonical, _digest

Expand Down
Loading
Loading