Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions docs/apps-module-migrations.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,12 @@
"docs/operations.md",
"src/inferrsi/apps/api.py",
"src/inferrsi/apps/archive.py",
"src/inferrsi/apps/workflows/benchmark_compare.py",
"src/inferrsi/apps/delivery/__init__.py",
"src/inferrsi/apps/handoff/bridge.py",
"src/inferrsi/apps/handoff/candidate_provider.py",
"src/inferrsi/apps/storage/artifact_reader.py",
"src/inferrsi/apps/views/attempt.py",
"src/inferrsi/apps/workflows/benchmark_compare.py",
"tests/check_installed.py",
"tests/test_apps_assess.py",
"tests/test_apps_module_migrations.py",
Expand All @@ -37,6 +37,7 @@
"tests/test_apps_publication.py",
"tests/test_apps_replay.py",
"tests/test_benchmark_preparation.py",
"tests/test_harness_entry_verification.py",
"tests/test_result_delivery.py"
]
},
Expand All @@ -60,7 +61,8 @@
"tests/test_apps_artifact_reader.py",
"tests/test_apps_assess.py",
"tests/test_apps_generation_plan.py",
"tests/test_apps_module_migrations.py"
"tests/test_apps_module_migrations.py",
"tests/test_harness_entry_verification.py"
]
},
{
Expand Down Expand Up @@ -139,7 +141,9 @@
"tests/test_apps_execution_binding.py",
"tests/test_apps_generation_plan.py",
"tests/test_apps_module_migrations.py",
"tests/test_benchmark_preparation.py"
"tests/test_benchmark_preparation.py",
"tests/test_harness_entry_verification.py",
"tests/test_harness_prepared_binding.py"
]
},
{
Expand Down
65 changes: 65 additions & 0 deletions docs/contracts/harness-entry-verification.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# Harness source-entry verification

The local CPU target can opt into the first frozen evaluation task with
`LocalCpuTarget(..., module_root=<trusted absolute directory>)`. The host owns
this configuration. Requests cannot supply the root. Existing targets with no
module root retain their ordinary-command behavior and refuse evaluation mode
before registration.

The first target supports a standalone source entry named
`module_root/<entrypoint>.py` with `entrypoint_kind="python_module"`. It reads a
regular file inside that root, without importing worker code or package
initializers. Dotted packages, bytecode/native modules and native executables
remain unsupported on this evaluation target. No ambient module/path fallback
is attempted. Ordinary-command module/executable resolution is unchanged.

## Ordering and retained facts

1. Static shape and fixed-preparation declaration checks run before registration.
Malformed bindings still produce no attempt or record.
2. RunStore registers the request and ordinary admission runs.
3. Harness copies the actual entry into its private attempt capture directory,
independently checks the retained bytes, and compares their SHA-256 with the
expected digest already bound to the frozen preparation.
4. A mismatch yields a fixed `not_started` record. Its
`entry_code_identity` check is `failed`, the runtime binding contains the
observed actual digest, and the check references the retained source bytes.
5. An unavailable/unsupported entry yields a fixed `not_started` record with a
`not_performed` check and no source reference. A source-module binding keeps
the actual digest null and the unresolved argv empty. No mismatch is invented.
6. A matching standalone module executes from its captured source file using the
current Python interpreter with `-I -B`. The workspace and `PYTHONPATH` cannot
replace the bound entry, and subsequent changes to its original source file
do not change the bytes being executed. Package execution is outside this
target's supported range. The runtime binding records the resolved command.

Runtime bindings describe resolution, not proof of a process launch. On an N1
mismatch, `argv` is the would-be command over the captured source and the step
remains `not_started`; `started` and step facts determine whether anything ran.
On N2 it is empty because the source command could not be resolved. This target
intentionally binds the frozen recipe's single worker step; declaration checks
reject multiple steps, so each private entry directory holds one snapshot.

Both refusal cases launch zero worker processes, acquire no process lease, and
retain the preparation/protocol/candidate identity in the fixed record. The
source proof uses `python-source@1`; its SHA is the observed byte identity, not
the frozen expectation copied into an observation. The complete interpreter is
not archived. Re-reading the fixed revision preserves those facts.

The entry capture is a Harness write. A capture failure uses the existing
`ExecutionPersistenceError` path with `started=false`. Record commit failure
also preserves that known start state and cannot produce a committed RecordRef.
There is no process fallback after an entry failure.

## Delivery boundary

These checks implement the post-registration T37-N1/N2 boundary separately from
the pre-registration binding tests. A real matching source entry is a positive
control. CPU execution facts and exit zero are not a Benchmark verdict.

The production preparation/Apps archive cross-tests use the accepted D023 /
`t108-v4` family and actual source bytes. Their small software worker does not
claim complete F/C/M evidence, a service gain, or whole-Issue production
acceptance. Existing ordinary CPU supervision is reused. The
[original-limit/T40 contract](harness-evaluation-replay.md) describes the next
unit; complete CPU acceptance retains its separate launch-boundary prerequisite.
61 changes: 61 additions & 0 deletions docs/contracts/harness-evaluation-replay.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Harness evaluation replay and original limits

`Harness.execute` accepts the saved canonical ExecutionSpec. Static schema and
field checks run first. Harness then snapshots the request and checks its
existing RunStore identity/digest before reading preparation bytes, resolving a
target, admitting a budget, binding source or starting a process.

| Existing state | Outcome |
| --- | --- |
| Finished committed attempt, same digest | Original fixed RecordRef and payload; no new binding, registration or worker |
| Same attempt, different digest | RequestConflict before artifact IO or target checks |
| Live owner, no terminal record | AttemptInProgress; attempt count unchanged |
| Owner gone or persistence failed without a record | RecoveryRequired; no implicit resume or new worker |
| No registered attempt | Validate fixed preparation/support, then atomically claim and execute |

The non-registering lookup does not replace the atomic claim. A caller can
register while another is reading the preparation; the later claim still
applies the same digest/state table. A corrupt or unavailable RunStore is an
error, never permission to start again. Lookup failures do not fabricate a
known start state.

Finished positive and refusal records both remain fixed. Repairing a missing
or mismatching source does not turn replay into a new run. The currently
configured target may be retired and the preparation reader unavailable on the
Harness replay path. The original record remains authoritative. Apps performs
its own saved-request binding before handing the ExecutionSpec to Harness; this
does not change Apps archive retention requirements.

## Identity, deadline and budget

The original request retains its attempt, request digest, target request,
deadline and complete budget. Host callbacks receive no mutable alias to the
internal request snapshot. A changed deadline, budget scope or preparation Ref
changes the digest and conflicts with an existing attempt. Reentry never
allocates a new identity, resets a deadline, reserves capacity or charges again.

Fresh evaluation requests keep the original absolute attempt deadline when
step timeout is null. Known expiry/cancellation stops before source capture.
Mid-run expiry and explicit caller cancellation use the existing process-domain
supervisor and retain process exit, cleanup and lease release as separate facts.

The initial CPU target keeps the existing free-CPU budget support boundary.
Unsupported policies, dimensions or enforcement requirements are explicitly
rejected before worker binding. Reserved/charged amounts remain unknown or
inapplicable rather than invented zeros. Actual observed usage and the original
budget admission remain in the fixed record. This slice adds no budget provider,
currency model or numeric acceptance gate.

## Verification boundary

The software tests combine real production preparation, Apps archives and the
configured source target. They cover completed/refused replay, digest conflicts,
live-owner and abandoned state, real running replay, real null-timeout expiry,
caller cancellation with a descendant, fixed cleanup facts, commit failure after
a real start, cross-process replay and registration races. Execution facts do
not provide a complete Benchmark F/C/M proof or a service-gain claim.

The launch-boundary prerequisite still has its own delivery and review. The
deadline/step-timeout check during log setup must be integrated and verified
before whole CPU acceptance of this slice. The replay implementation does not
duplicate that supervisor change or close the governing Harness/Apps Issues.
56 changes: 56 additions & 0 deletions docs/contracts/harness-prepared-binding.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Harness preparation binding before registration

The Harness evaluation path checks declarations before registering an attempt.
`inferrsi.harness.prepared_binding.verify(spec, artifact_reader)` returns a
`FixedPreparation` projection or raises a Harness `PublicError`. The spec must
first pass `execution_service.validate`'s static shape checks; `Harness.execute`
performs both checks in that order. Ordinary command mode reads no preparation.

The projection's `read(ref, reader)` port obtains a binary stream through
`reader.open(ArtifactRef)`, closes it, bounds reads by the declared size plus one
byte, and independently checks size and SHA-256. Missing bytes or a missing
reader produce `ArtifactUnavailable`; changed bytes or noncanonical JSON
produce `ArtifactIntegrityError`. Unknown host programming errors propagate.
`project(bytes)` is pure: it parses the canonical v1 envelope and validates the
fields used by Harness. It does not read a protocol, resolve an entry or start a
process. A schema version other than v1 is explicitly unsupported.

Binding compares the semantic prepared ID, protocol digest, operation and scope
with the fixed preparation. Disagreement is `IdentityMismatch`. It also checks:

- Candidate kind and role, a frozen expected candidate ID/hash/format when
present, and the supported `captured-sha256@1` or `expected-sha256@1` identity
rule. An unknown rule/revision is `UnsupportedRuleVersion`. The expected rule
requires a declared expected input hash even without a frozen candidate.
- A non-null expected code digest on every evaluation step, matching the frozen
recipe. This is an expected declaration, not an observed entry-code hash.
- One worker step preserving runtime role, entry kind/name/revision, all output
declarations, and arguments with whole-argument `{input:role}` / `{output:role}`
expansion. Inputs may use caller-chosen materialized paths; arguments must
match those paths. Paths are relative to the attempt directory (`cwd="."`),
and input/output paths must not overlap. Required frozen capabilities cannot
be dropped; additional declared capabilities remain subject to admission.

Malformed or conflicting declarations are `InvalidExecutionSpec`. For a fresh
request these refusals precede `RunStore.claim`: neither attempt nor record is
registered. Existing request state is handled after static field checks and
before preparation IO by the [replay contract](harness-evaluation-replay.md).
The public projection does not interpret candidate formats, grading rules,
evidence requirements or scores, and does not recompute Benchmark's semantic
prepared ID. The fixed ArtifactRef authenticates the preparation bytes; Apps
performs the separate archived-protocol binding through its canonical
`inferrsi.apps.handoff.bridge.verify_protocol` port.

The D023 conformance vector is kept separately under
`tests/fixtures/harness-prepared-binding/`. It adds the frozen code declaration
and a null step timeout without rewriting historical fixtures. Its canonical
bytes must agree between the Apps binder and Harness `request_digest`.
The vector is documentary; its historical code pin is not an available worker.

Targets without an explicit `module_root` keep valid evaluation-mode execution
unsupported before registration. A configured source target performs the separate
[post-registration entry check](harness-entry-verification.md) (T37). The
[original-limit/replay path](harness-evaluation-replay.md) covers the next
delivery unit, whose complete CPU acceptance retains its launch prerequisite.
Declaration checks alone do not establish
production CPU acceptance or completion of the governing Harness or Apps Issues.
3 changes: 3 additions & 0 deletions docs/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,9 @@ Effect classes used below:
| Rebuild scoped legacy evidence views | `python tools/rebuild_model_evidence.py <manifest> --archive <path> --output <new-directory>` | CPU-only; creates a new output directory | Authenticates original bytes and pinned projection/code; no extraction, network, archived-code execution or campaign qualification |
| Inspect evidence inventory / generate budget input | `python tools/validate_evidence.py [--budget-assets]` | Read-only | Offline layout/hash/schema checks; optional disposable #328 metadata view derived from campaign authority; no downloads |
| Execute one ordinary local CPU command with a fixed record | Python `inferrsi.harness.api.Harness.allocate` / `execute` / `read` | Experiment-executing / record-writing | One explicit RunStore root and one trusted local CPU target; ordinary command mode only; a record holds execution facts, so exit 0 is not a verdict; no device, model or network fallback |
| Verify a frozen preparation binding before registration | Python `inferrsi.harness.prepared_binding.verify` / `read` / `project` | Read-only / pure projection | Fixed ArtifactRef and read-only stream capability; no registration, entry resolution or execution; [binding contract](contracts/harness-prepared-binding.md) |
| Verify and execute one frozen source evaluation entry | Python `inferrsi.harness.api.Harness.execute` with trusted `LocalCpuTarget(module_root=...)` | Experiment-executing / record-writing | Fixed preparation and expected code digest; actual source captured after registration; T37 refusals start no worker; [source-entry contract](contracts/harness-entry-verification.md) |
| Replay the same saved execution request | Python `inferrsi.harness.api.Harness.execute` | Existing-state query / fixed record-reading | Same attempt and canonical digest; existing state short-circuits fresh binding/execution; [replay and original limits](contracts/harness-evaluation-replay.md) |
| Freeze a configured P1 preparation | Python `benchmark.preparation.PreparationService.prepare`; [production prepare](contracts/production-prepare.md) | Read-only resource access / pure encoding | Verifies supplied frozen worker/resource bytes and returns a preparation, protocol and role-keyed closure; Apps separately archives them before publishing a Ref |
| Visualize a recorded run or campaign | CLI `dashboard`; [dashboard](dashboard.md) | Report-generating / local read-only serving | Writes an offline HTML snapshot or serves an optional live view on loopback; does not run an agent or validate a claim |
| Capture a knowledge context | CLI `campaign capture-context`; [offline dashboard](dashboard.md#record-the-optimization-story-campaign-trace) | Local evidence-writing | Saves assembled context and resource hashes in a new `knowledge/<id>.json`. Supports ordinary questions and explicit IDs. The only accepted explicit `--relevance-method` value is `find-lexical-v1` |
Expand Down
2 changes: 1 addition & 1 deletion src/inferrsi/harness/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ def allocate(
)

def execute(self, spec: Mapping, *, cancellation=None) -> dict:
"""The flat ExecutionRecord of one ordinary command request, or a PublicError.
"""The flat ExecutionRecord of one supported local CPU request, or a PublicError.

``cancellation`` is a host handle with ``is_set()``, such as ``threading.Event``.
"""
Expand Down
60 changes: 60 additions & 0 deletions src/inferrsi/harness/entry_code.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
"""Capture a trusted local source entry without importing or executing worker code."""

from __future__ import annotations

import re
from dataclasses import dataclass
from pathlib import Path

from . import input_binding

_MODULE = re.compile(r"[A-Za-z_][A-Za-z0-9_]*", re.ASCII)


class EntryUnavailable(Exception):
"""The supported entry bytes cannot be resolved; no actual digest is known."""

def __init__(self, reason: str):
super().__init__(reason)
self.reason = reason


class EntryCaptureError(Exception):
"""A resolved entry could not be retained safely before execution."""


@dataclass(frozen=True)
class CapturedEntry:
path: Path
sha256: str
size_bytes: int


def capture_module(root: Path, module: str, directory: Path) -> CapturedEntry:
"""Snapshot one source module from an explicit trusted root.

The first CPU binding supports a standalone ``<module>.py`` entry. Dotted
package names, bytecode/native modules and import hooks are not resolved by
this port. In particular, neither a package initializer nor a worker module
is imported to locate its code. The caller owns the private capture directory.
"""
if not isinstance(module, str) or _MODULE.fullmatch(module) is None:
raise EntryUnavailable("entry_kind_unsupported")
name = module + ".py"
try:
stream = input_binding.open_regular(root, name)
except (OSError, input_binding.BindingError):
raise EntryUnavailable("entry_bytes_unavailable") from None
destination = directory / name
with stream:
# A capture directory belongs to one registered attempt. Never replace
# an existing entry snapshot on an interrupted or repeated operation.
if destination.exists():
raise EntryCaptureError("entry snapshot already exists")
try:
digest, size = input_binding.copy_stream(stream, destination)
if input_binding.file_sha256(destination) != (digest, size):
raise EntryCaptureError("entry snapshot differs from the captured bytes")
except OSError as error:
raise EntryCaptureError("entry snapshot could not be retained") from error
return CapturedEntry(destination, digest, size)
Loading
Loading