Repository navigation
fix(harness): preserve evaluation limits and replay original attempts (PR-3) - #564
zengchuang-hw wants to merge 11 commits into
Conversation
Adapt the authorized pool draft and freeze the null-timeout binding vector. Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Verify candidate and worker declarations against fixed preparation bytes. Preserve ordinary command behavior and leave evaluation startup disabled until entry verification is implemented. Agent-assisted-by: Codex (author) Agent-assisted-by: Unknown PR Pool agent (draft contributor) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Resolve standalone source entries under an explicit trusted root without imports. Retain the actual bytes and digest and preserve existing snapshots. Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Capture the configured standalone source entry after registration, retain observed code identity, and commit fixed zero-start refusals for missing or mismatching bytes. Execute matching entries from their retained source snapshot. Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Clarify mismatch argv and the single-step source target. Refresh the binding-vector guide and assert unstarted step facts alongside the resolved command. Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Apply existing attempt state and digest conflicts before fresh preparation IO or target support checks. Snapshot the saved request before host callbacks and preserve original records without new registration or execution. Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Intercept the supervisor used by Harness rather than the legacy execution wrapper in zero-start binding tests. Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Carry the corrected pre-registration guard and intercept the real Harness supervisor for T37 refusals. Positive controls observe a real process ID at the same boundary. Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Stop known expired or cancelled evaluation requests before source capture. Cover real null-timeout expiry, cancellation and descendant cleanup, durable cross-process replay, live owners, commit failure and registration races. Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Agent-assisted-by: Codex (author) Signed-off-by: zengchuang <zengchuang3@huawei.com>
Dong1017
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base 15f1703 and head d34a045. This is a full main-target diff, so it includes the unmerged #561/#562 units. No formal reviews are seeded for this head and there is no outstanding CHANGES_REQUESTED.
Status of this vote. It comes from Dong-bot (authorized delegate) on Dong1017's queue. The PR body names Dong as the accountable Harness parent, so this is a technical check. It is not an independent vote and should not count toward the two required independent write+ approvals.
Verified by reading the diff and the surrounding code
- Replay ordering.
execution_service.executeruns_validate_fieldsand then snapshots the spec withcopy.deepcopy. It computes the digest and callsstore._reenter_if_registered. Only after that does it run_validate_fresh(preparation read, target support), thenstore.claim._reenter_if_registeredselects on the samerequest_idcolumn thatclaimuses. It reuses_reenter, so the digest-first table is identical: RequestConflict, finished record, AttemptInProgress, RecoveryRequired.- The lookup registers nothing. The atomic
claimafter the preparation read still arbitrates a registration race, andtest_registration_race_after_lookup_still_uses_the_atomic_claimexercises that path. - Lookup failures raise
ExecutionPersistenceErrorwithstarted=None, so no start state is invented.
- Pre-registration binding.
prepared_binding.readbounds reads tosize+1bytes, closes the stream, re-checks size and sha256, and passesdict(ref)to the reader.projectandverifyuse exact field sets and refuse unknown rule revisions. Malformed bindings raise beforeclaim, which the tests assert by checking the store is empty. - Source entry (T37).
entry_code.capture_moduleaccepts only[A-Za-z_]\w*module names. It opens throughopen_regular, which rejects symlinks and paths outside the root. It never imports the module, refuses to overwrite an existing snapshot, and re-hashes the copy.- A matching entry runs as
python -I -B <captured path>, so the workspace,PYTHONPATHand later edits to the origin cannot replace the bound code. - A mismatch or an unavailable entry produces a fixed
not_startedrecord, withentry_code_identityset tofailedornot_performed. The mismatch record references the observed bytes. No process is launched. - A capture write failure raises
_RetentionFailure, which becomesExecutionPersistenceErrorwithstarted=False.
- A matching entry runs as
- Limits and budget. Evaluation checks cancellation and the deadline before source capture. The absolute deadline is used when
timeout_secondsis null (bound = deadline)._budgetis unchanged and still rejects policy, dimension and enforcement requests before binding. Replay never re-admits a budget. - Ordinary mode.
- Ordinary-mode
_argvis unchanged and records no code digest. - The ordinary runtime bindings are still recorded only for launched steps.
_step_errorsstill requires a nullexpected_code_sha256in ordinary mode and a non-null one in evaluation (V32).- The
test_harness_execute.pyexpectation change toInvalidExecutionSpecfollows from that V32 rule.
- Ordinary-mode
- Tests and packaging.
- The tests intercept the real
local_process._superviseand use a positive control with real PIDs. - They cover completed and refused replay after the source is repaired or the target is retired.
- They cover digest conflicts, a live owner, an abandoned owner, and commit failure after a real start.
- They cover null-timeout expiry, caller cancellation with a descendant, a fresh-process replay and a registration race.
check_installed.pyand theNEW_H_CLOSUREupdate keep the new modules free of Apps and Benchmark imports.- The docs and the migration JSON are consistent with the new test files. I checked the
consumersregistration by readingtest_apps_module_migrations.py, not by running it.
- The tests intercept the real
Not verified here
- I did not execute any tests, and current-head CI had not completed per the PR body. The author's reported counts (111 core tests; 297 passed, 2 POSIX-only skips on Windows) are unverified.
- The process-domain and null-timeout cases depend on the supervisor fixes in #562 and the separate #552 launch-boundary fix. I did not audit those here.
- Two items are optional follow-ups, not blockers. First,
_capture_inputstill hands the readeritem["source_ref"]from the internalctx.spec. That is pre-existing and not covered by the new snapshot-isolation claim. Second, the merge order is #561 → #562 → #564, so this PR needs a re-sync and green current-head CI before squash.
I will re-read if the author pushes after a rebase onto the integrated dependencies.
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).
amy-why-3459
left a comment
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed increment 73c30b8...d34a045 (PR-3 unit on the current #562 head). The main-target base is 15f1703 and the head is d34a045; that full diff still includes the unmerged #561/#562 units already reviewed on those heads. Immediately before this vote I re-fetched the PR reviews and comments: the only formal review on this head is Dong1017 APPROVED (6005639524), which the review itself marks as a non-independent parent technical check. There is no CHANGES_REQUESTED or other blocking comment.
Verified by reading the increment and the surrounding execute / RunStore paths:
- Static field checks run first; the request is snapshotted before reentry, preparation IO, budget admission, source binding or a worker start.
_reenter_if_registeredreuses the existing digest-first_reentertable and does not register or refresh an attempt. A finished record returns the original RecordRef; a digest mismatch is RequestConflict; a live owner is AttemptInProgress; an unfinished attempt with no owner is RecoveryRequired. Lookup failures are ExecutionPersistenceError withstarted=None.- Fresh preparation and target-support checks still precede
claim. The atomic claim remains the registration-race arbitrator, as covered bytest_registration_race_after_lookup_still_uses_the_atomic_claim. - Reentry does not re-admit a budget, reset the clock, open preparation, capture entry code or call
_supervise. Replay coverage includes a repaired or missing source, a retired target, and a fresh controller without the original archive or reader. - Fresh evaluation requests keep the original absolute deadline when
timeout_secondsis null (bound = deadline). Known expiry or cancellation is refused before source capture; mid-run expiry and caller cancellation reuse the process-domain supervisor and keep cleanup and lease-release facts separate. - The documented rejection of unsupported budget policy, dimensions or enforcement is the existing
_budgetpath on a fresh claim, not a new admission model. Replay never re-enters that path. - Ordinary-mode field rules and
_argvare unchanged; the evaluation-only pre-capture limit checks do not change ordinary launch ordering.
Ready lint-and-smoke, installed-package (3.10/3.11), declarations, pr-references, dashboard-browser and advisory host-compatibility have passed on this head. full-check (3.10 and 3.11) is still in progress (https://github.com/InferMatrixLab/InferRSI/actions/runs/44910395728). I did not run the tests locally. Merge should wait for green current-head Ready CI, the #561 → #562 → #564 integration order, a re-sync plus a second independent write+ LGTM on the synced head, and the separate #552 launch-boundary fix before whole CPU acceptance. This vote does not close #429 or #371.
|
当前 @david6666666 请与 Amy 一起完成独立检视,有意见先处理。#561 → #562 的前置合入后仍要同步并复核最终头;#552 继续由 Inference1 修复。新 #565 已被 owner 作为单独生产证据桥接任务接受,不混入本 PR,不把当前源代码小用例等同于完整生产验收。 |
There was a problem hiding this comment.
LGTM
PR review verdict — PASS
Reviewed base 15f17037afb6a57a7a51348f73979f81f5e0721f and head d34a0451f0d4edd76c604632b44d673f4f5d43ae.
Identity: cursor[bot] independent third-party / technical verify only (collaborator permission none). Not write+ / dual-tip / merge-ready. Stacked on #561/#562 (+ #552 launch-boundary prerequisite per PR body); resync after those land before squash.
Clean-code / repo style: PASS
- Replay path splits cleanly:
_validate_fields→ digest →_reenter_if_registered(no registration) →_validate_fresh→ atomicclaim; deepcopy snapshot isolates later host mutation of the caller's mapping. - Deadline/cancel checks precede entry capture on evaluation; null
timeout_secondsuses absolute deadline; budget still refused before binding and never re-admitted on replay. run_store._reenter_if_registeredreuses_reenterdigest table; lookup faults map toExecutionPersistenceErrorwithstarted=None(no invented start state).- Ordinary mode argv/runtime binding behavior preserved; evaluation fields retained in fixed records. Tests cover completed/refused replay, races, null-timeout, and supervisor boundary. Layering/migrations/
check_installedremain consistent with prior units.
Verified: tip Dong1017 + amy Approves; Ready green. Read tip delta (execute/_validate_*, _reenter_if_registered, evaluation-replay tests) — not CI alone.
Refs #429
Refs #371
PR-3 of three. Integration order: #561 → #562 → this PR; the separate #552 launch-boundary fix is also a prerequisite. The new unit is 73c30b8f → d34a0451. The main-target diff temporarily includes the unmerged prior units. This PR will be synchronized after those dependencies integrate.
A saved evaluation request now resolves existing state before opening preparation artifacts, admitting another budget or binding a target. Completed and refused attempts return the original fixed RecordRef. Digest changes conflict; live owners and unfinished attempts never implicitly resume. Fresh evaluation requests preserve the original absolute deadline and budget, including when step timeout is null.
Authority: Dong1017's bounded PR-3 delegation. zengchuang-hw / Codex is the author; Dong remains the accountable Harness parent, whose technical review is not an independent vote. Initial increment
6bbc5507was pushed early. Current head:d34a0451f0d4edd76c604632b44d673f4f5d43ae, based on #562 and synchronized with its corrected supervisor-boundary tests at73c30b8f.Behavior
Validation on the current head
local_process._supervisepath. Positive controls observe real process IDs at that same boundary.The small software worker proves execution/reentry facts. It does not produce complete P1 F/C/M evidence, establish a service gain or complete #429/#371 acceptance. Full shared CPU acceptance remains gated on the prior units and #552.
Review
Please check replay priority, no new allocation/binding/launch on all reentry paths, original deadline/budget preservation, cleanup facts and ordinary-mode compatibility. Obtain two eligible independent write+ formal approvals with first nonempty line
LGTMon the final synchronized head, green checks and satisfied dependencies before squash. Author/parent votes do not count.