Skip to content

Skip a judgement with no change to judge instead of paying for it - #188

Merged
madskronborg merged 5 commits into
masterfrom
madskronborg/skip-judgement-with-nothing-to-cite
Oct 23, 2026
Merged

madskronborg merged 5 commits into
masterfrom
madskronborg/skip-judgement-with-nothing-to-cite

Conversation

@madskronborg

Copy link
Copy Markdown
Collaborator

Before this change, a Run's semantic Judgement on a change with no files to judge still called the evaluator. That happens when nothing changed since the base, or when the change touched none of the constraint's paths. The call was paid for, but a report must cite a changed file, so it could find nothing. The Run then read aligned: true on no evidence. judge in src/drift.ts now skips such a Judgement before the call and records the reason no change to judge. The Run then reads unverified and says why, as for any other skipped Judgement. docs/steering.md names this case.

Closes #184

Verified: on the branch as it stands, regression, typecheck, format-touched and drift-oversight-correction passed. Collie's own check failed (see below).

These are the verifications this run is held to:

  • check: /home/mk/.local/share/mise/shims/bun run check (in /home/mk/.herdr/worktrees/.collie/madskronborg/skip-judgement-with-nothing-to-cite)

No reviewer looked at the two gate fixes after the last review; their own checks and dispositions are all that cover them. Gate fix 1 removed a doc comment that restated the judgedWithNoCall test helper (test/drift.test.ts) and re-recorded regression on the committed tree. Gate fix 2 changed nothing.

Findings left open as non-blocking:

  • [minor] Process and timing tests in check-processes, host and cleanup time out under the VM's load, on the base as well (test/check-processes.test.ts).
  • [minor] The 'many checks vs one' test's wall time scales with the machine's process count, so it cannot pass on a busy shared host (test/check-processes.test.ts).

Not settled by the run

  • assumed in build: The guard sits after the failed-diff check and before askJudgement, so an empty change is reported as no change to judge even when the evaluator is missing or unavailable; that is why the missing- and unavailable-evaluator tests needed a real change, as the plan anticipated.
  • assumed in build: In the oversight tests the tracked, uncommitted edit is a new src/work.ts that is git added (staged) and not committed. It shows in git diff HEAD, and src/ is already un-excluded in the test repository.
  • assumed in build: judgementsAtFinish gets the edit for both of its tests, so the goal-only test still checks that no judgement is asked for without a semantic constraint, now with a real change present.
  • 01-empty-change-is-skipped-not-judged.md: after merge: The gate's full suite passes on the merged tree.
  • 01-empty-change-is-skipped-not-judged.md: after merge: Semantic judgements are paid for on an empty change, which can report nothing, and the Run then reads aligned: true #184 closes on merge; the merge request description says Closes #184.
  • check failed: gate fix 2 disputed it and changed nothing: The latest gate run on 4d14632 (2026-10-22T21-13-51) is down to one failing test plus the leftover-process error it causes. Format, lint and typecheck pass, and 4056 tests pass. The failing test is 'killing what a host ended takes no longer for many checks that refuse it than for one' in test/check-processes.test.ts. It fails in every gate run of every 0.87.0 branch on this VM, not only this one. It failed in all three of this branch's runs, and in the gate runs of improvement-offer-review-fix-only-where-it-can-run (18:42) and improvement-stop-criteria-loop-on-unchanged-blocked-build (19:09 and 20:01). Those runs are under the same evidence directory. It also failed on a plain checkout of the base b4ef74a. origin/master has no commit since b4ef74a, so no fix is waiting to be rebased onto. The cause is the machine, not the diff. The test's wall time is a fixed ~12 looks at /proc, each a sequential read of every process. On vm-mk, with ~2,100 processes, one look took 1.6 to 2.2 s, which exceeds the 30 s timeout (see the minor finding). This branch's diff touches only src/drift.ts's judge, its tests and a docs line. Nothing in check-processes reaches judge. No change on this branch can turn this check green on this VM without editing unrelated process-supervision code, which the ticket does not cover. The human must decide: run the gate where the test can pass, or accept the one failure as pre-existing.

Mads Kronborg added 2 commits October 22, 2026 18:19
A report must cite a file the pack showed, so a Judgement of an empty
change can find nothing, yet it was paid for and let a Run that built
nothing read aligned: true. Record it as skipped, so the Run reads
unverified until a Judgement of a real change answers it.

Tests that judged an empty change by accident now judge a real one.

Closes #184
Drop a test helper comment that restated the code below it.
@madskronborg madskronborg self-assigned this Oct 23, 2026
On a machine with ~2,100 processes one look read them one by one and took
about 1.8 s. Ending a host's checks looks a dozen times, so the gate's
check-processes test overran its 30 s timeout on vm-mk, and a real stop got
one look where it budgets ten.
@madskronborg

Copy link
Copy Markdown
Collaborator Author

Update for 27205b1: scanning process bindings now reads up to 32 processes concurrently, preserving process identity checks while avoiding the busy-host timeout. The regression failed before the fix; the process tests and all 26 check-processes tests subsequently passed on the recorded earlier tree. The previously failing check-processes test dropped from 23.5 seconds alone to 7.6 seconds. The empty-change judgement fix remains in this PR, which closes #184.

Verification: regression, typecheck, lint, format and check-processes passed on an earlier tree; no current-tree pass is recorded for those verifications; Collie's final check failed.

Required verification:

  • check: /home/mk/.local/share/mise/shims/bun run check (in /home/mk/.herdr/worktrees/.collie/madskronborg/skip-judgement-with-nothing-to-cite)

The review found no actionable issues, and this delivery step changed no source files or tests. No post-review source fix was made. The carried non-blocking minor finding about the helper comment at test/drift.test.ts:526 remains unchanged.

The branch was pushed without ci.skip; origin already had this commit. GitHub CI on this head is currently failing: test job reports 4057 passed, 11 skipped and one failure, “a continuation queued behind an interrupted turn never starts”. The macos, desktop-macos, desktop and security checks passed. The earlier build assumption below that GitHub CI was green is historical and no longer describes this head. No checks were rerun in this delivery step.

Not settled by the run

Mads Kronborg added 2 commits October 23, 2026 13:37
A server that reads the interrupt before the SIGTERM is not draining yet, so it
starts the queued continuation. Nothing interrupted that turn, the server never
went idle, and the stop waited out its budget with the server still up: CI's
codex-stop failure on this branch. The stop now interrupts any new running turn
on the threads it is stopping while it waits for the server to exit.
The host writes a running check's record to <record>.new and renames it.
The test took the first one-file listing, so on a slow tree it caught the
.new and expected a target ending in .json.new. That was the one failure in
Collie's check of 27205b1.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Semantic judgements are paid for on an empty change, which can report nothing, and the Run then reads aligned: true

1 participant