[M0-TOOL-02] Determinism checker skeleton: laige-detcheck + scenario contract + CI job - #12
Merged
Merged
Conversation
…contract + CI job
laige-detcheck (tools/detcheck): runs a named scenario in two build
configurations and compares per-tick state hash streams (FR-11.5,
AGENTS ARCH-010, TEST-004). The engine state-hash API arrives with
M1-DET-03, so the tool works against the documented hash-line output
contract.
Scenario contract (normative text in the tool header, mirrored in
docs/api/detcheck.md): one stdout line per tick `<tick> <hash>` — tick
starts at 0, step 1, no padding or leading zeros; hash = exactly 16
lowercase hex digits (the algorithm is NOT part of the contract —
lines compare byte-for-byte, so only run-to-run identity matters);
trailing newline optional, trailing \r tolerated (Windows CRLF);
stderr ignored; exit 0 on completion. Enforced strictly and bounded
(> 65536 ticks or > 64 bytes/line is a loud exit-2, CORE-008 — a
malformed scenario is never a silent mismatch).
Modes:
--scenario=synthetic|synthetic-perturbed [--ticks=N] [--seed=X]:
the built-in 32-body fpx16_16 + Prng workload, run twice in-process
(pure integer arithmetic — bit-exact across build, platform, ISA,
and compiler, ADR 0002; bodies seeded on an 8x8 grid spanning the
64-unit box so both wrap branches are exercised; the
synthetic-perturbed run-b adds 1 unit to body 3's x at tick 7 — the
step's Verify fixture).
--run-a=<bin> --run-b=<bin> [-- scenario-args...]: the real mode from
M1-DET-04 on — two builds of the same scenario source compared; the
`--` separator keeps scenario arguments unambiguous (strict surface,
API-008).
Report (stdout, stable and machine-greppable — LOG-001):
detcheck scenario=<name> result=OK ticks=<n>
run-a: <label-or-path>
run-b: <label-or-path>
detcheck scenario=<name> result=DIVERGED first_diff_tick=<t>
run-a: <t> <hashA> (the diverging tick pair, or
run-b: <t> <hashB> `stream ends: <n> ticks` when one run
ends early)
Exit 0 = match · 1 = divergence (loud) · 2 = usage / unknown scenario /
scenario failure / contract violation.
Scenario execution: fork/exec + pipe capture (POSIX), CreateProcessW +
PeekNamedPipe (Windows) — no shell, bounded memory, and the
scenario's stderr stays visible in the CI log. The build type is
stamped into the report labels (AGENTS 12) via
LAIGE_DETCHECK_BUILD_TYPE.
Tests (tests/detcheck, `ctest -R detcheck`) — 9 entries: the synthetic
self-check, the built-in perturbation fixture, identical and different
cross-binary pairs against fixture scenario binaries (one source,
five compiled variants: clean / perturbed / bad output / early exit /
short stream), the scenario-failure path, and unknown scenario. Each
test is a generated `cmake -P` check script asserting BOTH the exit
code and the required output fragments (the tests/api pattern; CTest
inverts PASS_REGULAR_EXPRESSION under WILL_FAIL, and a crash exits
non-zero like a correct failure, so the content must be checked).
CI: new `detcheck` job in ci-pull.yml and ci.yml (every PR and merge,
no ci:* condition — a tooling check like include-lint and
api-manifest, not an additional P0 OS build). It builds and runs
`laige-detcheck --scenario=synthetic`; the real-scenario step (two
build configurations of M1-SAMPLE-01's hello) is skipped until that
sample lands — M1-DET-04 activates it and records the result per
ARCH-010.
Docs (CORE-006, same change): docs/api/detcheck.md (contract, CLI,
report/exit codes, M0 hash scope, CI status), building.md (current
status entry + the tool-note now live), docs/README.md, tools/README.md,
root README.md layout line.
Local verify: ctest 31/31, zero warnings — g++ 16.2.1 Debug static,
g++ shared, clang++ 22.1.8 Debug, clang++ ASan+UBSan, clang++ TSan
(TSAN_OPTIONS=halt_on_error=1); `./build/bin/laige-detcheck
--scenario=synthetic` exits 0 and `--scenario=synthetic-perturbed`
exits 1 at first_diff_tick=7; the Windows execution path is
compile-verified by the CI MSVC job (not testable on Linux).
…7 active jobs) ci-pull.yml run of 82c548d: linux-gcc, linux-clang, linux-asan+UBSan, linux-tsan + include-lint, api-manifest, detcheck all passed; Windows/macOS label-skipped (default Linux lane); the new detcheck job's real-scenario step correctly reported 'skipped: no real scenario yet (M1-SAMPLE-01)'.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
M0-TOOL-02 (FR-11.5, AGENTS ARCH-010, TEST-004): the determinism
checker skeleton.
laige-detcheckruns a named scenario in two buildconfigurations and compares per-tick state hash streams. The engine
state-hash API arrives with M1-DET-03, so the tool works against the
documented hash-line output contract instead:
docs/api/detcheck.md): one stdout line per tick<tick> <hash>—tick starts at 0, step 1, no padding; hash = exactly 16 lowercase
hex digits; trailing newline optional, trailing
\rtolerated(Windows CRLF); stderr ignored; exit 0. The hash algorithm is not
part of the contract — detcheck compares lines byte-for-byte, so
only run-to-run identity matters.
gap, > 65536 ticks, or > 64 bytes/line is a loud exit-2 error — a
broken scenario is never a silent mismatch.
detcheck scenario=<name> result=OK ticks=<n>orresult=DIVERGED first_diff_tick=<t>+ run-a/run-b lines (thediverging tick pair, or
stream ends: <n> tickswhen one run endsearly).
unknown scenario / scenario failure / contract violation.
Modes
laige-detcheck --scenario=synthetic [--ticks=N] [--seed=X]fpx16_16+ Prng workload, two in-process runs — pure integer arithmetic, bit-exact across build/platform/ISA/compiler (ADR 0002)laige-detcheck --scenario=synthetic-perturbedlaige-detcheck --run-a=<binA> --run-b=<binB> [-- scenario-args...]--keeps scenario args unambiguous (strict surface, API-008)Scenario execution: fork/exec + pipe capture (POSIX), CreateProcessW +
PeekNamedPipe (Windows) — no shell, bounded memory, the scenario's
stderr stays visible in the CI log. Build type stamped into the report
labels (AGENTS §12) via
LAIGE_DETCHECK_BUILD_TYPE.Determinism scope (ARCH-010): same build/platform/ISA/compiler —
the built-in workload is pure unsigned-integer arithmetic
(
fpx16_16ops + xorshift128+), which the language standard makesbit-exact; no float anywhere. M0 hash scope: tick + seed + body
words; the definitive scope (including Prng state and the exact hash
function) is defined by M1-DET-03's
world.state_hash— the tool'sline-comparison is unchanged.
Tests (
ctest -R detcheck, 9 entries)The tool is tested with a synthetic two-run scenario (the step's
Verify clause): the built-in self-check, the built-in perturbation
fixture, and the cross-binary mode against fixture scenario binaries —
one source (
tests/detcheck/detcheck-fixture-scenario.cpp), fivecompiled variants (clean / perturbed / bad output / early exit / short
stream). Each test is a generated
cmake -Pcheck script assertingBOTH the exit code and the required output fragments (the tests/api
pattern — CTest inverts
PASS_REGULAR_EXPRESSIONunderWILL_FAIL,and a crash exits non-zero like a correct failure, so the content must
be checked).
CI
New
detcheckjob in both workflows (every PR and merge, noci:*condition — a tooling check like
include-lintandapi-manifest,not an additional P0 OS build): configure → build →
laige-detcheck --scenario=synthetic; the real-scenario step (twobuild configurations of M1-SAMPLE-01's hello) is skipped until that
sample lands — M1-DET-04 activates it and records the result per
ARCH-010. The
ci.ymlheader job count is fixed at the same time(it said eight jobs before M0-TOOL-01 even landed; now ten, matching
the actual job list).
Docs
docs/api/detcheck.md(contract, CLI, report/exit codes, determinismscope, CI status),
building.md(current-status entry; the reservedtool-note is now live),
docs/README.md,tools/README.md, rootREADME.md.Verification
ctest31/31 on g++ 16.2.1 Debugstatic, g++ shared (
LAIGE_BUILD_SHARED=ON), clang++ 22.1.8 Debug,clang++ ASan+UBSan (
LAIGE_ASAN=ON), clang++ TSan (LAIGE_TSAN=ON,TSAN_OPTIONS=halt_on_error=1)../build/bin/laige-detcheck --scenario=synthetic→ exit 0,result=OK ticks=256../build/bin/laige-detcheck --scenario=synthetic-perturbed→exit 1,
result=DIVERGED first_diff_tick=7(the step's Verifyclause).
testable locally on Linux).
Also in this change
17 → 19 done.
(
937ac7c/PR [M0-TOOL-01] Public API manifest: laige-api-scanner + checked-in laige-api.json + CI drift check #10 merged without updating the board or the log).building.md: fixed a missing line break in the current-statussection (pre-existing typo in the section this step extends).