[M0-TEST-01] Test infrastructure conventions: docs/testing.md + seed handling + SeededRandom KAT suite + fuzz lane semantics

.github/workflows/ci-pull.yml

@@ -31,6 +31,12 @@
 # the ci.yml header for the lane semantics (fatal sanitizer reports,
 # report archiving, toolchain choice).
 #
+# Fuzz lane (PRD §14 "every commit (bounded)"; M0-TEST-01): no separate
+# fuzz job — the `fuzz_json_parse` ctest entry (1000 deterministic runs
+# of laige-fuzz on the json_parse target) runs inside every build
+# job's ctest, instrumented in the ASan tree. Seed and nightly-long-run
+# conventions: docs/testing.md.
+#
 # Include-graph lint (M0-CI-03; NFR-8.11, NFR-8.13): the `include-lint`
 # job runs on EVERY pull request, independent of the ci:* label selector
 # — it is a platform-independent repository check (Python 3 stdlib only),

.github/workflows/ci.yml

@@ -12,6 +12,12 @@
 #     the include-graph lint with dependency-count metric (M0-CI-03),
 #     the public API manifest drift check (M0-TOOL-01), and the
 #     determinism check (M0-TOOL-02)).
+#
+# Fuzz lane (PRD §14 "every commit (bounded)"; M0-TEST-01): there is no
+# separate fuzz job — the `fuzz_json_parse` ctest entry (1000
+# deterministic runs of laige-fuzz on the json_parse target) runs inside
+# every build job's ctest above, instrumented in the ASan tree. Seed and
+# nightly-long-run conventions: docs/testing.md.
 #   * Pull requests run exactly ONE P0 OS (label-selectable, default
 #     Linux) in the companion workflow .github/workflows/ci-pull.yml.
 #

README.md

@@ -96,9 +96,12 @@ serializer (`ctest -R config_json`, API contract in
 [docs/api/json.md](docs/api/json.md)), and the budget harness
 (`ctest -R budget_harness`, API contract in
 [docs/api/budget_harness.md](docs/api/budget_harness.md); canonical
-benchmark command `./build/bin/laige-bench --suite=<name>`). Engine
-targets compile with `-Wall -Werror` and with exceptions and RTTI
-disabled (NFR-8.10).
+benchmark command `./build/bin/laige-bench --suite=<name>`). The test
+   infrastructure conventions (M0-TEST-01: test layout,
+   `regress_<short-id>` regression tests, fuzz lane semantics, and seed
+   handling for randomized tests — `ctest -R test_infra`) are in
+   [docs/testing.md](docs/testing.md). Engine targets compile with
+   `-Wall -Werror` and with exceptions and RTTI disabled (NFR-8.10).
 
 ## Documentation
 

docs/README.md

@@ -34,6 +34,14 @@ sections below mark what exists and what is still to land.
   the scenario hash-line contract (`<tick> <hash>` lines, two build
   configurations) (M0-TOOL-02).
 
+## Testing
+
+- [Testing conventions](testing.md) — test layout (module dirs mirror
+  `src/`, `<module>_tests` executables), the `regress_<short-id>`
+  regression-test convention, `laige-fuzz` target registration and CI
+  lane semantics, and the seed-handling convention for randomized tests
+  (M0-TEST-01).
+
 ## Architecture decisions (ADRs)
 
 - [ADR index](decisions/README.md) — 0001 (name and license), 0002

docs/api/detcheck.md

@@ -49,6 +49,7 @@ output: at most **65536** ticks and 64 bytes per line.
 laige-detcheck --scenario=<name> [--ticks=N] [--seed=HEX|DEC]
 laige-detcheck --run-a=<scenario-bin-A> --run-b=<scenario-bin-B>
                [-- scenario-args...]
+laige-detcheck --compare-combined=<combined-stream-file>
 ```
 
 **Mode 1 — `--scenario`** (M0 built-in scenarios, in-process):
@@ -67,6 +68,41 @@ configurations) are executed and their hash streams compared. Everything
 after the `--` separator is passed to both scenario binaries, so
 scenario arguments can never collide with tool flags.
 
+Mode 2 is **two-stage**, and `--compare-combined` is its second stage:
+
+- **Phase 1 — `--run-a`/`--run-b`** spawns both scenario binaries. Each
+  child inherits this process's **stdout** (no capture pipe is created for
+  it), so its tick lines land in whatever captures the checker's stdout,
+  delimited by the marker lines the checker itself emits:
+
+  ```text
+  @@DETCHK-RUN-A-BEGIN@@
+  <tick A stream>
+  @@DETCHK-RUN-A-END <exitcode>@@
+  @@DETCHK-RUN-B-BEGIN@@
+  <tick B stream>
+  @@DETCHK-RUN-B-END <exitcode>@@
+  ```
+
+  Phase 1 exits `0` when both scenario processes ran to completion, `2`
+  on a spawn failure or a non-zero scenario exit (the reason on stderr).
+  It does **not** read back or compare the streams.
+
+- **Phase 2 — `--compare-combined=<file>`** reads the combined stream the
+  caller wrote (phase 1's captured stdout), splits it at the markers,
+  re-runs the scenario contract on each run, compares the two streams,
+  and reports (the report below).
+
+The split is forced by the CI Windows runner: it does not deliver handles
+the checker process creates (pipes or files, even with the `INHERIT` bit
+set, even after duplication) to child processes through `STARTUPINFO` —
+only handles the process itself inherited from its parent are delivered
+(measured in the M0-TEST-01 CI, runs 24/25). A scenario child therefore
+cannot be handed a capture pipe; its stdout must be the checker's own
+stdout, which the CTest check script (`execute_process`) captures and
+hands back in phase 2. The mechanism is identical on every platform, so
+the two-stage flow is exercised by the local suite on POSIX as well.
+
 ## Report and exit codes
 
 stdout (stable and machine-greppable — LOG-001):
@@ -83,18 +119,28 @@ detcheck scenario=synthetic-perturbed result=DIVERGED first_diff_tick=7
   run-b: 7 93a3363d5a1dffa9
 ```
 
-In mode 2 the first line is
-`detcheck scenario=<basename-a> vs <basename-b> result=... ticks=<n>` and
-the `run-a`/`run-b` lines carry the full scenario paths. When one stream
-ends early, the tick lines become stream-length notes
-(`stream ends: <n> ticks`) with `result=DIVERGED`.
+In mode 2 (phase 2) the first line is
+`detcheck scenario=combined result=... ticks=<n>` with `run-a`/`run-b`
+labels. When one stream ends early, the tick lines become stream-length
+notes (`stream ends: <n> ticks`) with `result=DIVERGED`.
 
 | Exit | Meaning |
 |---|---|
 | 0 | the two runs agree on every tick (deterministic) |
 | 1 | divergence detected (a determinism failure — loud, CORE-008) |
 | 2 | usage error, unknown scenario, a scenario run failed (non-zero exit, spawn failure), or a scenario violated the output contract (malformed line, tick gap, unbounded output) |
 
+`--run-a`/`--run-b` (phase 1) exits `0` when both scenario processes ran
+to completion and `2` on any spawn or scenario failure; the `0`/`1`
+comparison result comes from the `--compare-combined` phase.
+
+On Windows, phase 1 spawns with `CreateProcessW` (no `STARTUPINFO` — no
+handles are handed to the child, see above), waits for the child with
+`WaitForSingleObject`, and reads its exit code only after termination, so
+the `STILL_ACTIVE` sentinel (`259`) is never reported as a scenario exit
+code; a signalled child is reported as `128 + signal`, a non-zero scenario
+failure either way.
+
 ## The built-in synthetic workload
 
 32 bodies of Q16.16 position/velocity (the default deterministic backend,
@@ -118,9 +164,12 @@ computation of scenarios (the tool's line-by-line comparison is unchanged).
 
 A CI tool, not a hot path: one scenario run is O(ticks × 32); captured
 streams are bounded (65536 lines × 64 bytes ≈ 1.5 MiB worst case per
-run). Process execution is a plain pipe capture (fork/exec on POSIX,
-`CreateProcessW` on Windows) — no shell, no temporary files, bounded
-memory, and the scenario's stderr stays on the CI log.
+run). Process execution is plain inheritance (fork/exec on POSIX,
+`CreateProcessW` on Windows) — no shell, no capture pipe, bounded memory,
+and the scenario's stderr stays on the CI log. Phase 2 reads the combined
+stream file back with a bounded read capped by the contract (2 × 65536
+lines + markers ≈ 1.5 MiB worst case); the file is written by the check
+script between phases and lives in the build tree.
 
 ## Test suite
 

docs/getting-started/building.md

@@ -33,6 +33,7 @@ The canonical-commands table in [roadmap/README.md](../../roadmap/README.md)
 | TSan build | `cmake -S . -B build-tsan -DCMAKE_BUILD_TYPE=Debug -DLAIGE_TSAN=ON` |
 | Test (TSan tree) | `ctest --test-dir build-tsan --output-on-failure` |
 | Fuzz (bounded) | `./build/bin/laige-fuzz <target> --runs=1000` |
+| Fuzz (long, nightly form) | `./build/bin/laige-fuzz <target> --runs=1000000` |
 | Benchmarks | `./build/bin/laige-bench --suite=<name>` |
 | Determinism check | `./build/bin/laige-detcheck --scenario=<name>` |
 | API manifest | `cmake --build build --target laige-api` |
@@ -42,11 +43,15 @@ Notes:
 
 - `Debug` is the canonical `CMAKE_BUILD_TYPE`; `Release` is supported.
 - The tool rows above the lint row are live targets now: `laige-fuzz`
-  (minimal form from M0-CORE-07: the `json_parse` target and deterministic
-  bounded runs; M0-TEST-01 extends it with CI lane semantics and nightly
-  long runs), `laige-bench` (M0-CORE-08), `laige-detcheck` (M0-TOOL-02),
-  and target `laige-api` (M0-TOOL-01). Their command forms were fixed here
-  when they were reserved, so no step can drift them.
+  (M0-CORE-07: the `json_parse` target and deterministic bounded runs;
+  M0-TEST-01 documents the CI lane semantics — bounded `--runs=1000` in
+  every P0 job's `ctest`, the nightly long-run form above — and the
+  seed-handling rules), `laige-bench` (M0-CORE-08), `laige-detcheck`
+  (M0-TOOL-02), and target `laige-api` (M0-TOOL-01). Their command forms
+  were fixed here when they were reserved, so no step can drift them.
+  Fuzz and randomized-test seeds: fixed default `0x1F055EED`,
+  overridable (`laige-fuzz --seed=…`; tests via the `LAIGE_TEST_SEED`
+  environment variable) — see [docs/testing.md](../testing.md).
 - Include-graph lint (M0-CI-03): platform-independent (Python 3 stdlib
   only, no setup). It parses the `#include` edges of `src/**` and enforces
   the PRD §10.1 rules (laige-core includes nothing internal; arrows only
@@ -226,6 +231,13 @@ pinned set and the NaN/Inf policy):
   the self-check on every PR and merge, with the real-scenario comparison
   (two build configurations) skipped until M1-SAMPLE-01 (M1-DET-04
   activates it).
+- `test_infra` is the M0-TEST-01 CTest entry (`tests/testing`): the
+  `SeededRandom` suite of the `test_infra_tests` executable pins
+  known-answer hashes for the seed-handling convention (the default seed
+  `0x1F055EED`, the `LAIGE_TEST_SEED` override, the loud-failure path of
+  an invalid seed, and substream isolation) and prints the
+  machine-greppable `test-seed-check` lines that let two CI runs of the
+  same commit be compared byte-for-byte (docs/testing.md §4).
 - Every configure verifies the vendored dependency lock
   (`cmake/laige-deps-lock.cmake` against `deps.lock`); a tampered or
   unlisted file under `deps/` fails the configure loudly. GoogleTest is the

docs/testing.md

@@ -0,0 +1,145 @@
+# Testing conventions
+
+Source of truth for how Laige tests are laid out, named, seeded, and
+fuzzed (finalized by M0-TEST-01). Normative for every later step; the
+normative rule IDs are AGENTS.md §12 (TEST-001…TEST-010), PRD §14 (quality
+and verification cadence), and NFR-8.7 (parsers are fuzzed).
+
+## 1. Test layout (GTest integration)
+
+- **One directory per engine module under `tests/`, mirroring `src/`:**
+  `tests/laige-core` ↔ `src/laige-core`, and later `tests/laige-sim` ↔
+  `src/laige-sim`, etc.
+- **One test executable per module, named `<module>_tests`:**
+  `tests/laige-core/CMakeLists.txt` builds `laige-core_tests`. All of a
+  module's unit suites are separate source files in that one executable
+  (one `TEST` suite per concern), linked against `gtest_main` (which
+  provides `main()`) and the module library under test.
+- **Module suites are exposed as their own CTest entries** with an
+  unquoted `--gtest_filter` selecting exactly the step's suites; the
+  unfiltered entry runs the whole module. Each roadmap step names its
+  Verify command as `ctest -R <entry>` (pattern: M0-CORE-01…08). The
+  gtest filter must be one *unquoted* argument — CTest passes quoted
+  arguments through with the literal quote characters, which silently
+  under-runs the suite (see `tests/laige-core/CMakeLists.txt`).
+- **GoogleTest is a dev-only dependency** (PRD §11): vendored in
+  `deps/googletest`, integrity-locked in `deps.lock` (M0-DEP-01), and
+  linked into test executables only — **never into engine libraries**
+  ([ADR 0004](decisions/0004-google-test-vendoring.md)).
+- **Non-module test directories** (no `src/` counterpart):
+  - `tests/tools`, `tests/api`, `tests/detcheck` — checks that exercise
+    the tools in `tools/` (include-graph lint, API manifest scanner,
+    determinism checker); CTest entries named after the tool.
+  - `tests/support/` — shared test-only headers (header-only, no build
+    targets). Currently `laige_test_seed.h` (this step).
+  - `tests/testing/` — the test-infrastructure checks; executable
+    `test_infra_tests`, CTest entry `test_infra`.
+- **Every test TU is compiled with the NFR-8.10 policy**
+  (`laige_apply_engine_policy`: `-Wall -Werror -fno-exceptions
+  -fno-rtti` / the MSVC equivalent), and the module test executables
+  self-check that policy with `static_assert`s (a policy violation
+  fails the build loudly).
+- **Test executables are single-owner and run single-threaded per CTest
+  entry** (CONC-001); concurrency under test gets its own suites
+  (pattern: `LogConcurrency`) and the TSan tree (NFR-8.2).
+
+## 2. Regression tests (AGENTS TEST-003)
+
+- Every bug fix ships a regression test **named `regress_<short-id>`**
+  inside the existing module suite (the suite name is unchanged; the
+  short-id names the defect).
+- The test **must fail before the fix and pass after it** (TEST-003).
+  Demonstrate that in the fix step: write the test first, record the red
+  run, apply the fix, record the green run — both belong in the step's
+  Verify note.
+- The fix is incomplete without its `regress_` test in the same change
+  (CORE-007, TEST-003).
+- First test under the convention (renamed by this step):
+  `ConfigJsonValid.regress_json_object_member_ws` in
+  `tests/laige-core/config_json_tests.cpp` — the M0-CORE-08 finding that
+  the JSON parser rejected object members separated by `", "` (the
+  hand-formatted repo-root `budgets.json` demonstrated it).
+
+## 3. Fuzz runner (`laige-fuzz`; PRD §14, NFR-8.7)
+
+`tools/fuzz/laige-fuzz.cpp` (M0-CORE-07) is the deterministic bounded
+runner: every input is generated from `laige::Prng` (M0-CORE-06), so a
+given `(target, runs, seed)` reproduces the exact same input sequence on
+every platform (the Prng's cross-platform bit-exactness, ARCH-010). A
+target may return any `Status`; the run fails only on process death
+(crash or sanitizer report), which ctest turns into a test failure
+(CORE-008: no silent failure).
+
+- **Registering a fuzz target:** add a
+  `void target(const std::uint8_t*, std::size_t)` entry point and one
+  `kTargets` row in `tools/fuzz/laige-fuzz.cpp`, then register a CTest
+  entry `fuzz_<name>` next to it (`COMMAND laige-fuzz <name>
+  --runs=1000`, `TIMEOUT`, and `TSAN_OPTIONS=halt_on_error=1` in the TSan
+  tree — the shape of `fuzz_json_parse` in `tools/fuzz/CMakeLists.txt`).
+- **Bounded runs in CI, every commit:** the `fuzz_<name>` CTest entries
+  run inside every P0 job's `ctest` (both `ci.yml` and `ci-pull.yml`),
+  1000 runs per target — the PRD §14 "every commit (bounded)" lane. In
+  the ASan tree (`build-asan`) the runs are instrumented, so a crash or
+  UB fails the job loudly (NFR-8.7).
+- **Nightly long runs (PRD §14 "nightly (long)"):** the canonical form is
+  `./build/bin/laige-fuzz <target> --runs=1000000 [--seed=HEX]`. The
+  scheduled nightly lane is documented here but not yet wired: it lands
+  with the first M1 fuzz target (asset import / network packets, PRD §14
+  fuzz row), when a long run protects more than the parser. Until then
+  the bounded lane above is the complete fuzz cadence in M0.
+- **Seed handling:** fixed default seed `0x1F055EED` ("one-fuzz-seed"),
+  overridable with `--seed=` (0x-prefixed hex or decimal). This is the
+  same default seed the test suites use (below) — one documented
+  default seed repo-wide.
+
+## 4. Seed handling for randomized tests
+
+- **No nondeterministic sources in tests.** Every randomized test draws
+  from `laige::Prng` (M0-CORE-06) — never from wall-clock time,
+  `std::random_device`, or any other source that varies between runs.
+  CI must be deterministic: the same commit produces the same test
+  values on every CI run, on every P0 platform.
+- **Seed source:** `tests/support/laige_test_seed.h` (this step).
+  `laige::testing::TestSeed()` returns the fixed default
+  `kDefaultTestSeed = 0x1F055EED` — identical to laige-fuzz's default —
+  unless `LAIGE_TEST_SEED` is set (0x-prefixed hex or decimal, read at
+  call time). A set-but-unparseable value records a test failure with
+  the offending value and falls back to the default (CORE-008: an
+  explicit misconfiguration is loud, not silent).
+- **Stream isolation:** `laige::testing::TestPrng(id)` derives the
+  substream `Prng::deriveSubstream(TestSeed(), id)`. Each randomized test
+  file declares its own stable, named substream-id constant so two tests
+  never share a stream position (the Prng contract: a copy shares the
+  position; interleaved draws are a caller bug, not a detectable error).
+- **CI determinism, checkable in the logs:** the `SeededRandom` suite
+  (`tests/testing/`, CTest entry `test_infra`) pins known-answer FNV-1a
+  hashes of 65536 draws under the default seed and under a documented
+  override seed, and prints one machine-greppable line per KAT:
+
+  ```
+  test-seed-check default seed=0x000000001f055eed stream=1 draws=65536 fnv1a=0x7ea4049545656830
+  test-seed-check override seed=0x2468acce01234567 stream=2 draws=65536 fnv1a=0x535d2ca741b61cbf
+  ```
+
+  The line lands in the ctest output, every CI job's log, and the
+  archived `Testing/Temporary/LastTest.log` (linux-asan / linux-tsan
+  artifacts) even when a KAT mismatches. Two CI runs of the same commit
+  must show byte-identical `test-seed-check` lines — that is the step's
+  cross-run identity check (M0-TEST-01 Verify).
+- **Investigating a flaky randomized test:** set
+  `LAIGE_TEST_SEED=<hex>` for the ctest run to reproduce the exact
+  stream that produced the failure (the committed KATs pin the
+  default-seed values, so the override never hides a KAT regression).
+
+## 5. Running this step's checks
+
+| Purpose | Command |
+|---|---|
+| Seeded-random KAT suite | `ctest --test-dir build -R test_infra --output-on-failure` |
+| Bounded fuzz, `json_parse` | `./build/bin/laige-fuzz json_parse --runs=1000` |
+| Fuzz with an explicit seed | `./build/bin/laige-fuzz json_parse --runs=1000 --seed=0x12345678` |
+| Nightly long run (documented form) | `./build/bin/laige-fuzz json_parse --runs=1000000` |
+| Reproduce a randomized test's stream | `LAIGE_TEST_SEED=0x… ctest --test-dir build --output-on-failure` |
+
+The full canonical command table (build trees, sanitizers, tools) stays
+in [getting-started/building.md](getting-started/building.md).