[M0-CORE-08] Budget harness: Histogram/TimeIt/budgetCheck + budgets.json + laige-bench

CMakeLists.txt

@@ -225,4 +225,5 @@ if(LAIGE_BUILD_TESTS)
   # Dev tools that need the built engine library (gated with tests: a
   # library-only build does not need them).
   add_subdirectory(tools/fuzz)
+  add_subdirectory(tools/bench)  # M0-CORE-08: laige-bench
 endif()

README.md

@@ -14,8 +14,12 @@ isometric-first rendering, and a server-authoritative MMO path.
   (math, pools, Result, logging, config) lands over the remaining M0 steps
   in [roadmap/M0-foundations.md](roadmap/M0-foundations.md). So far:
   `laige::Result<T,E>` / `laige::Status` plus the error-code registry
-  (M0-CORE-01) and the structured logging facade (M0-CORE-02). No game-facing
-  engine features are buildable yet.
+  (M0-CORE-01), the structured logging facade (M0-CORE-02), the SimMath
+  deterministic-math interface with the default `fpx16_16` backend
+  (M0-CORE-03/04), memory pools (M0-CORE-05), the deterministic PRNG
+  (M0-CORE-06), the bounded JSON parser + serializer (M0-CORE-07), and
+  the budget harness (M0-CORE-08). No game-facing engine features are
+  buildable yet.
 
 ## Built by a local LLM
 
@@ -79,12 +83,22 @@ both variants. It carries the first functional engine code:
 `laige::Result<T,E>` / `laige::Status` plus the error-code registry
 (M0-CORE-01, `ctest -R result_status`), the structured logging facade
 (M0-CORE-02, `ctest -R logging`, API contract in
-[docs/api/logging.md](docs/api/logging.md)), and the SimMath
+[docs/api/logging.md](docs/api/logging.md)), the SimMath
 deterministic-math interface (M0-CORE-03 `fp32_pinned`,
 `ctest -R math_float`; M0-CORE-04 default `fpx16_16`,
 `ctest -R math_fixed` — API contract in
-[docs/api/sim_math.md](docs/api/sim_math.md)). Engine targets compile
-with `-Wall -Werror` and with exceptions and RTTI disabled (NFR-8.10).
+[docs/api/sim_math.md](docs/api/sim_math.md)), the memory pools
+(`ctest -R pools`, API contract in
+[docs/api/pools.md](docs/api/pools.md)), the deterministic PRNG
+(`ctest -R prng`, API contract in
+[docs/api/prng.md](docs/api/prng.md)), the bounded JSON parser +
+serializer (`ctest -R config_json`, API contract in
+[docs/api/json.md](docs/api/json.md)), and the budget harness
+(`ctest -R budget_harness`, API contract in
+[docs/api/budget_harness.md](docs/api/budget_harness.md); canonical
+benchmark command `./build/bin/laige-bench --suite=<name>`). Engine
+targets compile with `-Wall -Werror` and with exceptions and RTTI
+disabled (NFR-8.10).
 
 ## Documentation
 

budgets.json

@@ -0,0 +1,126 @@
+{
+  "version": 1,
+  "description": "Laige performance budgets (PRD 8.1). Each entry names one hard budget: 'target' is the limit in 'unit' (every budget is an at-most upper bound; target 0 is a hard zero budget, not 'unset'), 'measured' is the last recorded value of that budget (0 = not yet measured, the M0 convention), and 'metric' is the histogram statistic the check evaluates. Schema: docs/api/budget_harness.md, section 'budgets.json schema'.",
+  "budgets": [
+    {
+      "name": "frame_time_render",
+      "metric": "p95",
+      "unit": "ms",
+      "target": 8.3,
+      "measured": 0,
+      "workload": "worst-case isometric reference scene @ 1080p, mid-range laptop (PRD 8.1)"
+    },
+    {
+      "name": "sim_tick_avg",
+      "metric": "mean",
+      "unit": "ms",
+      "target": 3.0,
+      "measured": 0,
+      "workload": "10k entities, 2k dynamic bodies (PRD 8.1)"
+    },
+    {
+      "name": "sim_tick_p99",
+      "metric": "p99",
+      "unit": "ms",
+      "target": 5.0,
+      "measured": 0,
+      "workload": "10k entities, 2k dynamic bodies (PRD 8.1)"
+    },
+    {
+      "name": "sprites_50k_draw_calls",
+      "metric": "max",
+      "unit": "draw_calls",
+      "target": 30,
+      "measured": 0,
+      "workload": "50k visible sprites (worst-case isometric overlap), 3 parallax layers, UI (PRD 8.1)"
+    },
+    {
+      "name": "sprites_50k_cpu",
+      "metric": "mean",
+      "unit": "ms",
+      "target": 2.0,
+      "measured": 0,
+      "workload": "50k visible sprites (worst-case isometric overlap), 3 parallax layers, UI (PRD 8.1)"
+    },
+    {
+      "name": "iso_depthkey_rebuild",
+      "metric": "mean",
+      "unit": "ms",
+      "target": 0.2,
+      "measured": 0,
+      "workload": "10k dirty cells after a terrain edit (PRD 8.1)"
+    },
+    {
+      "name": "iso_picking",
+      "metric": "mean",
+      "unit": "ms",
+      "target": 0.01,
+      "measured": 0,
+      "workload": "one isometric screen-to-grid pick, O(1) (PRD 8.1)"
+    },
+    {
+      "name": "sim_heap_allocs",
+      "metric": "max",
+      "unit": "allocs_per_frame",
+      "target": 0,
+      "measured": 0,
+      "workload": "steady-state sim loop heap allocations, asserted in debug builds (PRD 8.1)"
+    },
+    {
+      "name": "engine_base_rss",
+      "metric": "max",
+      "unit": "mb",
+      "target": 100,
+      "measured": 0,
+      "workload": "engine base memory, empty scene running (PRD 8.1, all P0 platforms)"
+    },
+    {
+      "name": "cold_start_ssd",
+      "metric": "max",
+      "unit": "s",
+      "target": 2.0,
+      "measured": 0,
+      "workload": "game process to first frame on SSD (PRD 8.1, P0 platforms)"
+    },
+    {
+      "name": "cold_start_cold",
+      "metric": "max",
+      "unit": "s",
+      "target": 5.0,
+      "measured": 0,
+      "workload": "cold start (PRD 8.1, P0 platforms)"
+    },
+    {
+      "name": "build_time_ci",
+      "metric": "max",
+      "unit": "min",
+      "target": 10,
+      "measured": 0,
+      "workload": "clean build, engine + sample (PRD 8.1, CI)"
+    },
+    {
+      "name": "build_time_local",
+      "metric": "max",
+      "unit": "min",
+      "target": 5,
+      "measured": 0,
+      "workload": "warm local build, engine + sample (PRD 8.1)"
+    },
+    {
+      "name": "zone_server_tick_p95",
+      "metric": "p95",
+      "unit": "ms",
+      "target": 8.0,
+      "measured": 0,
+      "workload": "zone server, 2k players @ 20 Hz (PRD 8.1, 8-core server class)"
+    },
+    {
+      "name": "zone_server_ram",
+      "metric": "max",
+      "unit": "gb",
+      "target": 4,
+      "measured": 0,
+      "workload": "zone server RAM, 2k players @ 20 Hz (PRD 8.1, 8-core server class)"
+    }
+  ]
+}

docs/api/budget_harness.md

@@ -0,0 +1,211 @@
+# Budget harness (`laige::Histogram`, `laige::TimeIt`, `budgetCheck`)
+
+The measurement half of the PRD §8.1 performance-budget policy (M0-CORE-08;
+CORE-001: no performance claim without a reproducible measurement; AGENTS
+§12 report requirements). Public header:
+`src/laige-core/include/laige/budget_harness.h`; implementation:
+`src/laige-core/budget_harness.cpp`. The operator-facing tool is
+`laige-bench` (`tools/bench/laige-bench.cpp`; canonical command form in
+[building.md](../getting-started/building.md)). Unit suite:
+`ctest -R budget_harness` (`tests/laige-core/budget_harness_tests.cpp`).
+
+## Quick start
+
+```cpp
+#include <laige/budget_harness.h>
+
+// Measure one workload: one TimeIt per iteration, recorded in a
+// histogram sized to the run (every sample kept).
+laige::Histogram hist(laige::Histogram::Options{1000});
+for (int i = 0; i < 1000; ++i) {
+  laige::TimeIt t;                 // scope start
+  workload();
+  hist.record(t.elapsedMs());      // milliseconds
+}
+const laige::HistogramStats s = hist.stats();  // min/mean/p50/p95/p99/max
+
+// Check against a named PRD 8.1 budget (budgets.json, repo root).
+auto table = laige::loadBudgets("budgets.json");   // Result<BudgetTable, ErrorCode>
+if (table.isError()) { /* log table.errorText() (LOG-002); abort the run */ }
+laige::BudgetReportContext ctx;
+ctx.workload = "worst-case isometric reference scene @ 1080p";
+ctx.build    = "GCC 16.2.1, Debug";       // caller records what it owns
+ctx.machine  = "mid-range laptop (2019-2023 class), Linux";
+ctx.warmup   = 100;
+const laige::BudgetCheckResult r =
+    laige::budgetCheck(*table.value().find("frame_time_render"), hist, ctx);
+if (!r.passed) { /* print r.report; fail the CI run (PRD 8.1 policy) */ }
+```
+
+The end-to-end form is the tool:
+`./build/bin/laige-bench --suite=synthetic --runs=1000 --warmup=100
+[--budget=<name>]` — exit code 0 on pass, 2 on a failed budget check.
+
+## `laige::Histogram`
+
+A fixed-capacity **rolling-window** sample store. `record()` is O(1),
+allocates nothing, and takes no lock — the only hot-path-safe operation.
+Construction performs the two backing allocations (setup path).
+
+**Window semantics.** The histogram keeps at most `Options::capacity`
+samples. `record()` beyond capacity drops the *oldest* sample;
+`totalRecorded()` counts every sample ever recorded, so truncation is
+observable (`count() < totalRecorded()` means the window dropped samples —
+a benchmark that needs every sample sets `capacity >= runs`).
+
+**Statistics scope.** `stats()` describes exactly the stored window (the
+last `min(totalRecorded, capacity)` samples). `mean` is computed over that
+window (no running sum — no float drift). When `n == 0` the six
+statistics are NaN; callers check `n` (and `budgetCheck` turns an empty
+histogram into a loud `NO_SAMPLES` failure instead of reading NaN).
+
+**Percentiles (nearest-rank, the exact documented definition).** For the
+sorted stored window `v[0..n-1]` (n ≥ 1) and percentile p (0..100):
+rank `r = ceil(p·n/100)` in exact integer math, clamped to ≥ 1; the
+percentile is `v[r-1]`. p=0 is the min, p=100 the max, and a one-sample
+window returns that sample for every p. Nearest-rank (over linear
+interpolation): no fractional indices, no extra allocation, bit-identical
+on every platform (CORE-004).
+
+**Errors.** None — a histogram cannot fail. (Overflow of a full window is
+the documented drop-oldest behavior; `capacity 0` is legal.)
+
+**Performance.** `record()` — O(1), no allocation, no lock, no I/O (hot
+path). `stats()` — O(n log n) time (sorts a pre-allocated scratch buffer),
+no allocation (cold path: reports, budget checks — never frame/tick loops).
+Copy is O(capacity) (deep, cold path); move is O(1).
+
+**Threading (CONC-001).** One owner thread while mutable; `stats()` on a
+fully built histogram is a safe const read (publish contract, like
+`Result`/`Status`).
+
+**Misuse.** Recording wall-clock timestamps is a unit error (the harness
+measures durations, typically `TimeIt::elapsedMs()`). Ignoring
+`totalRecorded() > count()` on a percentile claim is a silent-window bug.
+
+## `laige::TimeIt`
+
+A scope timer over `std::chrono::steady_clock` (monotonic — immune to
+wall-clock adjustments; the right clock for durations). Milliseconds as a
+double. No allocation, no lock; the start point is immutable after
+construction, so `elapsedMs()` is a safe const read; `reset()` belongs to
+the owner thread. For "what time is it" use the logging facade's
+timestamps (system_clock, RFC 3339) — not `TimeIt`.
+
+## `loadBudgets` / `budgetCheck` / `BudgetTable`
+
+**`loadBudgets(path)`** — `Result<BudgetTable, ErrorCode>`. Cold path:
+file I/O + bounded JSON parse (ADR 0003: 1 MiB document, depth 32 — the
+file must also not exceed 1 MiB before it is read into memory).
+Errors (never silent, CORE-008):
+
+| Failure | Code |
+|---|---|
+| Unreadable file | `ErrorCode::IoError` (5) |
+| Malformed JSON | `ErrorCode::MalformedInput` (3) |
+| Unsupported version (`!= 1`) | `MalformedInput` |
+| Unknown/missing field, bad `name`/`metric`/`unit`, duplicate name, negative or non-finite number | `MalformedInput` |
+
+The loaded `BudgetTable` is immutable and safe to read from any thread;
+`find(name)` is a linear scan (the table is small by design — no hash map,
+PERF-006). A failed load produces no table (all-or-nothing).
+
+**`budgetCheck(entry, histogram, context)`** — `BudgetCheckResult`
+(`passed`, `measured`, `target`, `before`, `report`). Total: it cannot
+fail as an operation — its *outcome* is the pass/fail flag:
+
+| State | `passed` | `result` in the report |
+|---|---|---|
+| Histogram empty (n = 0) | `false` | `NO_SAMPLES` — a workload that recorded nothing is a broken harness; loud, never silent (CORE-008) |
+| `target > 0` | `measured <= target` | `PASS` / `FAIL` |
+| `target == 0` (hard-zero budget) | `measured == 0` | `PASS` / `FAIL` |
+
+`before`/`after` are the AGENTS §12 before/after pair: the entry's last
+recorded value (`measured` field of `budgets.json`) vs. the current
+measurement. Cold path: O(n log n) (the stats pass) plus report string
+building (allocates — reporting is never a hot path). Thread-safe on const
+inputs (the histogram must not be mutated concurrently — CONC-001).
+
+**Report format (stable, machine-greppable — LOG-001).** The first line is
+the grep contract; numbers are `%.6g` in the "C" locale (`nan`/`inf` as
+plain text):
+
+```text
+budget=<name> result=<PASS|FAIL|NO_SAMPLES> metric=<mean|min|max|p50|p95|p99> unit=<unit>
+  after=<measured> before=<last recorded> target=<limit>
+  stats: n=<n> min=<v> mean=<v> p50=<v> p95=<v> p99=<v> max=<v>
+  context: workload=<s> build=<s> machine=<s> warmup=<n>
+```
+
+The `context` fields are the AGENTS §12 machine/build facts **the caller
+harness records** (the tool supplies compiler/build type, the operator
+supplies the machine — see `LAIGE_BENCH_MACHINE`). The stats line comes
+from `laige::formatStatsLine`, the single source of the stats text.
+
+## `budgets.json` schema
+
+Repo root, versioned (ARCH-007: the reader rejects unsupported versions and
+fields explicitly). Schema version **1**:
+
+```json
+{
+  "version": 1,
+  "description": "...",          // optional; human notes (JSON has no comments)
+  "budgets": [
+    {
+      "name": "frame_time_render",   // required; snake_case id; unique
+      "metric": "p95",               // required; mean|min|max|p50|p95|p99
+      "unit": "ms",                   // required; identifier (ms, draw_calls, ...)
+      "target": 8.3,                  // required; finite, >= 0 (the PRD 8.1 limit)
+      "measured": 0,                  // required; finite, >= 0 (last recorded value)
+      "workload": "worst-case ..."    // required; non-empty (what the budget applies to)
+    }
+  ]
+}
+```
+
+- **`target`** is the hard PRD §8.1 limit — every budget is an *at-most*
+  upper bound. `target == 0` is a **hard-zero budget** (e.g.
+  `sim_heap_allocs`: zero steady-state heap allocations per frame), **not**
+  "not set".
+- **`measured`** is the last recorded value of the budget (the "before"
+  number). `0` is the M0 convention for *not yet measured*; when a budget
+  is first measured, the benchmark runner records the value here (this
+  file is the baseline home — see `docs/benchmarks/`, M0-EXIT-01 writes
+  the first baseline).
+- **All 15 PRD §8.1 targets** are present as named entries (sim tick,
+  50k-sprite scene, cold start, and the zone server each carry two
+  budgets). Until their subsystems exist (M1+) their `measured` values
+  stay 0 = not yet measured.
+- **Versioning:** an incompatible schema change bumps `version` and ships
+  a migration note here; `loadBudgets` rejects every other version
+  (never guesses).
+- **Validation (strict, ARCH-007):** exactly the six entry fields — a
+  missing or unknown field is `MalformedInput`; `name` must be unique
+  snake_case; `target`/`measured` must be finite and ≥ 0 (a well-formed
+  overflow token such as `1e999` parses to `+inf` and is rejected —
+  ADR 0003).
+
+## Performance (DOC-004)
+
+| Operation | Complexity | Allocation | Path |
+|---|---|---|---|
+| `Histogram::record` | O(1) | none | **hot path safe** (one index arithmetic + one store) |
+| `Histogram::stats` | O(n log n) | none (pre-allocated scratch) | cold (reports) |
+| `TimeIt::elapsedMs` | O(1) | none | hot path safe (one clock read) |
+| `budgetCheck` | O(n log n) + report build | report string (one or two) | cold only |
+| `loadBudgets` | O(file) parse | file buffer + table | setup only (file I/O) |
+
+Traps: calling `budgetCheck`/`stats` from a frame or tick loop (cold-path
+work in a hot loop, PERF-002); recording into a histogram whose capacity
+is smaller than the run and then claiming "all samples"; using
+`TimeIt` for timestamps (it measures durations).
+
+## Determinism
+
+The harness measures wall-clock durations: it is *not* deterministic
+simulation state and plays no part in replay/lockstep (ARCH-010). The
+synthetic `laige-bench --suite=synthetic` workload is deterministic by
+construction (fixed LCG constants — Marsaglia 2003 — no RNG, no
+allocation), which is what makes its baseline reproducible
+(`docs/benchmarks/baselines/m0-synthetic.md`, M0-EXIT-01).