Skip to content

session: hash actuals for the merkle update on multiple threads - #951

Draft
citizen-stig wants to merge 3 commits into
thrumdev:masterfrom
citizen-stig:nikolai/parallel-compact-actuals
Draft

citizen-stig wants to merge 3 commits into
thrumdev:masterfrom
citizen-stig:nikolai/parallel-compact-actuals

Conversation

@citizen-stig

@citizen-stig citizen-stig commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Session::finish hashes every written value on the calling thread before the merkle update can start. For sessions with many or large writes, this serial loop sits on the critical path of every commit.

Changes

  • New compact_actuals helper. It counts the written values to hash and picks threads = min(actuals_concurrency, available_parallelism, written / 8192). With fewer than 2 threads, or fewer than 16,384 actuals, it runs inline as before. Otherwise it splits the actuals into threads equal chunks and runs to_compact on scoped threads.
  • New Options::actuals_concurrency(usize): the maximum number of hashing threads per session. Default 16; 1 hashes sequentially. Independent of commit and I/O worker concurrency.
  • Chunks are joined in order, so the output is identical to the old loop. Hashes, ordering and the merkle update input don't change.

Touches the same lines of Session::finish as #952. Whichever lands second needs a trivial rebase.

Performance impact

No effect for sessions of up to ~24k keys, which use at most 2 threads. Above that, the gain grows with session size: Session::finish is about 2.5% faster at 48k–96k keys, about 3–4% at 240k keys, and 5–6% at 960k keys (85–105 ms per session). The serial loop cost about 90–110 ns per key, which is all this PR can save.

Session::finish time (median of 6 runs, base → this PR):

keys / session sov-hetzner-6 (Ryzen 7 7700) tower (Ryzen 9 3950X)
24k 76.9 → 76.0 ms (−1.1%, not significant) 108.0 → 108.0 ms (0%)
48k 147.5 → 144.0 ms (−2.4%) 206.5 → 202.5 ms (−1.9%)
96k 274.5 → 268.0 ms (−2.4%) 382.0 → 372.5 ms (−2.5%)
240k 581.0 → 558.5 ms (−3.9%) 807.0 → 786.0 ms (−2.6%)
960k 1470.5 → 1387.0 ms (−5.7%) 1987.0 → 1883.0 ms (−5.2%)

Per-session wall time without the fsync improves by the same absolute amount: −1% at 48k keys, −2.4 to −3.3% at 240k, −4.1 to −4.5% at 960k. Throughput including the durable commit is fsync-bound, so it moves within noise (+0 to +3%). Reads and durable_commit are unchanged.

How it was measured: benchtop transfer workload, built from master + #943 (seeded, reproducible workload) + #944, plus a session_finish span around Session::finish. Every run starts from a fresh copy of the same seeded 4M-account DB (-c 22 --buckets 4000000). The order is shuffled per repetition, with 6 repetitions per point, and runs are pinned to 6 cores + SMT siblings:

NOMT_BENCH_SEED=20261008 NOMT_DB_FOLDER=<db copy> taskset -c 2-7,10-15 benchtop run -b nomt -w transfer -c 22 \
  --workload-fresh 0 --distribution uniform --workload-concurrency 6 --commit-concurrency 4 --io-workers 3 \
  --buckets 4000000 --overlay-window-length 0 --page-cache-upper-levels 3 --page-cache-size 256 --leaf-cache-size 256 \
  -s $S --warm-up-ops $((2*S)) --op-limit $((8*S))      # S transfers = 2S keys per session; S = 12000 … 480000

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@citizen-stig

Copy link
Copy Markdown
Contributor Author

max parallelism must be configurable

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant