Skip to content

README: benchmark at 48 validators, 5,000 TPS, 1 s; apply-time tx-set validation results - #5499

Merged
SirTyson merged 5 commits into
stellar:overlay-v2-sharedfrom
SirTyson:perf/readme-48-validator-results
Nov 11, 2026
Merged

SirTyson merged 5 commits into
stellar:overlay-v2-sharedfrom
SirTyson:perf/readme-48-validator-results

Conversation

@SirTyson

@SirTyson SirTyson commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

README only, no code change.

The benchmark for this branch is now 48 validators (16 organizations, 4 regions), 5,000 SAC TPS, 1,000 ms target. The README reported the 30-validator / 3,000 TPS runs of #5495 and #5497; this updates it to the runs at the new fixture and adds #5498.

What the README now says

Tx-set push alone (#5497, 8f7768be7) With apply-time validation (#5498, 0c9a02e56)
Block time windows missed the first window on all 48 nodes on target in all four windows, on all 48 nodes
p75, average over 48 nodes 1,461 ms (limit 1,200) 1,081 / 1,132 / 1,118 / 1,142 ms
Time per ledger 1.42 s 1.03-1.08 s
Trigger to ledger closed 1.37 s 0.76-0.84 s
Run verdict FAIL (block time) FAIL (inclusion check only)
  • The run with the change fails the inclusion check, and only that check. Every node holds 4,800,060 transactions against 4,800,000 offered. The check is an exact comparison (performance-testing#4, which would allow this, is not merged).
  • The 60 extra transactions are not explained. The 30-validator run of Prototype: push the leader's tx set to peers when it is built #5497 had exactly 60 extra too. The explanation given then (late copies included twice in overlay-only mode) was an inference, and it does not predict the same number at a different rate and network size. The README now says so.
  • The IPC bug. The change was first run without its last commit (a33da62c4): same block times, but one node stopped reading its overlay IPC socket 15 s before the end of the load. The fix (0c9a02e56) is in the reported run, where fourteen sweeps of all 48 nodes (44 checkpoint publishes per node) found no affected node. One clean run does not prove the fix; the "before" run had no such event either.
  • One run per build. The gain from speculative execution is an upper bound, because apply is simulated by a sleep.

Changes

Corrections to the previous README

  • It said the perfnet inclusion check "has since been relaxed". That change is not merged.
  • It presented the cause of the 60 duplicate inclusions as understood. It is not.

Not verified

  • The quorum threshold (11 of 16 organizations) is computed from stellar-core's rule for generated quorum sets, not read back from a node.
  • Round-trip times between regions are the ones measured on the 30-validator fixture.

…time tx-set validation

The benchmark fixture is now 16 organizations (48 validators) in 4 regions
at 5,000 SAC TPS and a 1,000 ms target. Report the two runs at that
fixture: the leader tx-set push alone (stellar#5497) misses the first window
(p75 1,461 ms), apply-time validation with speculative execution (stellar#5498)
is on target in all four windows (p75 1,094-1,137 ms), with a FAIL verdict
for a node that lost its IPC reader (fixed in 0c9a02e, not rerun).

Describe the third experiment, update the measurement setup, limits and
known limitations, and condense the 30-validator results.
… test helper

It fails for one combination of the day's test seed and a partition's
instance number, also on the binary of stellar#5495 alone.
…ply-time validation

All four windows on target on 48 nodes (p75 1,081-1,142 ms), no node lost
its IPC connection. The verdict is FAIL on the exact inclusion check alone:
60 transactions more than offered on every node, the same excess as in the
30-validator run with the push; say that it is not explained. The run
without the fix becomes the earlier run that found the IPC bug.
@SirTyson
SirTyson marked this pull request as ready for review November 11, 2026 21:58
@SirTyson
SirTyson merged commit 5e695d4 into stellar:overlay-v2-shared Nov 11, 2026
2 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant