Repository navigation
Speed up large JSON responses and refresh Java/Go benchmarks - #2
Merged
Merged
Conversation
Large response bodies exposed per-request costs that small benchmarks hid: - Cache templates by total size instead of skipping sources over 64 KiB, which were re-lexed on every request (~14 ms per 100 KiB). - Record which helper names a cached template mentions instead of scanning the body for ~70 names per render. - Serve stored jsonBody bytes directly; cache gzip output of static bodies by content; cache body files, validated per request by inode, size, mtime and ctime; encode journal response bodies on read. - Remove whole-body copies in entity text and the Handlebars evaluator. Decode request header bytes as ISO-8859-1, as Jetty does, so matching, templates and the journal agree with WireMock for non-ASCII headers. Add json_benchmark.py (any builds or the WireMock JAR, byte-checked), dynamic_template_differential.py (per-request values, random helpers, mapping/file edits, journal integrity; registered in verify.py) and a --pad/--repeat mode for template_differential.py. Against WireMock 4.0.0-beta.40: small responses 14-33% more throughput, 18-36% less CPU at 10k RPS, ~90% less memory; ~100 KiB JSON 1.6-11.8x throughput and 55-96% less CPU, with byte-identical responses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Extend the Java comparison to ~1 MiB and ~10 MiB JSON bodies. BreakMock's lead grows with size: 3.2-79x throughput at 1 MiB and 3.2-180x at 10 MiB, with 86-99.6% less CPU per request. Raise the gzip cache limit to 16 MiB bodies (64 MiB budget) so large static bodies are not recompressed. Render a binary-body mismatch the way WireMock does, as Java's default byte[] string "[B@" plus a random identity hash. The diagnostics suite masks only that token, width-for-width, requires it on both sides and compares every other byte. Earlier builds that printed base64 still fail. verify.py now passes all suites. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The README's small-response results came from a build made before the last commits, while changes were uncommitted, so their reports recorded the old base revision. Rerun every published benchmark (small responses, ~100 KiB, ~1 MiB and ~10 MiB JSON) from a clean checkout of 68cd311. Each report now records that commit, a clean tree and the binary hash. Regenerate the README and large JSON report tables from the new data, and add the in-process small-response A/B output. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A message action writes its reply frame before the handler journals the
"sent" and "received" events, so a client reading /__admin/messages right
after the reply could see neither. This made the streaming journal
differential flaky on CI ("websocket matched received and sent journal").
Incoming message handling now holds a read lock, and journal reads and
clears briefly take the write lock, so they wait for in-flight handling.
Event order and error handling are unchanged. A deterministic test reads
the journal from inside the reply send; without the barrier it sees an
empty journal.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Benchmarking ~100 KiB JSON responses exposed per-request costs that the 17-byte benchmark workloads hid. This PR removes them, fixes a header-decoding incompatibility found while verifying, and refreshes the Java-vs-Go benchmark results.
Against WireMock 4.0.0-beta.40 (same host and session; all results measured from a clean checkout of
68cd311, recorded in each report):The lead grows with body size: Java's per-request CPU rises with the body, while BreakMock serves cached bytes and renders templates in one pass. At 10k RPS, small-response p99 latency was not lower for Go (2.5–3.8 ms vs Java's 2.0–6.1 ms).
Before this PR, BreakMock was 30–51× slower than Java for large templated bodies.
Changes
jsonBodyis served from its stored wire form instead of re-validated and re-encoded per request.os.Root. Not cached on platforms without ctime.body/bodyAsBase64when read, instead of copying 2.3× the body per request.byte[]string,[B@plus a random identity hash (previously base64).diagnostics_differential.pymasks only that token, width-for-width, requires one on each side and compares every other byte; builds printing anything else still fail. This was the last failing compatibility case; CI'sverify.pysteps now pass.E9and one containing UTF-8ématched different stubs on Java and Go; matching, templates, cookies and journal now agree with WireMock.Small-response performance is unchanged (in-process A/B within ±2%,
BenchmarkSmallResponse).Verification
scripts/verify.pyagainst the pinned JAR: all 20 suite runs pass (diagnostics 41/41 in three consecutive runs).scripts/dynamic_template_differential.py(registered inverify.py), 238/238 vs Java at normal and >64 KiB sizes: per-request values, random UUID/string/int/now/pickRandomfreshness, mapping/file/jsonBodyedits with and without gzip, raw non-ASCII headers, and journal bodies under 120 concurrent requests. Original and optimized builds failed exactly the same checks before the header fix, so no regression was found.template_differential.py --pad 80000 --repeat 3: all 429 template cases inside 80 KB templates, byte-identical to Java.go test -race ./...,go vet ./..., and the cgo-free build's tests pass. New unit tests cover large-template responses (including concurrency), journal body output, body-file cache freshness/symlink escape (fails if the ctime check is removed), and header decoding.Known gaps (pre-existing, documented in docs/COMPATIBILITY.md)
responseDefinitionholds the template source where Java logs the rendered body; gzip responses are journaled compressed where Java logs them uncompressed. Being fixed in separate branches.Docs and data
testdata/reports/java-go-2026-10-13/,testdata/reports/benchmark-small-ab.txtandtestdata/reports/benchmark-large-json{,-1mib,-10mib}.json(with binary and JAR hashes).🤖 Generated with Claude Code