Repository navigation
fix(brotli): resume a part-received meta-block instead of re-decoding it - #154
Merged
Merged
Conversation
When a compressed meta-block was not fully buffered, the decoder parsed and decoded it from its first command, hit the end of input, rolled everything back and started over on the next call. Total work grew with the square of the meta-block size: a 12 MB `brotli -q11` stream fed in 8 KiB pieces took 6.3 s instead of 0.07 s. Zero-length dictionary references let a hostile meta-block take any number of bytes, so a crafted stream could keep the decoder busy almost indefinitely. The meta-block's tables and command-loop position now live in the decoder between calls. The loop advances in units (an insert-and-copy symbol with its extra bits, one literal, one distance), and a unit that runs out of input is undone and retried once more bytes arrive. Output from finished commands is handed out right away. Decode time is now linear in the input whatever the chunk size, and one-shot decode speed is unchanged. Also reject a back-reference that runs past the meta-block before copying it, not after (it can be ~16 MiB). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hardening that came out of checking our exposure to CVE-2026-111497 ("Node2Shell", libbrotli). We are not affected by that CVE: our decoder is native Rust, has no
unsafe, and already handles a block switch with a single block type. This PR fixes a separate denial-of-service problem found during that review.Problem
If a compressed meta-block wasn't fully buffered, the decoder decoded it from its first command, hit the end of input, rolled back, and started over on every later
decodecall. Total work grew with the square of the meta-block size. Zero-length dictionary references let a hostile meta-block be any size compressed, so a crafted stream could keep the decoder busy almost indefinitely.Measured on a 16 MB base64 file compressed with
brotli -q11(12 MB):Output is byte-identical to the original in every case.
Change
MetaBlock,BlockCursor). The loop advances in units: an insert-and-copy symbol with its extra bits, one literal, or one distance. A unit that runs out of input is undone and retried once more bytes arrive.examples/micro) is unchanged within noise.Tests
decoder_resumes_inside_a_meta_block: qualities 1/6/11, input fed in 1-, 7- and 1000-byte chunks, output must appear from half a stream, and truncation must still be reported. It fails onmaster.cargo fmt --check,clippy --all-targets --all-features -D warningsandcargo test --all-features(1983 passed) all clean.🤖 Generated with Claude Code