[MagicalTux]

Fixes from a full audit of the crate before the 0.7.4 release. It's one commit per fix; each commit message gives the trigger, the effect and the fix, and every fix has a regression test that fails without it. Where a reference tool exists, the fix was checked against it (xz, zstd, bzip2, brotli, gzip, lz4, lzop, cabextract, unrar, lha, 7z, compress).

Wrong output or rejected valid data

• lznt1: the offset/length split was reversed, so real Windows/NTFS streams failed and our output only decoded with this crate. Now matches the worked example in MS-XCA §3.3.
• lzx (CAB): the decoder didn't realign at each 32 KiB frame boundary, the E8 filter was off by one, and odd-sized uncompressed blocks were rejected, plus three smaller bitstream issues. Almost every real CAB stream larger than 32 KiB decoded wrongly. It now agrees with cabextract on 3000 random streams.
• bcj: the x86, ARM64 and RISC-V filters didn't match xz, so data filtered by xz decoded wrongly here, and the other way round, with no error. They are now byte-identical to xz 5.8.
• lzfse: bvx2 blocks, which Apple's encoder uses for all but tiny inputs, couldn't be decoded. Checked with streams from lzfse_rust, which validates itself against Apple's library.
• rar2: symbol 256 didn't record its distance, so valid archives decoded to wrong bytes.
• lha: lh6/lh7 used the wrong dictionary sizes, so lha and 7z rejected inputs of 64 KiB or more, and 7z also rejected blocks with no matches. Every method except lh2 is now checked against the lha CLI in tests; lh2 is in fix(lha): make -lh2- match the LHarc format (needs a licensing decision) #146.
• brotli: the decoder rejected valid streams (a complex prefix code with a single code-length code).
• xpress_huffman: matches crossing the 64 KiB block end were cut short, and inputs over 4 GiB had their length silently truncated.
• lzs: a match exactly 2048 bytes back overflowed the 11-bit offset field, so MAX_DISTANCE is now 2047.
• bzip2: concatenated streams (pbzip2/lbzip2 output, or cat) now decode in full instead of stopping after the first.
• quantum and amiga_lzx: a debug-build panic on 19–21-bit windows, a dropped final odd byte, and rejected odd-sized uncompressed blocks.

Memory and CPU on untrusted input

• lha lh4–7: the decoder built its whole output at finish; 208 bytes of input could allocate 512 MiB. It now streams.
• rar5: the decoder buffered a whole block, using 1.25 GiB for 440 KiB of input; it now yields output as it goes. Overlapping filters made it stall while memory grew without bound.
• lzfse: blocks above Apple's per-block limits are now rejected; a 124-byte block used to decode to 15.7 MB before any output came back.
• huffman_codec, bcj2, rar2: declared lengths from the stream are now checked before anything is allocated. bcj2 and huffman_codec could panic with "capacity overflow".
• bzip2 and brotli: decoding was quadratic when input arrived in small chunks (bzip2: 155 s → 26 ms) or when one call held a large buffer. Encoding with brotli was also quadratic.
• .lzma encoder: it kept every compressed byte it produced (537 MiB peak for 512 MiB of input, now about 5 MiB).
• lzma and lzma2 encoders: the match finder stopped finding matches after 4 GiB of input. Output stayed valid, but the fifth GiB came out 88% larger for xz.

Streaming contract

• finish drain: finish failed with UnexpectedEnd instead of handing out output it still held, in the deflate, deflate64, zlib, lzss, embed (exact-size output buffer) and arc_squeeze decoders.
• flush() on zstd, xz and lzma2 encoders: it was a no-op that held back up to 1 MiB of input. These now emit complete blocks or chunks. .lzma can't fully support flush because the format has no sync point; the docs now say so.
• decode reaching end of stream: it now returns StreamEnd, and stops consuming trailing bytes, for gzip, brotli, xz and lzfse. The default discard_output now stops at the end of the stream too.
• reset(): deflate and deflate64 kept their preset dictionary only until the first reset. rar5 lost its declared size after reset, which truncated the next stream silently.
• zstd: skippable frames are recognised again; a byte-order bug made them return BadHeader.

I/O adapters (std and tokio)

• Readers: a retry after Interrupted or WouldBlock fed the previous chunk to the codec again, duplicating or corrupting output.
• Writers: if the inner writer failed partway through, output was lost and a retry corrupted the stream. The tokio write_all could fail with WriteZero. The tokio DecoderReader held decoded output back until more input arrived, which can deadlock a request/response protocol.

Build, CI and docs

• brotli without std: any build with brotli but without std failed to compile. A new CI step builds every algorithm on alloc alone; its feature list is read from all.
• docs.rs build: it now passes with embed.
• Deprecation versions: since on the hpack/qpack deprecations is corrected to 0.7.4.
• CLI test: a test that needed seven codecs is now gated on them.

Behaviour changes worth a release-note line

• EncoderWriter::flush(), both std and tokio, now does a real sync flush, the same as flate2. For gzip that adds a small marker to the stream on each call.
• When a bzip2, gzip or brotli stream ends exactly where the input ends, decode reports InputEmpty and finish reports StreamEnd. This is because another member could follow. Bytes after the stream make decode report StreamEnd immediately.
• The value of lzs::MAX_DISTANCE is now 2047; its type is unchanged.

Known issues not fixed here (each needs a redesign or a policy decision)

• rar3: it buffers all input and output before returning anything, so LimitedDecoder can't bound its memory.
• brotli decoder: a meta-block that arrives in pieces is decoded again from its start on every call. It's fine with large chunks but very slow with tiny ones.
• brotli encoder: flush() is still a no-op, holding back up to about 64 KiB.
• .lzma decoder: the dictionary is capped at 64 MiB. Raising it needs a growable dictionary or a memory-limit option, and the decoder's config type is currently ().
• amiga_lzx: never checked against real Amiga archives, because no reference tool or sample files were available.

Verification

On the final tip, in a clean private target directory, all of these pass:

• fmt, all 5 clippy variants and both full test suites (1946 tests with all features);
• every no_std build, including thumbv7em, and the alloc-free embed tests;
• the rustdoc build and the docs.rs-style build;
• MSRV 1.95, the embed footprint ceilings, the embed cross-check and the fuzz build;
• cargo semver-checks against v0.7.3.

🤖 Generated with Claude Code