Skip to content

fix(lha): implement the real -lh2- format (clean-room) - #152

Merged
MagicalTux merged 2 commits into
masterfrom
fix/lha-lh2-cleanroom
Oct 23, 2026
Merged

MagicalTux merged 2 commits into
masterfrom
fix/lha-lh2-cleanroom

Conversation

@MagicalTux

Copy link
Copy Markdown
Member

Replaces #146, which reproduced LHa for UNIX's tree rules from a reading of its source (a licensing problem for this MIT crate).

Problem

Our -lh2- encoder and decoder only agreed with each other. The codec used a 510-symbol length alphabet, a 14-symbol bit-length position code and Okumura's LZHUF tree rules, none of which is the -lh2- format. It could not decode real -lh2- data, and LHa could not decode what it wrote.

Clean-room process

  • Spec side: one agent read the LHa for UNIX source and wrote a functional spec of the format: prose, math, tables and test vectors, with no code, pseudocode or identifiers. It checked the spec by encoding streams with its own throwaway model and having LHa for UNIX decode them. It also built LHa for UNIX into a standalone binary to serve as a black-box oracle.
  • Review: the coordinator read the spec for leaked code before passing it on.
  • Implementation side: a separate agent with a fresh context wrote this change. It had only the spec, the oracle binary run as a black box, and master. It never saw fix(lha): make -lh2- match the LHarc format (needs a licensing decision) #146, the LHa source or the spec writer's context.

Fix

  • Char tree: 286 symbols covering literals, lengths 3..=31, and an escape followed by 8 raw bits for longer lengths.
  • Position tree: codes P >> 6 and grows by one symbol per 64 output bytes. It keeps LHa's root sentinel and its separate update counter, and the low 6 bits follow raw.
  • Window: 8 KiB, filled with spaces at the start.
  • Limits: the encoder caps lengths at 256 and distances at the bytes already output. The decoder accepts lengths up to 287.

Tests

  • Spec vectors: initial codes and an adaptation trace; five streams encoded and decoded bit for bit; a 4 MB random token stream that rebuilds both trees, checked by SHA-256.
  • Fixture: a new oracle-verified fixture, checked against the decoder only.
  • LHa for UNIX conformance: opt-in. LHa for UNIX must test CRC-clean and extract our lh1, lh2 and lh4–lh7 output. The binary comes from COMPCOL_LHA, or from lha on PATH only if it reports itself as LHa for UNIX. Otherwise the test skips; Homebrew's lha is Lhasa, which can't decode -lh2-, so it skips there.
  • Fuzzing: the decoder_lha fuzz target now covers lh2.

Results:

Not verified

  • No fuzzing was run; the fuzz target only builds.
  • The MSRV build wasn't checked.
  • There are no historical -lh2- archives, and LHa for UNIX can decode -lh2- but not create it. Compatibility is shown against its decoder only.

🤖 Generated with Claude Code

Trigger: any `-lh2-` stream, in either direction. Effect: the codec was
an LZHUF variant with a 510-symbol length alphabet and a 14-symbol
bit-length position tree, which is not the -lh2- format: it could not
decode real -lh2- archives, and LHa could not decode what it wrote.
Our own round-trip tests could not notice.

Fix: rewrite dynamic_huff.rs clean-room from a written functional
specification of the format: a 286-symbol C-tree (literals, lengths
3..=31, and an escape followed by 8 raw length bits), a P-tree over
`P >> 6` that grows by one symbol per 64 output bytes plus 6 raw low
bits, with its root sentinel and separate update counter, and an 8 KiB
space-filled window. The encoder caps lengths at 256 and distances at
the bytes already output; the decoder accepts lengths up to 287.

Tests: the spec's vectors (initial codes, adaptation trace, five
streams encoded and decoded bit-exactly, a 4 MB random token stream
that rebuilds both trees, checked by SHA-256), an oracle-verified
fixture, and an opt-in conformance test that has LHa for UNIX
(`COMPCOL_LHA`, or `lha` on PATH if it is LHa for UNIX) test and
extract our lh1, lh2 and lh4..lh7 output. The fuzz target now drives
lh2 too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MagicalTux
MagicalTux merged commit df2dbd9 into master Oct 23, 2026
50 checks passed
@MagicalTux MagicalTux mentioned this pull request Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant