Skip to content

fix(lha): make -lh2- match the LHarc format (needs a licensing decision) - #146

Merged
MagicalTux merged 1 commit into
fix/pre-0.7.4from
fix/lha-lh2-format
Oct 22, 2026
Merged

MagicalTux merged 1 commit into
fix/pre-0.7.4from
fix/lha-lh2-format

Conversation

@MagicalTux

Copy link
Copy Markdown
Member

Stacked on #145. This is kept out of that PR because of a provenance question, which is your call.

Problem

Our -lh2- encoder and decoder only agreed with each other. The real lha tool (LHa for UNIX) fails its CRC check on any non-empty lh2 data we encode. Our implementation got three parts of the format wrong:

  • Length alphabet: the real format has 286 symbols, with an escape symbol followed by 8 raw bits for long matches. Ours had 510 symbols.
  • Positions: the real format codes the top bits (pos >> 6) with a tree that grows as output is produced, then sends the low 6 bits raw. Ours coded a fixed 14-class bit length.
  • Tree updates: LHA uses its own adaptive-Huffman update and rebuild rules, which decide every code bit. Ours used Okumura's LZHUF rules.

Fix

The encoder and decoder are rewritten to the real format. lha t now reports CRC OK on our archives, and lha p gives back the original input. tests/lha.rs now wraps each method's encoder output (lh1, lh2, lh4–lh7) in a .lzh archive and has lha verify and extract it; the check is skipped when lha isn't installed. A 208-byte lh2 fixture covers machines without lha.

⚠️ Provenance: please decide before merging

The other LHA methods in this crate are documented as clean-room. To get every code bit right, this rewrite was written from a reading of the LHa for UNIX sources (dhuf.c, slide.c), reproducing their tree-update and rebuild rules in new Rust code. LHa for UNIX has its own license with conditions, and the crate is MIT. The module docs now say the rules were "matched to the behaviour of the LHa for UNIX extractor".

Options:

  • Accept it as is. The code is independent Rust that reproduces a format's behaviour.
  • Clean-room it. Have someone else write a spec from that source, and reimplement from the spec alone.
  • Drop it. Close this PR and make -lh2- return Error::Unsupported instead of producing data that lha rejects.

Also: no tool available here can create lh2 data, so the decoder hasn't been tested on lh2 written by another encoder. It runs the same tree code that lha accepts, so it is expected to work.

🤖 Generated with Claude Code

Trigger: any non-empty input encoded with Lh2 and stored in a .lzh
archive as -lh2-, or a genuine -lh2- member fed to Lh2's decoder.

Effect: `lha t` reported a CRC error for every archive we produced. Our
encoder and decoder agreed only with each other: they used an invented
510-symbol length alphabet, a 14-symbol bit-length position class, and
Okumura's LZHUF tree update, none of which is the -lh2- bitstream, so
real -lh2- data could not be decoded either.

Fix: implement the format LHA actually uses: a 286-symbol char tree
(lengths 3..=31 direct, symbol 285 + 8 raw bits for longer matches), a
position tree over pos >> 6 that starts as a single leaf and gains a
symbol each time the output count passes a multiple of 64, followed by
6 raw low bits, and LHA's leader/block adaptive-Huffman update and
rebuild rules for both trees (their node layout fixes every code bit).
New tests have the lha CLI verify and extract our output for every LHA
method (skipped when lha is absent), pin the lh2 bitstream with an
lha-verified fixture, and check decoder reset.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MagicalTux
MagicalTux merged commit f0b0e9c into fix/pre-0.7.4 Oct 22, 2026
50 checks passed
@MagicalTux

Copy link
Copy Markdown
Member Author

Superseded by #152, a clean-room reimplementation: one agent wrote a code-free spec from the LHa for UNIX source, and a separate agent with a fresh context implemented #152 from that spec and black-box oracle testing alone. Note: this PR was merged into fix/pre-0.7.4 after #145 had already taken that branch into master, so this code never reached master.

@MagicalTux
MagicalTux deleted the fix/lha-lh2-format branch October 23, 2026 03:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant