Skip to content

P2 performance: benchmarks, chumsky 0.13, fused lexer/parser (19× faster parse), O(n log n) verify - #13

Merged
leonbohn merged 7 commits into
mainfrom
benches
Nov 6, 2026
Merged

leonbohn merged 7 commits into
mainfrom
benches

Conversation

@leonbohn

@leonbohn leonbohn commented Nov 6, 2026 •

Copy link
Copy Markdown
Owner

Summary

P2 (performance) of REFACTOR.md, preceded by a benchmark suite so the claims are reproducible.

Before After
HoaRepresentation::try_from, 200k states / 15 MB 10.63 s (1.4 MiB/s) 0.547 s (27.6 MiB/s), 19.4×
verify(), 200k states 1.141 s 0.4 ms
parse, instructions (generated 250 / 1000 / 4000) 103M / 413M / 1655M 4.5M / 19.1M / 72.8M (≈ 22.9×)
parse, instructions (ltl / synthetic / spot fixtures) 248M / 329M / 1205M 9.2M / 13.4M / 49.9M (24–27×)

Full tables, one per step: BENCHMARKS.md.

Commits

  1. Benchmarks. gungraun instruction counts (parse, verify, write; deterministic, same setup as automata) and a criterion wall-clock target (HOARS_BENCH_XL=1 adds the 200k-state input). Inputs: the generator from REFACTOR.md plus fixtures in hoa/bench (the DPAs from automata, and 8 Spot-generated Büchi/generalized-Büchi/Rabin/Streett/nondeterministic/state-acc automata, script included). scripts/check now also builds the benches.
  2. chumsky 0.9 → 0.13 with borrowed tokens: 2.4×.
  3. Profile-driven lexer fixes (single-char tokens first, Cheap errors): 6.7×. Chumsky's expected-token bookkeeping for failed alternatives was ~22 % of the time although no error occurs.
  4. Cheap errors first (EmptyErr, re-parse with Rich only on failure) and choice.
  5. Lexing and parsing fused: the grammar runs on &str, every token parser skips trailing whitespace/nested comments (small custom parser). lexer.rs remains for splitting multi-automaton files and for classifying failures (lexical error / --ABORT--).
  6. Issue 12: verify() sorts ids instead of Vec::contains (+ test with an unsorted body).
  7. Issue 13: alias_iter(), get_version() -> Option<&str>, exact-size merge of acceptance, no per-parse timing calls.

Also: round_trip_fixture_files now walks hoa/ recursively.

Behaviour changes to review

  • Breaking API: all parser() functions are public and now have chumsky 0.13 / &str types; Header::get_version returns Option<&str>.
  • Parse error text is character-level now (e.g. [0 &] reports expected '1', '2', … '!', '@', '(' instead of token names); locations are unchanged. Lexical errors report the offending character but no longer a list of expected tokens. "Unclosed delimiter" reports are gone (Rich has no such variant).
  • Errors from parse_hoa_automata refer to the text of the single automaton, not the whole file.
  • Maintenance: the lexer.rs grammar is now a second copy of the token grammar (used only for splitting/diagnosis); a future change to token syntax must touch both. Folding the splitting into the fused parser would remove it.

Verification

./scripts/check (fmt, clippy -D warnings, doc, benches compile, 45 tests) passes. Instruction counts are reproducible; wall-clock numbers are from one machine. Not done: automata's automata-bench compare with a patched hoars (CLAUDE.md describes how), and no write measurement at 200k after the changes.

@leonbohn
leonbohn merged commit 4c8505a into main Nov 6, 2026
3 checks passed
@leonbohn
leonbohn deleted the benches branch November 6, 2026 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant