fix: Parser.parse returns the documented tree; maps() sees the ISC corpus - #773
Merged
Merged
Conversation
JsonIR merged every library's aliases into every document and wrapped
ISC any()-sets in group nodes. The production corpus carries only the
document's own aliases and serialises any()-sets as {kind: "any",
"of": [...]} — consumers resolve dependency and library aliases at
runtime via the loader, matching interscript-ts's converter.
Also renders string_case/compose/decompose stage items as Funcall
rules in the NodeAdapter (4fdf15b never reached main): bare Symbols
raise 'Cannot serialise rule of type Symbol' in JsonIR.
…rpus Parser.parse leaked raw parslet nodes (char-array strings, identifier wrappers, field_name/field_value pairs, split test entries) where the documented contract — and the parser specs — promise plain strings and keyed hashes. DocumentBuilder compensated for every node form; a new Normalizer flattens the six scalar shapes at the parse boundary so the tree matches its documented shape and the builder's compensations stay harmless. Corpus gate: 289/289 parse+build. Interscript.maps globbed only the legacy .imp extension, so on ISC-only installs (the monorepo CI checkout) it returned an empty list — Detector#call then crashed on nil.first for the unpatterned cases. It now lists .isc alongside .imp (and .iml for libraries), ISC winning on name collisions. Also fixes a NameError in Codemod#tokenize_sub_rule (iteration variable renamed around it: line[_i + 1] -> line[i + 1]).
…braries Enabling ISC visibility in Interscript.maps exposed two follow-on breaks: - Interscript.parse dispatched every located map to the legacy .imp DSL (instance_eval of Ruby source) — a .isc file is a syntax error there. Paths ending in .isc now go to Compiler.parse_isc. - The four dependency libraries ported to libs/*.isc appeared in the systems listing, so the per-map suites tried to run them as systems. Files under a libs/ directory now classify as libraries. Also scopes spec/interscript_spec.rb's per-compiler sweep to the legacy .corpus it historically covered: sweeping the ISC corpus needs a per-map timeout harness first (at least one map hangs the interpreter). StandardRB: hash-literal brace spacing and chain indentation.
…etector Interscript.maps returning the ISC corpus turned every suite that was vacuously enumerating an empty list in CI into a failing one: map_name_and_metadata walks 578 examples of .imp-era naming and metadata rules the ISC corpus doesn't satisfy yet, and the detector's full-corpus walks crash on the first map the Ruby compiler can't compile. Suites written for the compiled-DSL pipeline now enumerate through legacy_maps (spec_helper) until each gains ISC-era conformance; Detector skips maps that fail to load or execute, mirroring the TypeScript detector.
The unpatterned cases walk every map on the load path. Against the ISC corpus that means compiling ~290 maps per compiler and at least one hangs the interpreter — the CI matrix ran 20+ minutes without completing. Gate them on the presence of the legacy corpus until the sweep gains a per-map timeout harness; the patterned case stays (a single-map walk).
… raw leaves
Two more ISC-only breaks, both invisible on .imp installs:
- NodeAdapter stored alias values as bare items. The interpreter
resolves aliases through AliasDef#data and feeds it to
Item.try_convert — a bare Any hands it Any#data, an Array, and every
library-dependent map dies with 'Wrong type Array' the moment
locate() resolves a library to libs/*.isc instead of a legacy .iml.
- Normalizer#normalize_meta_field assumed Hash entries; empty metadata
blocks carry a bare Parslet::Slice. scalar() also learns the {raw:
} string form the grammar emits for some metadata fields.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes 9 of the 77 pre-existing
rakefailures (6 parser, 2 detector, 1 codemod) with zero new failures.Parser.parse contract (
Normalizer)Parser.parseleaked raw parslet nodes where its own docstring — andspec/isc/parser_spec.rb— promise plain values:{string: [{char: "B"@8}, ...]}instead of"BGN-PCGN:..."{identifier: "main"@31}instead of"main"{field_name:, field_value:}pairs instead of keyed hashesDocumentBuilder compensated for every node form. A new
Isc::Normalizerflattens the six scalar shapes at the parse boundary so the returned tree matches the documented shape; the builder's tolerances stay and are still exercised by non-normalized leaves. Verified:spec/isc/parser_spec.rb8/8 (red on main), full corpus parse+build 289/289.Interscript.mapssees.iscmaps()globbed only the legacy.impextension, so on ISC-only installs (the monorepo CI checkout, where the maps path-gem ships.isc) it returned an empty list —Detector#callcrashed onnil.firstfor the unpatterned cases, and every suite that enumerates maps got nothing. It now lists.iscalongside.imp(.imlfor libraries), ISC winning on name collisions, withlibs/paths classified as libraries.Two follow-on breaks surfaced and are fixed here:
Interscript.parsedispatched every located map to the legacy.impDSL (instance_evalof Ruby source — a.iscfile is a syntax error there). Paths ending in.iscnow route toCompiler.parse_isc..imp-era naming/metadata rules; the per-compiler sweep) now enumerate throughlegacy_maps(spec_helper) — each gains ISC-era conformance in its own work order. At least one ISC map hangs the interpreter, so the sweep also needs a per-map timeout harness first.Detector#callskips maps that fail to load or execute, mirroring the TypeScript detector.Codemod
tokenize_sub_rulereferencedline[_i + 1]after the iteration variable was renamed toi— NameError on the first arrow-form rule.Remaining rake failures
The other 68 are two coherent clusters: the
Compiler::Pythonlegs (53) — the bridge still targets the pre-rewrite python contract (load_mapon a generated file; the rewritten python runtime resolves.isc/.impsources from configured load paths instead) — and 6reversibilityspecs behind the same PyCall environment. Both are follow-up work.