fix: NodeAdapter system-code identity + the ISC_SWEEP measurement gate - #778
Merged
Merged
Conversation
The per-example timeouts already bound the sweep (16,262 examples, 16k pass, 6 fail — all iso-mal-Mlym-Latn-15919-2001 on the Ruby compiler; the Interpreter passes the same map, so it is a codegen divergence, not a corpus gap). The gate keeps CI on the legacy scope while the conformance tail closes.
The adapter read isc_doc[:system_code] but DocumentBuilder emits :systemCode — every ISC-parsed document got name: nil, and the Ruby compiler (which registers compiled maps under Document#name) compiled every ISC map under the empty string. Transliterate then crashed with 'undefined method call for nil' — the same crash the detector's full-corpus walk hit. Interp-vs-compiled divergence on iso-mal-Mlym-Latn-15919-2001 is also gone for the direct paths; the remaining sweep failures there are a separate codegen bug (an object inspect-string interpolated into a lookahead character class).
any([boundary, ZWNJ, ZWJ]) leaked the boundary primitive as a raw Isc::Items::Primitive into Any's value; the Ruby compiler then baked the object's inspect string into the compiled character class character-by-character, and the rule's lookahead matched positions it must not. Set entries now convert through the alias/primitive paths.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two pieces of the sweep-enablement work (task: full-corpus conformance):
The fix
The NodeAdapter read
isc_doc[:system_code]but DocumentBuilder emits:systemCode— every ISC-parsed document gotname: nil. The Ruby compiler registers compiled maps underDocument#name, so every ISC map compiled under the empty string and everyCompiler::Rubytransliteration crashed withundefined method 'call' for nil. This was the detector walk crash, and it masked Ruby-compiler corpus coverage entirely.Pinned by a spec on the adapter (red on main: name was nil).
The measurement gate
ISC_SWEEP=1un-gates the corpus sweep inspec/interscript_spec.rb(per-example timeouts already bounded it). Measured with this fix:iso-mal-Mlym-Latn-15919-2001on the Ruby compiler: one lookahead character class is compiled from an object inspect-string (#<Interscript::Isc::Items::Primitive:0x…>interpolated character-by-character) instead of the ZWJ/ZWNJ codepoints, so the ം → m rule over-fires. Root cause located (compiler/ruby.rb Any-value construction); the fix is the next unit.The gate keeps CI on the legacy scope while that tail closes; the sweep exists to measure exactly this gap.