feat: Python bridge via ISC generation - #774
Merged
Merged
Conversation
The rewritten python runtime executes ISC sources from configured load paths; the bridge still wrote generated .py modules into the python package tree and called load_map on them — every Compiler::Python spec leg failed with 'map not found in load paths (none configured)'. In-memory documents now cross the boundary as generated ISC text: a new Isc::Generator serializes a Node::Document back to ISC source (the inverse of Parser + DocumentBuilder + NodeAdapter, pinned by round-trip behavior specs), the bridge writes it to a per-process temp dir registered once via add_load_path, and load_map compiles it. The generated-into-the-package-tree write is gone. Notes: - The engine cache is keyed by document identity, not name: synthetic spec documents reuse example-N names and a stale engine would serve the previous document; a reused name now evicts via unload_map. - sys.path is inserted at 0: an editable install of the python package otherwise shadows the monorepo checkout. - INTERSCRIPT_PYTHON_SRC overrides the python source path for out-of-monorepo setups. - spec_helper resets the $compiler global per example; dsl_stage_spec leaked Compiler::Python into every later file, silently rerouting suites that never opted in. With interscript-py#6 (to-position case functions, same-map stage runs): dsl_stage python legs 53 -> 18 failures, reversibility 6 -> 0, round_trip 9 -> 0.
PyCall builds against whichever libpython it finds, which is not the python that bare pip targets on the runners — the python compiler legs then fail every load_map with ModuleNotFoundError. Resolve the bound version and pip-install into that interpreter.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes 15 of the remaining rake failures (53 dsl_stage Python legs → 18, reversibility 6 → 0, round_trip 9 → 0) — requires interscript/interscript-py#6 to merge first (CI's monorepo bootstrap clones interscript-python main).
The break
The python runtime was rewritten to execute ISC sources from configured load paths. The bridge still targeted the pre-rewrite contract: it wrote generated
.pymodules into the python package tree and calledload_map(name)on them — everyCompiler::Pythonspec leg failed withmap 'example-N' not found in load paths (none configured), and the write violated the no-side-effects rule for library packages.The fix
Isc::Generator— serializes aNode::Documentback to ISC source (the inverse of Parser + DocumentBuilder + NodeAdapter). Pinned byspec/interscript/isc/generator_spec.rb: every construct round-trips with identical Interpreter behavior (sub with all four constraints,to upcase, parallel, any() sets/ranges, capture+ref, document aliases, multi-stage with same-map runs). Astral code points pass through as raw UTF-8 — surrogate-pair\uescapes are not re-joined by either parser.add_load_path, thenload_map. No more writes into the python tree. The engine cache is keyed by document identity with eviction on name reuse (synthetic example-N docs would otherwise serve stale engines).sys.path.insert(0, …)so an editable install can't shadow the monorepo checkout;INTERSCRIPT_PYTHON_SRCoverrides the path.$compilerresets per example; dsl_stage_spec leakedCompiler::Pythoninto every later file (reversibility's plain.calls were silently rerouted to the python runtime).The 18 that remain (python engine vocabulary, tracked separately)
capture groups in patterns, aliased captures,
non_word_boundary, local/remote alias resolution semantics, any() multi-version replacement (the Ruby any-of-outputs feature), parallel any ordering, title_case/separate with kwargs (no ISC syntax yet).Verification (local, live PyCall against the rewritten runtime)