Repository navigation
Conversation
skhaz
approved these changes
Oct 3, 2026
wolfy-j
force-pushed
the
feature/lua-eval-host
branch
from
October 3, 2026 14:05
450cab9 to
5bdd02b
Compare
skhaz
approved these changes
Oct 3, 2026
wolfy-j
force-pushed
the
feature/lua-eval-host
branch
2 times, most recently
from
October 3, 2026 18:00
d0014b1 to
182e8d4
Compare
process.Start gains Admission{Factory, Meta}: a process definition that is
not a registry entry, such as a compiled eval program. Only Go callers can
set it. The manager forwards an admission only to hosts that implement
process.AdmissionHost (local hosts; remote hosts cannot run an in-memory
factory), and the host builds the process from the admitted factory instead
of the registry. A named admission never takes the spawn-or-signal shortcut:
a taken name is an error carrying the existing PID.
A process whose frame carries ProcessSendGrants may only address PIDs the runtime handed to it: itself, its parent (both seeded by whoever restricts it), processes it spawned, senders of messages it received (msg:from()) and registry lookup results. Addressing any other PID through process.send, terminate, cancel, monitor or link fails with "not allowed to <op>". Grants live in a non-inheritable frame key, so children start without them, and checks happen in the sending process only: nothing changes on the wire. Processes without grants behave exactly as before.
ChildSlots bounds how many child processes a process may have running at once. ChildSlotResolver, a frame resolver, reserves a slot as a frame attachment of each child process the limited process spawns: the slot is released when the child's frame is released at exit and rolled back when the spawn fails or targets a host that does not accept attachments. The limit lives in a non-inheritable frame key.
api/host defines the eval host boundary with v2's names and shapes: EvalPolicy, EvalCompileSpec, EvalSpawnSpec, EvalProgram and the eval admission errors. evalhost.Admitter implements it: - Compile validates source and policy with v2's limits and defaults, canonicalizes the policy and caches the compiled program keyed by source, policy and imported library sources (LRU, superseded on library change). - Spawn starts the program as a real process through process.Start with an admission. The process frame carries its own actor and a deny-by-default scope built from the policy alone: process.send per send mode (cap mode defers to send grants seeded with the parent; explicit allows only send_targets) and the spawn family only with the spawn command, bounded by max_children child slots. tick_budget and max_steps become the process's execution budgets. Link modes follow v2 (linked by default, monitor_only, detached only when allowed). - Modules are explicit; a module needs an allowed class and no process, storage or network class, except process, which is allowed by name. compile = "typed" type-checks strictly; jit and the upgrade command are rejected as unsupported. memory and mailbox limits are accepted and not enforced. The compiler gains opt-in ExplicitModules, AllowModules and Strict; the eval permission checks move to evalhost for both eval surfaces; process.pid() grants the process's own PID; process.ErrInvalidHost makes a start on a non-process host detectable, so eval can point at lua.eval.spawn_host.
The eval module exposes the eval host to Lua with v2's surface: eval.compile(source, opts?) -> Program, eval.spawn(source_or_program, opts?) -> pid, eval.evict(program), Program:spawn/evict and tostring(Program). Options and their errors match v2 (closed option records, nested policy, flat or nested limits, link modes, send modes). Calls check eval.compile, eval.spawn, eval.module, eval.import and eval.class. Boot creates the admitter next to the existing eval host, registers the child slot frame resolver and the eval module. New config keys: lua.eval.spawn_host (process host for evals from callers that are not on a process host, such as functions) and lua.eval.program_cache_size. eval_runner is unchanged and marked deprecated in its description and spec. App tests cover the new module end to end; the eval_runner suites pass unchanged.
Store.Close walked the cleanup list outside its lock while a cleanup's remover could relink the same nodes, a data race that could lose or corrupt cleanups. Close now detaches every cleanup under the lock and runs them afterwards; a remover called after Close finds its node detached.
Every execution has an ExecutionScope: a process for its lifetime, a function call for one invocation. A spawn with process.owned (and every spawn by an owned process) reserves a slot in the spawner's scope before the child starts, as a frame attachment: it is rolled back when the child is not admitted, rejected by hosts that cannot carry attachments, bound to the child's PID once it starts and released when the child completes. When an execution completes, its scope refuses new children and terminates the ones still running, through process.Manager, for every process type. Completion is signalled explicitly (process hosts via ctxapi.CompleteFrame, the function executor when the call returns), not by frame reclamation, which waits for frames forked from the execution. Child slots use the same completion signal, so a child that leaves a grandchild running no longer holds its parent's slot.
Send grants were not inherited, so a process spawned by a capability- restricted process ran unrestricted under the inherited eval scope. Grants are now inherited as an independent copy: a child can address what its spawner could, and later acquisitions stay with the process that made them. A restricted process may always address its parent.
- Evals are owned by the spawning execution unless detached (process.owned); a function call or an owned process cannot detach an eval, so a function never outlives its evals and an eval subtree stays contained. - The eval scope no longer grants process.context or process.exec: named spawn-or-signal delivered messages past the send mode, and exec bypassed max_children. - network selection requires network.select, as for process.with_options. - Bindings are copied at admission as bounded plain-data trees (depth and table limits, shared tables and mixed tables rejected) on both the Lua and Go paths. - Imports are compiled at admission and kept with the program: a spawn runs the library revision the program was admitted with. - Typed compilation declares bindings and imports to the type checker. - jit is not a compile mode. - Spawning a program refreshes its cache recency.
A detached eval has no owner to end it, so it now runs under a finite lifetime (lua.eval.detached_lifetime, default 1h): the Lua process reports it through process.ExecutionTimeoutProvider and the scheduler cancels the process at the deadline, waking it if it is blocked. Owned evals and all other Lua processes report no lifetime and are unchanged.
A frame fork copies Cloner values, but PropagatedPairs, which spawns use to carry inheritable values into a new process, passed the spawner's value itself. A capability-restricted process and its children then shared one send-grants set: acquisitions leaked between them and concurrent grants raced on the map.
process.upgrade had no authorization, so eval code with the process module could replace itself with any registry process and shed its admission policy, budgets and ownership. process.upgrade is now refused when the security scope explicitly denies process.upgrade (security.IsDenied), which leaves processes under existing policies unchanged, and the eval policy denies every action it does not allow instead of leaving it undefined.
An upgrade forked the new code's frame from the previous incarnation's frame and made it the process context. Values set on the process frame without inheritance (the execution scope, ownership, child limit, terminal port) were lost to the upgraded code, completion ran on the incarnation frame so owned children outlived the process, and the host's frame was never reclaimed. Frame keys can now be marked Execution: ctxapi.ContinueFrameContext forks the execution's frame and carries those values, which nothing else receives. The scheduler keeps the execution's root context, runs each incarnation in a continuation of it, releases the previous incarnation's frame on upgrade and the last one after completion, and reports completion on the root context. PID, spawn options, execution scope, the owned marker, child slots and the tty port are execution values; the owned child's release attachment stays with the frame that admitted it.
The admitter's binder ran each import chunk with PCall while creating the process, before its budget, deadline or ownership applied, so an import that never returned blocked eval.spawn. Imports are now deferred initializers that run in the eval's first step, in admission order, preemptible and counted against its tick budget and steps.
Admission accepted int8 and int16 bindings that payload conversion rejects at spawn, hashed bindings as JSON so 1 and 1.0 shared a cached program although math.type tells them apart, and copied a Go map or slice once per reference, so a value repeated thousands of times multiplied copying and hashing under the table limit. Bindings are now copied into the values a program receives (int64, float64, strings, booleans, []any, map[string]any), a table reachable twice is rejected as the Lua converter does, and the policy hash encodes values with type tags; a spawn request's policy is normalized the same way before it is matched against the admitted program.
When a named owned spawn reached an existing process, the host rolled the ownership registration back and returned the existing PID, and the manager bound it to the owner anyway; if the owner ended meanwhile the manager terminated that unrelated process. Bind now reports ErrOwnedNotStarted for a rolled-back registration and the spawn fails with the name-taken error, leaving the existing process alone.
…nt definition A no-argument process.upgrade reloads the process's current definition, but the permission check saw an empty resource, so a policy denying upgrades of that definition was bypassed by omitting it.
…owning them ContinueFrameContext refused execution values that are Closers, so upgrading a process attached to a terminal (tty port and binding) failed and the process was terminated. A continuation frame now refers to the execution's values and leaves closing them to the execution's frame. The execution's frame is sealed first, so a continuation is always a fork of it.
program_lifecycle waited for the first spawn's EXIT and discarded every other event, so when the second spawn exited first its EXIT was lost and the test timed out.
A frame fork copies the grants from the goroutine that forks it, such as an asynchronous function call, while the process keeps acquiring grants.
A process rejected at Init or lifecycle start released its frame without completing it, so owned children it had already spawned outlived it.
…reused A lease released or read after its store was closed and pooled acted on the store's next user.
A restricted process accumulated a grant for every process it spawned; an exited process is never addressable again, so its grant is released when the exit event is delivered.
Completion was reported on the execution's root frame, so after a cross-source upgrade lifecycle observers saw the original definition (the otel span was named after the source the process upgraded away from). The scheduler reports completion on the current incarnation's context again, and hosts complete and release the frame they created through ctxapi.ExecutionFrame, which a continuation frame records.
…eriting the starter's
…ed start cannot roll them back
… inheritable execution values
…arnation of an upgraded process
Admitted processes start from a clean frame so they inherit no caller authority, which also cut them off from the caller's trace. The admitter now adds ctxapi.PropagatorPairs: the caller's values that implement Propagator, the values designed to cross into another process, in their cross-process form.
A process started by a process with a child limit did not carry the limit, so a child of an eval could start any number of processes of its own. The limit now follows the tree: descendants draw on their ancestor's slots.
A detached eval is not owned, so what it spawned was not owned either and outlived the eval and its bounded lifetime. A detached eval now owns every process it starts.
The metrics module wrote to the host collector without consulting the security scope, so an eval admitted with it recorded arbitrary series though its policy allows no such action. Recording is now refused when the scope denies metrics.record; scopes without an opinion are unaffected.
…n of privileged modules Eval code reaches no dynamic loading, debug or host access, cannot alter tables shared across processes or leak state into other evals, finds no imports or bindings changed by another run, keeps explicit send grants against lookups, starts children that stay under its policy though their entries configure allow-all, and still ends with its owner when it traps links and retries upgrades.
process.registry.lookup resolved any registered name for an eval and granted it the PID, so a capability-mode eval could discover and message any named process. The lookup now passes the security scope, which denies it for evals unless the policy allows the lookup command. Processes whose scope has no opinion are unaffected.
…f running evals Spawning a compiled program with a method option ignored it and ran the compiled method. The spawn method now overrides it. The program cache evicted by recency alone, so a program whose evals still ran could be dropped and its handle fail on the next spawn. Running evals now pin their program and capacity trimming skips pinned programs. Evicting a program that is not cached reports ErrEvalProgramNotFound, and Evict returns only an error.
Commands are command IDs: numeric IDs and the process control command names parse, spawn and lookup are admitted, and every other command, including a granted upgrade, fails admission with the plain eval policy unsupported error. compile="jit" parses and fails admission with that error too. Limits require integer values; a float such as 2.0 is rejected.
…to end The test app configures a 2s lua.eval.detached_lifetime; a detached eval spinning under the largest step limits ends when it elapses.
wolfy-j
force-pushed
the
feature/lua-eval-host
branch
from
October 3, 2026 22:08
182e8d4 to
15e6795
Compare
Context keys gain a Process flag independent of Inherit. The scheduler upgrade path copies every process pair (pid, lifecycle options, terminal port) onto the replacement frame instead of special-casing the pid.
skhaz
approved these changes
Oct 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reason for This PR
v1's
eval_runnerruns dynamic Lua synchronously inside the caller, with the caller's authority (bounded only by a step limit and cancellation). This ports v2's eval model: dynamic code (for example model-written code) runs as a supervised process admitted under an explicit policy, with execution limits, capability-restricted messaging and a lifetime bounded by its owner.eval_runnerkeeps its current surface and is deprecated. Stacked on #893; shares the upgrade continuation foundation with #909.Description of Changes
process.Start.Admissionstarts a process from an in-memory factory. An admitted process starts from a clean frame: it inherits nothing from its caller except trace context (ctxapi.PropagatorPairs).ExecutionScope. Spawns withprocess.owned, and every spawn of an owned process, are owned and terminated when the owner completes, however it ends. Monitor-only evals are owned too; function calls and owned processes cannot spawn detached evals; detached evals own what they spawn and run underlua.eval.detached_lifetime(default 1h). These are stricter than v2. A child terminated because its owner ended completes withprocess.ErrOwnerEnded(explicit terminations and deadlines keep their errors); the owner's EXIT is not ordered after its children's.Execution(PID, spawn options, scope, ownership, child limit, send grants, tty port) are shared by every incarnation throughctxapi.ContinueFrameContext; hosts release the frame they created viactxapi.ExecutionFrame.lookup) names it looked up; forged PIDs fail as not allowed. Unlike v2, PIDs passed as input are not granted (v1 PIDs are strings).evalmodule (v2 surface): compile/spawn/evict, program cache with pinning of running programs, policy hashing with typed bindings (plain data only; v2 also admits bytes and records), imports run as deferred initializers under the eval's budget, method overrides, v2 command IDs (pluslookup),tick_budget/max_steps/max_children(the child limit covers the whole spawned tree).memory_limit_bytes,heap_reserve_bytesandmailbox_capacityare accepted and not enforced.process.upgradeandprocess.registry.lookuphonour explicit denials (security.IsDenied);metricswrites are gated for evals.Clonervalues are copied across spawns; a failed owner start left owned children running; an owned spawn could adopt an existing named process.Tests
api/context,api/process,api/security,system/process,system/scheduler/actor,runtime/lua/evalhost,runtime/lua/modules/eval,runtime/lua/modules/process,service/host.app.test.eval_module(sandbox surface, forged PIDs, registry lookup, child authority, descendant limits, detached lifetime with a spinning eval, upgrade denial, lifetime resistance) alongside spawn, program lifecycle, send capability, limit, admission and function-lifetime tests.-race; the native application suite passes all 411 tests (environment-dependent suites retain their existing skips); full repository lint reports zero issues.Known limits: memory is not bounded in v1 (a single eval can exhaust host memory), and Lua inside sort/gsub callbacks is bounded only by termination and the lifetime, not by
max_steps.Landing constraints from integration review
#887 and #893 pin divergent go-lua branches. Composing them with the current #893 dependency does not compile (
ast.HasTypeSyntaxis absent). Before landing this eval/preemption stack together with argument validation, it needs a go-lua revision containing both features and tests that validate the actual selected method after deferred bootstrap, before its body runs. Green individual-branch CI does not establish that combined compatibility.The memory and mailbox settings above remain unenforced compatibility fields; this is not an OS-isolated or memory-safe sandbox for hostile code. Tick budgets also do not hard-preempt arbitrary Go-native builtins.