Repository navigation
Conversation
A Lua actor in a long loop held its scheduler worker until it blocked or finished, starving every other actor on that worker. Lua actors now run each step under a VM tick budget (go-lua SetTickBudget) and are suspended at a safepoint when it is used up. - process.Preemptible (EnablePreemption) and StepPreempted. The actor scheduler enables preemption at admission and requeues a preempted process through the global FIFO behind other ready work, also after dispatching the same step's yields. Pool, Temporal and eval drivers do not enable it and are unchanged. - The Lua process spends one budget per step across all its coroutines; tasks that did not finish run in the next step. - Options match v2: tick_budget (0 = default 512, negative = unlimited) and max_steps (0 = unlimited, exceeding fails with process.ErrStepLimitExceeded), read from entry meta.options and overridden by process.with_options. Keys directly under meta are rejected; with_options validates values at the call site. - go-lua bumped to the preemption commit (wippyai/go-lua#60).
skhaz
approved these changes
Oct 2, 2026
Upgrade replacements are admitted for preemption like any other process, inherit the spawn lifecycle options of the actor, and continue its step count so max_steps bounds the actor over its whole lifetime. A preempted process goes straight back to its worker when nothing else is ready and wakes another worker only when one is parked.
…he tick budget Module chunks run inside the main task of the first step instead of Init, so they are scheduled and preempted like any other Lua code while the exported method cache keeps serving later executions. An exhausted budget ends the step without walking the run queue, and the preempted coroutine queues behind the waiting ones. A step rejected by max_steps releases the retention leases of the messages it was delivered.
The test closed its process explicitly and again through the helper's cleanup, putting one Process into the shared pool twice; a later test could then receive a Process whose state another test released.
A receive that parks on the step's last tick now sees the messages held back for it, so the process reports preempted instead of idle.
Library chunks no longer execute through PCall while the factory builds the state. A binder registers them with DeferInitializer; the entry bootstrap runs them as Lua frames of the main task in dependency order, so loops in library code are preempted and cannot block the spawner or an upgrade. Chunk environments resolve imports at lookup time. The bootstrap uses no guest-visible global.
SyncExecute initializes the process's library chunks, in order and with the same result handling, before it calls the main chunk.
A chunk environment decides which aliases already exist in the base when the chunk first looks up a global, after its dependencies have run, so a library's _G export under an alias name no longer overrides the import while helpers installed later still resolve.
Each chunk environment seals its set of import aliases against the globals as they are once every library has initialized, so a later sibling library's _G export cannot override an earlier library's import regardless of when that library first looked up a global.
…rnation On timeout Stop read each processor's cancel field while Submit could still be assigning it after publishing the processor, and while completion could be clearing it for pooling. It now cancels through the processor's atomically published signal reference, as Terminate does; the cancellation still wakes parked processes through the wake registered at submit.
…ed admission Completion owns closing a non-pooled process and the submitter owns a process Submit rejects. The worker closed the process before completing it on every failed upgrade, and Submit closed it on OnStart rejection while the caller closed it again; a Lua process returns itself to its pool on each Close, so the duplicate put could hand one process to two users.
Handlers received the pooled Processor as their receiver, and its CompleteYield read the current queue generation. A completion that outlived its actor and arrived after the slot was reused enqueued an event into, and woke, the new actor. Handlers now receive the queue's generation-bound YieldCompleter, and Processor no longer implements the receiver interface.
… state The registry-held initializer list kept every completed Fn, Done and OnInitialized callback, retaining chunk environments and their captures for the life of the state. A completed entry is cleared and the callbacks are released after they run; failed initializers stay registered for retry.
Pooled tasks truncated retBuf and resumeBuf without clearing, so the backing arrays kept the previous execution's Lua values reachable.
Reset truncated the overflow slice without clearing it, so the pooled output kept references to the previous step's commands.
Submit and CreateProcessor stored StateReady before setting the context, cancel function, queue generation and completer, and the processor pool built processors in the Ready state. A stale queue entry left from a pooled processor's previous incarnation could win Ready->Running and step the new process half-initialized, which the race detector reported as a write of the context racing the worker's read. Processors are built and pooled as Complete, and Ready is the last store before the processor is scheduled.
This was referenced Oct 3, 2026
The test waited for sync.Pool to hand the finished processor to a new process, which the race detector's randomized pool drops make nondeterministic. It now asserts that yield handlers receive the incarnation's generation-bound completer; the queue test covers that a stale completer is inert.
StepsUsed reported zero for actors without max_steps, so an upgrade into a definition that sets a limit restarted the lifetime count.
…mption a function
Document the Set flags of ExecutionBudgets and drop a fairness comment that contradicts guest preemption.
The registry helper wrapped a nil-safe lookup that attrs.Bag already provides; api/runtime/lua no longer imports api/registry.
A present meta.options of another shape registered with default budgets, and ExecutionBudgetsFromOptions dereferenced a nil interface.
Yield tags restart in the replacement, so a completion of the replaced code could resume an unrelated task. EventQueue.RetireYieldCompletions makes earlier completers inert and drops queued completions; messages and wakes keep the queue generation.
…sition A completion or message bound to an earlier incarnation could pass the generation check, then wake or flag the incarnation that reused the slot, costing it a spurious step. The check and the transition now run under one per-processor lock that slot reuse also takes.
wakeAny notified the first worker whose token was free, which could be the worker running the step that queued the work. A worker that then blocked in a long step left the work stranded while another worker stayed parked. Workers publish whether they are parked and only those are notified.
Submit published a processor to the routing maps before its admission ended, so Stop could cancel and complete it while Submit still finished initialization and published it Ready. Admissions are now registered under the stopping check and Stop waits for them before it acts.
…n release CreateProcessor did not register the context-cancel wake that Submit registers, so Terminate or a cancelled context left an idle pooled processor parked forever. ReleaseProcessor lowered the live count without telling a draining Stop, so Stop waited out its whole deadline after the last pooled processor was released.
Reject unknown meta.options fields on process entries (accepting the v2 process keys and default_host) and parse tick_budget and max_steps with the v2 numeric carriers and ranges, so max_steps spans uint64. Add fuzz targets for options parsing.
skhaz
approved these changes
Oct 4, 2026
Integrate current main without restoring early processor publication or double-closing failed upgrades. Observe the shutdown deadline during Init/OnStart, cancel unfinished admissions, and pair successful late starts with completion only after the callback returns. Retire yield completion data through the canonical discard hook and allocate yield bookkeeping only for external yields. Add ownership, deadline, failed-frame, late-rejection and CPU-only allocation regressions.
skhaz
approved these changes
Oct 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reason for This PR
A Lua actor in a CPU-bound loop holds its scheduler worker until it blocks or finishes, starving every other actor on that worker; the only remedy today is cancelling it. This ports v2's tick-budget preemption to v1. Depends on wippyai/go-lua#60 (go.mod pins its commit; bump to the release tag once it is merged).
Description of Changes
process.Preemptible,StepPreempted+StepOutput.Preempt(),process.ErrStepLimitExceeded,process.StepAccounted, andapi/runtime/luaexecution options (ExecutionBudgets,EntryExecutionBudgets,ExecutionBudgetsFromOptions).tick_budget(0 = default 512, negative = unlimited) andmax_steps(0 = unlimited) read from entrymeta.optionsand overridable byprocess.with_options{...}. Unknownmeta.optionskeys are rejected as in v2 (default_hostis also accepted because v1's process functions read it); numeric parsing matches v2. Differences from v2: one tick budget covers a whole scheduler step across the actor's coroutines (v2 resets it per resume);max_stepskeeps counting across upgrades so it caps an actor's lifetime (v2 starts a replacement fresh); entry options come from the entry being run, spawn options carry over upgrades.engine.DeferInitializer), so they are preemptible too; a failing module chunk or a missing method surfaces fromStep(after any preemption of the initialization) rather than fromInit(script parse errors still come fromInit).max_steps; it stops on termination/cancellation.main): a pooled processor could be stepped half-initialized by a stale queue entry;Stopraced with admission and with cancel fields; failed upgrades closed a process twice; late async completions could wake a reused processor; pooled processors ignored cancellation and delayedStop; pooled tasks, step outputs and completed initializers retained previous users' values; a declared import now wins over a library's_Gexport of the same name.Review corrections
Tests
max_steps, initializers, close ownership, stale completions, publication order, admission vsStop, wakes.system/scheduler/actor/chaos_test.go,runtime/lua/engine/chaos_test.go) checking exactly-once completion/close, no wrong-incarnation completions,max_steps, no stranded processes, counters and goroutines back to baseline; fuzz targets for option parsing.-race; full app suite passes; lint clean on changed packages.Integration with #887
Runtime #887 is now stacked on this PR and contains the verified handler-validation compatibility corrections. Companion go-lua #58 is stacked on #60. Existing PR branches now own the integration commits; redundant staging branches are removed.
Land this PR before #887. Combined integration was verified with race suites and full repository lint against the real dependency pin; updated #887 CI is rerunning. Nothing has been merged or released by this update.