Repository navigation
Conversation
skhaz
approved these changes
Oct 2, 2026
skhaz
approved these changes
Oct 3, 2026
Add a VM tick budget that suspends a running coroutine at a safepoint (backward jumps, loop iterations, calls) once exhausted. The resumer gets ResumePreempted and continues the thread by resuming it with no values. - SetTickBudget/TickBudget on LState, shared by all threads of a state; negative (default) disables preemption. - Preemption propagates through yieldable boundaries (opcode metamethods, pcall/xpcall via CallK, coroutine.resume/wrap) and is suppressed under Go callers that cannot suspend Lua (Call, PCall, Go library callbacks such as table.sort and string.gsub, resumes issued from Go functions). - Public Call/PCall count as non-yieldable; internal metamethod paths that handle yields call callR directly. Fixes two yield bugs the preemption paths exercise: - Opcode yield continuations were stored once per thread, so a yield through nested metamethod frames resumed the outer opcode with the inner result. Continuations are now per frame (callFrameExt + frame flag). - A tail-called Go function collapsed its caller after it returned, so a yield with nested frames above it merged the wrong frames. The caller is now collapsed before the call, and returns through stacked Go continuation frames complete each of them.
…e validation, skip results for preempted resumes
Every Go call looked up the frame extension map for a continuation and every Go return deleted from it once any frame had used an extension, and every Lua return called resumeGoFrames. A frameHasExt flag on the frame makes the lookups a bit test when the frame owns no extension (an entry left at the same index by an earlier frame is replaced, never reused), and an inlined check skips resumeGoFrames when returning to a Lua frame.
… completes handleYieldContinuation cleared the continuation fields but left the entry in LState.frameExt and the frameHasExt flag set, so every opcode that resumed after a yield or preemption left one extension per frame index behind until the thread was pooled or collected. The entry is now deleted when the continuation is consumed. Adds leak tests covering frame extension release, suspended-coroutine collectability, heap stability across coroutine cycles, hold teardown and counter restoration on error paths.
… holds resetLState left the hold links, the cancel function, the context and the frame slots of the previous user in place: popped frames kept their closures and Go functions reachable from the pool. Stacks gain Reset, which empties frame slots, and freed stack segments are cleared before they return to the segment pool.
…distinct extensions callFrame.Idx and the frameExt keys were int16, so frames past index 32767 turned negative and frames 65536 apart shared one extension entry, delivering one frame's continuation to another. Both are int32.
Upvalue.Close kept reg, so a closure that outlived its thread kept the thread's registry, and through it the state, frames and context, reachable.
…ates A coroutine.wrap child that raised an error panicked into the resumer without being killed, leaving it live, still attached to its resumer and still the current thread. It is now killed and detached, and the resumer made current, as for a failed coroutine.resume.
…rands stringConcat returned its parts slice to the pool with the concatenated strings still in its capacity and kept the builder's buffer, so old operands stayed reachable from the pools.
…inuation When a held child was torn down and its state reused before the owner resumed, the continuation read wrapped from the replacement user and reported the dead thread in the wrong form. The continuation context records the kind at suspension and compares the child by identity only.
Frame indices key per-frame continuations and are int16, so a stack deeper than 32768 frames gave frames negative or shared indices. Widening the index grew every call frame from 40 to 48 bytes for depths Wippy never configures (reference Lua caps calls at 20000). The index stays int16 and a CallStackSize above MaxCallStackSize is refused at state creation.
…op variable The closure captured cf by reference, keeping the dispatch loop's frame pointer addressable for every opcode. Passing it as an argument leaves the closure with only loop-invariant captures.
…'s options A pooled state kept its call stack regardless of the requested CallStackSize and MinimizeStackMemory, and a pooled thread also kept the registry's old growth limits. Pooled states and threads now share one constructor path that reuses a stack or registry only when it matches the options.
Resume and ResumeInto pushed the entry frame onto an unstarted thread before checking whether it could be resumed, leaving a frame on a dead thread. A closed thread dereferenced its released state. The rejection now runs first and reports a closed thread.
…nd the preempted yield state
…sleeping runSliced, runToCompletion and driveToCompletion drive a thread to completion the same way; they become resumeThrough and runToCompletion. Finalizer waits share waitCollected. The sprintfBody wrapper, a stray discard and the misnamed mustDoString go.
…budget from the local global in Call
… one After an error surfaced past a suspension, handleProtectedError wrote the two results but left the registry top and the caller's expected count alone, so a multi-result caller saw the discarded frames' values and a padded caller saw stale slots. The handler also swallowed its own failure. Results now follow the frame's NRet and the xpcall handler runs through invokeErrorHandler, the same path as the synchronous xpcall.
SetTickBudget raised through its receiver, which a Go callback may hold for a different state, pushing the error onto an idle stack that a later resume then read as a status.
…om Go Call counted entries into Lua execution, but host API calls that reach a metamethod (GetField, GetTable, SetField, Concat) enter through callR directly, so a metamethod could change the tick budget mid-execution.
…ng the Go caller with garbage
…ion and call-name lookups match a completed call
…owth in source fuzzing
…ch on context cancellation
…e; assert every field is cleared
Prepare the unsigned remaining iteration count once and enter the first iteration with the original integer index and its initial safepoint. Round and clamp floating limits for integer starts and steps, and reject zero integer steps before division. Leave float control loops on their existing advancement path. Update the count in place to avoid boxing it on every long-loop iteration. Return integer snapshots through the debugger and preserve integer counter writes. Cover integer boundaries, full unsigned counts, float controls, debugger access, and resumption with a one-tick budget. Add a numeric-for benchmark beyond the integer preload range.
Keep integer limits in ordinary registers and check unsigned remaining distances before advancing. Reuse existing integer limit values, restore direct debugger access, and cover loop entries with allocation regression tests.
The snapshot test asserted the counter value the overflow fix first stored in the limit slot. The slot holds the limit again, as in reference Lua.
wolfy-j
force-pushed
the
feature/tick-preemption
branch
from
November 10, 2026 03:00
397cc2b to
fa08172
Compare
…e the call site OP_TAILCALL moved a Go frame into its caller's slot before the function ran, so argument errors and positions read a stack without the tail call's call site: `return string.rep()` reported "bad argument #1 to main chunk" without a position. The Go frame now runs above its caller, as a non-tail call does, and the caller's slot is released when the function returns or suspends. The Go frame and the frames it keeps across a suspension move down into that slot, so the stack seen at every suspension is the one preemption relies on. removeFrame generalizes removeCallerFrame to frames below the top. TestSuspendedOpcodeFramePosition expects the message the stable line reports for a Go function tail-called from a metamethod.
Contributor
Author
|
Rebased onto
Verification at the tip: |
skhaz
approved these changes
Nov 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reason
Wippy v1 runs Lua actors on a fixed worker pool. A CPU-bound actor (
while true do end, long loops) holds its worker until it yields or its context is cancelled, starving every other actor on that worker. v2 solves this with VM tick-budget preemption; this ports the mechanism to go-lua. The runtime side is wippyai/runtime#893.Changes
LState.SetTickBudget(n)/TickBudget(): the number of ticks (backward jumps, loop iterations, calls) Lua may run before the running coroutine is suspended at a safepoint. The budget lives on the sharedGlobal; negative (default) disables preemption, so existing callers are unaffected. Changing it while Lua code is executing raises an error.Resume/ResumeIntoreturnResumePreempted; the thread is resumed with no values to continue.pcall/xpcall,coroutine.resume/coroutine.wrapat any depth. A coroutine held by a pendingcoroutine.resumecannot be resumed by anyone else.table.sortcomparators,string.gsubreplacement functions,__tostringinvoked from Go,xpcallhandlers, hostCall/PCall). These still observe context cancellation at every safepoint, so cancelling the state's context stops them. v2 behaves the same for sort/gsub callbacks.forloops never overflow (overflow-safe unsigned distance check, Lua 5.4 limit rounding and clamping), with no allocation on loop entry.attempt to yield across a C-call boundaryinstead of corrupting the caller.MaxCallStackSize(32768): frame indices identify per-frame continuations and fit in an int16 (callFramestays 40 bytes); a largerCallStackSizepanics at state creation.Fixes (several pre-existing on
main)__ltresumed incorrectly after suspension; protected calls that failed after a suspension lost their error or returned stale values; error positions after a suspended metamethod pointed at the wrong instruction.Tests
TestPreemptDifferentialSweep,FuzzPreemptGenerated,FuzzPreemptSourcecompare preempted against unpreempted runs (results with exact numeric type, errors, prints, yields, tables) over handwritten, generated, fixture and test-corpus programs; the full sweep (PREEMPT_DIFF=full) ran about 235k preempted executions with no divergence.Current landing-pass review
No go-lua release or runtime dependency bump is included in this landing batch.
Current-main integration review — 2026-10-31
Exact head
397cc2bb4e4dfbc527e4c9b213258687cb24cf71has twelve successful hosted checks. A fresh isolated merge against current main conflicts inbaselib.go,compile.go,state_pool.go,stringlib.goandvm.go. This is a semantic integration, including protected-call frames, goto/upvalue handling, pooled-state ownership, current native gsub callbacks and VM continuations. Neither side can be selected wholesale without risking fixes that have since shipped.The recorded Rodrigo approvals name
a0cd38afeb7d958ad5f9dc5806ca97d60cdb721eandc14848bbdb76c0ac3390fe527920021eb734d81d, before later substantial changes. The trial merge was aborted; no conflict markers or guessed production resolution were pushed. Preserve the passing exact-head evidence, but qualify the integrated revision with the complete differential/race/native regression gates before landing. Runtime #893 remains untouched. No new release or runtime dependency bump is included.