Skip to content

Experimental threaded VM interrupt watchdog - #90

Draft
HaroldCindy wants to merge 4 commits into
mainfrom
harold/threaded_watchdog
Draft

Experimental threaded VM interrupt watchdog#90
HaroldCindy wants to merge 4 commits into
mainfrom
harold/threaded_watchdog

Conversation

@HaroldCindy

@HaroldCindy HaroldCindy commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

May not even go with this implementation, simply an experiment in deferring setting up an interrupt handler until we know the deadline has passed, in order to keep the luau_execute loop hot. If it doesn't improve things, we'll ditch most of the work except for moving GC work out of the execution loop.

NB: This is currently a rough PoC mostly written by Claude, so don't mind the noxious comment spam, the actual impl would be cleaned up.

@HaroldCindy HaroldCindy changed the title Initial threaded watchdog Initial threaded VM interrupt watchdog Aug 30, 2026
@HaroldCindy HaroldCindy changed the title Initial threaded VM interrupt watchdog Experimental threaded VM interrupt watchdog Aug 30, 2026
@HaroldCindy

HaroldCindy commented Aug 31, 2026

Copy link
Copy Markdown
Contributor Author

Hmmm, compelling results! I think the bottom result may still be slightly faster than it should be with a properly-functioning scheduler, the watchdog thread currently spins in a loop with 50us granularity when it should probably just wait on a condition of either quanta reached or watchdog disarmed. That would also remove the problem of the kernel having to switch to the watchdog thread just for it to realize it has no work to do when scripts aren't even running.

Merits some investigation, but for now, we'd probably just want to have the threaded watchdog flagged off except in dev. Need to do some research about how quickly we can expect the kernel to switch over to our watchdog thread, particularly under extreme contention.

# With a `lua_Callbacks->interrupt` always set, constantly checking if we've run out of time within the VM itself
$ ./bin/slua-harness --sync-arming ../mandelbench.lua 
checksum 109250 over 1000 sweeps of 64x64
Runtime: 12.828395, Accum. Sleep: 0.000000, Time Slices: 64001

# With just the threaded watchdog timer, we do _okay_, but the watchdog often fails to set
# the "must yield" flag on the executing script by the deadline. This can be seen through the
# scheduler punishing scripts that ran way too long by no fault of their own.
$ ./bin/slua-harness ../mandelbench.lua 
[INFO] ../mandelbench.lua: Punishing abusive script that took 0.003954 (charged 0.003954) on quanta of 0.000200
[INFO] ../mandelbench.lua: Punishing abusive script that took 0.004643 (charged 0.004643) on quanta of 0.000200
[INFO] ../mandelbench.lua: Punishing abusive script that took 0.003998 (charged 0.003998) on quanta of 0.000200
[INFO] ../mandelbench.lua: Punishing abusive script that took 0.006668 (charged 0.006668) on quanta of 0.000200
[INFO] ../mandelbench.lua: Punishing abusive script that took 0.004971 (charged 0.004971) on quanta of 0.000200
[INFO] ../mandelbench.lua: Punishing abusive script that took 0.003348 (charged 0.003348) on quanta of 0.000200
[INFO] ../mandelbench.lua: Punishing abusive script that took 0.003039 (charged 0.003039) on quanta of 0.000200
# ...
[INFO] ../mandelbench.lua: Punishing abusive script that took 0.004021 (charged 0.004021) on quanta of 0.000200
checksum 109250 over 1000 sweeps of 64x64
Runtime: 9.308527, Accum. Sleep: 0.437514, Time Slices: 27157

# With real-time scheduling on the watchdog thread so we try to pre-empt as soon as the timer is hit.
# Slower than above, but _correctly_ slower because it's not letting super-long timeslices sit around
# in `luau_execute`. Note the this requires a `setcap` on the executable to allow the watchdog to
# set its own thread priority on Linux.
$ ./bin/slua-harness --fflags=true ../mandelbench.lua 
[INFO] QuantaWatchdog: Watchdog thread scheduling class raised to SCHED_RR
checksum 109250 over 1000 sweeps of 64x64
Runtime: 9.655174, Accum. Sleep: 0.000000, Time Slices: 35936

@HaroldCindy

Copy link
Copy Markdown
Contributor Author

Funnily enough, ARM64's RDTSC equivalent for our purposes (reading the cntvct_el0 register, it has a 20mhz resolution) is so stupid cheap that the watchdog is more expensive than an interrupt there. Poor x86.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant