Conversation
`ensurePiWebRunning()` runs on every extension load — i.e. once per pi process, including the `pi --mode rpc` workers that pi-web itself spawns. When its health check failed, it called `startPiWeb()`, which ran `launchctl kickstart -k`. The `-k` flag stops the running instance first, and pi-web handles SIGTERM by exiting 0, so launchd recorded a *successful* exit and the kill was invisible in `last exit code`. That single kill briefly takes the port down, so every other pi process starting in that window fails its own check and issues its own kickstart. Each restart manufactures more failures: a self-sustained restart storm (observed here at ~6 exits/min, `runs = 582`, with 502s from `tailscale serve` making the phone PWA unreachable). - `startPiWeb()` no longer kills: if something is already bound to the port it is not "not running", so exit without touching the job; otherwise plain `launchctl start`. - `ensurePiWebRunning()` confirms with a second check before acting, because one fetch is not proof that a server other processes are using is dead. - Health-check timeout 1s -> 4s: a healthy server answers in ~3ms, but 401/403 already count as up, so the only way to fail is a timeout — exactly the transient state a loaded machine produces. `/pi-web restart` keeps `kickstart -k`: an explicit user request to restart should still kill. Refs ygncode#112
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
.pi/extensions/pi-web.tsstarts the server opportunistically on every extension load — once perpiprocess, including thepi --mode rpcworkers that pi-web itself spawns. When the health check failed,startPiWeb()ran:launchctl kickstart -k "gui/$(id -u)/com.pi-web"-kstops the running instance first. pi-web handlesSIGTERMand exits0, so launchd records a successful exit and the kill is invisible inlast exit code— it looks like a voluntary quit, not a kill.The kill briefly takes the port down, so every other
piprocess starting in that window fails its own health check and issues its ownkickstart -k. Each restart manufactures more failures, and because pi-web's own RPC workers load this extension, the server ends up being restarted by its own children.Observed on a macOS 15.8 / arm64 host, beta.36, ~200 sessions, load average 40–59:
with intermittent
502s fromtailscale serve, which is what made the phone PWA unreachable. Bursts came and went depending on how manypiprocesses were starting; overnight, with nothing starting, zero restarts.The confusing part for diagnosis: a manually started
pi-webis immune, becausekickstart -kaddresses the launchd job, not a hand-run process. That reliably sends you hunting through launchd environment,stdin,ThrottleIntervaland plist churn instead of here. Full write-up with A/B evidence in #112.Change
One file,
.pi/extensions/pi-web.ts:startPiWeb()can no longer kill. If something is already bound to the port it is not "not running", so it exits without touching the job; otherwise plainlaunchctl start(no-k).ensurePiWebRunning()confirms before acting. A second check 1.5 s later, because one failed fetch is not proof that a server other processes are using is dead./pi-web restartkeepskickstart -k. An explicit user request to restart should still kill; only the implicit, load-time path needed the kill removed.Verification
All markers asserted by
internal/app/extension_static_test.go(pi.registerCommand("pi-web"|"remote"|"refresh"),Usage: /pi-web …,launchctl,systemctl,import("qrcode")) are present.go test ./internal/app/could not run in my environment — it fails atweb/assets_embed.go:8: pattern all:dist: no matching files foundbecause the frontenddistisn't checked in; I verified those markers by inspection instead.A/B against the live launchd job on the reporting machine (
state = running, child pid 6326):runslaunchctl kickstart -k(beta.36 behaviour)The reporting machine has run this patch since it was written; no further restarts.
Not in this PR
Findings 2 and 3 from #112 are separate and untouched here: the
serveRuleConflictfalse alarm intailscaleServeRuleState(internal/app/tailscale.go:120, which looks for the port as a JSON key and so matchesTCP[port] == {"HTTPS": true}— zero strings — instead ofWeb["<host>:<port>"].Handlers[*].Proxy), andMain()returning normally afterSIGTERMso the shutdown cause is indistinguishable from a clean exit. Happy to send either as a follow-up if useful.