Skip to content

signals: a signal to the tree is the payload's to answer, and snug is the only author of the run's death (closes #595) - #596

Merged
vyskocilm merged 4 commits into
mainfrom
sig/595-who-answers-the-signal
Sep 21, 2026
Merged

vyskocilm merged 4 commits into
mainfrom
sig/595-who-answers-the-signal

Conversation

@vyskocilm

@vyskocilm vyskocilm commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Closes #595.

Red-team round is DONE: no sandbox escape, no boundary crossing. Two findings, both answered on the branch — one fixed, one kept and pinned. Definition of done is green: make gate, make verify (6 passed, 0 skipped, 0 failed), make integration with SNUG_REQUIRE_SANDBOX=1 (sandbox 555.2s, signals 68.3s).

What the round held against

sigseal on both arms (launcher with INT/HUP SIG_IGN and USR1 + an RT signal blocked -> payload reads SigBlk 0, SigIgn 0); no SIG_IGN leak into a live podman engine; the relay cannot reach host pids (payload sees pids 1-5, host-range kill -> ESRCH); grace is single and bounded (second signal cuts at 0.21s, ignoring payload capped at 1.02s); a setsid+disown daemon spawned DURING the grace dies with the pidns collapse; localhost still refused from @net after pasta's Setpgid; unshare -Urpf inside the staged sandbox -> EPERM; terminalWillDeliver is evaluated pre-fork so a payload sharing the pty cannot influence it with tcsetpgrp.

Finding 1 — the relay reached the payload but not its foreground child (FIXED, 52008b5)

trap "echo TRAPPED; exit 7" TERM; sleep 30   +   kill -TERM <snug>, no tty
  before   trap never fired, burned the whole 1.018s, then swept
  after    trap fired at 11-12ms, 3/3

A POSIX shell defers a trap until its foreground child returns; signalling the shell and not the sleep leaves the handler unrun. The relay now names the sandbox's process group — what a terminal does. Safe because bwrap's init calls setsid() and LEADS that group, so the group id is the init's own pid, already pidfd-pinned. Both conditions checked, never assumed.

My own measurements missed it because every payload used sleep 0.05 in a loop — short enough for the shell to reach its trap inside the budget. The 4/4 below was true and proved less than it looked: the exact error this PR accuses #595's reproduction of making. TestAHandlerBehindALongBlockingChildStillRuns is the missing case and FAILS against the old relay (verified by disabling the group path).

This also flipped TestGroupDeliveredSIGINTTearsDownTheWholeSandboxWithoutForwarding, whose payload is that same blocking-child shape: the trap now fires and snug exits 7, which is what the stage commit's by-hand transcript expected before #105's guard flipped it as a side effect. Renamed TestAGroupDeliveredSIGINTLetsThePayloadAnswerItFirst; its header keeps the old name, assertions and false #13 justification verbatim.

Finding 2 — exit-code substitution (KEPT, pinned)

A payload that traps the signal and exits 0 makes a signalled run report 0, so CI gating on $? alone can be shown success for a run it force-stopped. Honest trade rather than a hole — nothing crosses the boundary and the payload already picks the code on every ordinary exit — but the new part is that "the operator killed me" can hide behind a zero. A harness that cares should treat "I sent a kill" as failure independently of $?. Named in grace's comment and pinned by TestASignalledRunCanReportThePayloadsOwnZero.

#595 asks the wrong question, measured

Its own reproduction does not reproduce. trap 'echo yes > f' INT is 10/10 on main with #101's nesting confirmed on (--dry-run TOPOLOGY: pid nesting depth 2). It measures whether the handler STARTED. Split apart — trap INT, write h-start, do N work, write h-done:

arm handler work h-start h-done
offline none 8/8 3/8
offline spin 100 (~1.5ms) 8/8 0/8
offline sleep 0.05 4/4 0/4
staged @net none 4/4 3/4
staged @net sleep 0.05 4/4 0/4
offline, snug SIGSTOPped first (control) none 4/4 4/4

Handler always ENTERS, never finishes past ~1.5ms of shell work, on BOTH arms — and staged @net has no intermediate namespace, so #101 is not the cause.

#13 is a STARTUP leak. SIGKILL of snug 1.0s into a running payload: survived 0/4 offline, 0/4 staged, heartbeat frozen at 0.4s and 1.4s after snug died, no Go code involved. teardown.go's own table: 0 leaks at 86-94ms, 8/8 at 110-160ms, ~206ms payload start. A grace gated on a named init cannot touch it.

The real question is who answers a signal delivered to the tree. It was decided by the scheduler and differed by topology, by host and by launcher.

Three independent authors of the staged tree's death

Nothing calls setpgid, so a terminal's ^C reaches every group member. One member at a time, @net:

SIGINT to the stage alone        run ends, snug exits 69
SIGINT to the outer bwrap alone  run ends, snug exits 255
SIGINT to the offline NP bwrap   IGNORED, payload keeps running, tree intact

The offline arm has ONE author because its bwrap is pid 1 of #101's namespace. So #101's nesting — which #595 blames — is what makes a grace possible, and the arm lacking it is the one that could not have one.

Signal state leaked into the payload, differently per arm

bash -c 'snug -p @net … &'      staged payload SigIgn 0x2   offline 0x0
python blocks USR1, execs snug  BOTH arms      SigBlk 0x200

SigIgn 0x2 is SIGINT, so trap … INT was a silent no-op; SigBlk 0x200 is SIGUSR1, so trap … USR1 never fired anywhere. execve resets CAUGHT handlers and preserves IGNORED ones and the mask — those two are the whole leak. The asymmetry was an accident of ordering: Go installs no handler for a signal already SIG_IGN at startup and os/exec only resets what it handles, so the offline arm escaped because armTeardown precedes cmd.Start() while the stage is forked before the guard.

What is built

  • internal/sigseal — clears the mask, forces ignored dispositions to default, both exec verbs. Arch-independent (signal.Notify over the SIG_IGN, execve clears it): no rt_sigaction, no cgo, no arm64 struct layout to get wrong.
  • internal/nestproc — the procfs an intermediate pid namespace needs, one author for both verbs.
  • Stage stops being an author. MainServe catches and drops INT/TERM/HUP/QUIT (Notify, never Ignore — SIG_IGN inherits into every child). runOneSandbox forks bwrap into CLONE_NEWPID|CLONE_NEWNS; no CLONE_NEWUSER, the stage already holds CAP_SYS_ADMIN in U, so it costs no nesting level. info.InitPID translated once.
  • The grace. INT/TERM/HUP, once the init is named: 1s to handle and exit, payload's own exit code reported, second signal cuts. Relay ONLY where the terminal could not have delivered (ttydelivery.go) — on a shared session the tty already delivered and relaying would make one keypress read as the two that mean force-quit to compose, npm, pytest, vite. Budget is snug's number: no flag, no profile key.
  • pasta gets Setpgid so a group ^C cannot kill the network out from under a payload still inside its window.

Measured

Real pty, handler doing sleep 0.5, 4 trials/cell. Every cell was 0/4 before:

delivery offline staged @net
^C 4/4 4/4
kill -INT snug, no tty 4/4 4/4
kill -INT snug, snug is fg tty job 0/4 0/4

The no-tty row reached the payload on NO host before. These payloads all loop on sleep 0.05; see finding 1 for the long-blocking-child shape they did not cover, which is now its own test. The last row is the deliberate residual — snug cannot tell it from a ^C (os/signal carries no si_code), so it relays nothing rather than double-delivering every real keypress; stated in ttydelivery.go.

Negatives, both arms: payload ignoring the signal exits 130 at 1.01-1.06s with 0 leftovers; second ^C cuts at 0.27s; payload sees exactly one SIGINT per keypress; $? is the payload's own 7; SIGKILL at steady state still leaves nothing.

Abuse sentence

A hostile payload can keep running inside the sandbox, with what the policy already granted it, for up to 1s after the operator asked it to stop — and can make that second happen every time by ignoring the signal.

Bounded by the constant, by a second signal, and by confirmTeardown running afterwards unchanged. No new path, no new visibility, no host reach.

Two hazards found in self-review, fixed here

  • The init is pidfd-pinned. The relay targets the init's CHILDREN with a CATCHABLE signal, so a reaped-and-recycled init pid would have snug SIGTERMing a stranger's children. A pidfd holds the kernel's struct pid, so the number cannot be reused. Named regression test reads the pin back out of /proc/self/fdinfo.
  • The payload is COUNTED even where it is not relayed to, or a terminal-delivered signal spent the whole budget on a container run whose payload was still PARKED on --block-fd and never forked.

What this forced

#594's container stop now runs on the signalled path, because a payload exiting inside its grace exits NORMALLY and st.Wait returns an ordinary status. Holding ^C on a container run can cost both budgets, ~2s. Two tests encoded the old guarantee; renamed and re-conditioned to assert the surviving negative (payload ignores -> no stop).

A test pinned the old behaviour with a FALSE reason. TestGroupDeliveredSIGINTTearsDownTheWholeSandboxWithoutForwarding said "this is the correct, safety-motivated behaviour to pin — a future change that made snug wait for the payload's own handler would reopen that window", citing #13. Refuted above. History: the stage commit's by-hand transcript (2026-08-13) expected trap … exit 7 to fire and snug to exit 7; #105's guard (2026-08-17) flipped it as a side effect; #592 (2026-09-20) turned the stale transcript into a test with the wrong reason. Nobody decided the current behaviour. The transcript's expectation is restored and now asserted.

Prose corrected where it was measurably false

  • exec.go's unconditional "Ctrl-C reaches the payload" — depends on --new-session, which policy.NewSession() emits on two independent conditions.
  • INDEX.md's "~40 ms before bwrap arms --die-with-parent" — teardown.go already says in capitals that number is wrong — and its "Ctrl-C reaches every stage", where reaching the stage is what killed it.
  • dryrun.go's claim that confirmTeardown SIGKILLs the stage on a catchable signal — on a pty the stage died of the tty's own SIGINT first.
  • stage.Wait's "check dmesg for an OOM kill" — the stage no longer exits on its own.

Tests

internal/sigseal with its unsealed arm as the positive control; test/integration/payloadgrace_test.go (5 tests — 4 fail against a build with grace() neutered, and the two upper bounds say in their own comments that they cannot stand alone); the pidfd-pin regression; and scripts/0040 for the real-pty ^C path CI structurally cannot run (with no terminal every Go test takes the relay path instead).

One CI failure, diagnosed twice

TestSealClearsWhatExecveWouldOtherwiseCarry/plain failed on CI, passed locally: SigBlk SIGUSR1 bit = 0x0, want 0x200 — the unsealed CONTROL, the arm whose job is to prove the test can fail.

ubuntu-latest's /bin/sh is dash, and dash clears the inherited signal mask at startup. The child exec'd /bin/sh -c 'grep …/proc/self/status', so the program asked to report the mask rewrote it first. One ubuntu:24.04 container, same binary, same blocked SIGUSR1: dash SigBlk 0, bash 0x200, grep with no shell 0x200, cat 0x200. SigIgn 0x2 survived all four — a disposition is not a mask, which is why it looked selective. Reporter is now cat /proc/self/status, parsed in Go; control both ways in that container, the old binary failing with CI's line byte for byte.

1e5866e diagnosed the same failure as a per-thread race and added runtime.LockOSThread(); the failure came back unchanged at 52008b5 with the lock in place. Lock stays (Seal() has always locked), its false cause comment does not. Product unaffected either way — a shell clearing MORE than Seal() cannot make a payload inherit anything.

Performance, attributed rather than assumed

A/B on the nesting alone, same two engine tests, -count=1: 17.941s disabled vs 17.748s enabled — it costs nothing. The sandbox suite's wall-clock spread is podman-tier variance; it once hit the 8m -timeout, which is the sizing the Makefile already documents as wrong for a host where the engine tier runs. Not changed here.

🤖 Generated with Claude Code

vyskocilm and others added 4 commits September 21, 2026 20:01
… the only author of the run's death (closes #595)

#595 asked whether a payload's own handler should get to finish, and framed it
as tension with #13. Both halves of the framing are wrong, measured.

#595's OWN REPRODUCTION DOES NOT REPRODUCE. `trap 'echo yes > f' INT` is 10/10
on main with #101's nesting confirmed on. It measures whether the handler
STARTED. Split apart (write h-start, do N work, write h-done): handler enters
8/8 and finishes 0/8 past ~1.5ms of shell work, on BOTH arms — staged @net,
which has no intermediate namespace, measures the same. So #101 is not the
cause.

#13 IS A STARTUP LEAK. SIGKILL of snug 1.0s into a running payload leaves
nothing: survived 0/4 offline, 0/4 staged, heartbeat frozen at 0.4s and 1.4s
after snug died, no Go code involved. teardown.go's own table: 0 leaks at
86-94ms, 8/8 at 110-160ms, ~206ms payload start. A grace gated on a named init
cannot touch it.

THE REAL QUESTION is who answers a signal delivered to the tree. It was decided
by the scheduler and differed by topology, by host and by launcher.

FOUR PARTS.

internal/sigseal (new), called from both exec verbs before bwrap. execve resets
CAUGHT handlers but preserves IGNORED ones and the thread's mask, and both
leaked: `bash -c 'snug -p @net … &'` gave its payload SigIgn 0x2 — SIGINT
ignored, so `trap … INT` was a silent no-op — while the offline arm did not,
and BOTH arms leaked a blocked SigBlk 0x200 so `trap … USR1` never fired
anywhere. The asymmetry was an accident of ordering: Go installs no handler for
a signal already SIG_IGN at startup and os/exec only resets what it handles, so
the offline arm escaped because armTeardown precedes cmd.Start() while the
stage is forked before the guard. Reset is arch-independent (Notify over the
SIG_IGN, execve clears it) — no rt_sigaction, no cgo.

STAGE STOPS BEING AN AUTHOR OF THE RUN'S DEATH. MainServe catches and drops
INT/TERM/HUP/QUIT (Notify, never Ignore — SIG_IGN would inherit into every
child), and runOneSandbox forks bwrap into CLONE_NEWPID|CLONE_NEWNS so bwrap is
pid 1 there and ignores what it has no handler for. No CLONE_NEWUSER: the stage
already holds CAP_SYS_ADMIN in U, so it costs no nesting level. Measured
before: SIGINT to the stage alone ended the run (snug exit 69), SIGINT to the
outer bwrap alone ended it (255), while the offline arm's nested bwrap ignored
the same signal. So the staged tree had three authors where the offline tree
has one. internal/nestproc is the shared procfs step, one author for both
verbs. info.InitPID is translated once — bwrap now reports pid 2 of the new
namespace.

THE GRACE. On INT/TERM/HUP, once the init is named, the payload gets 1s to
handle it and exit; its OWN exit code is then reported instead of 128+signal; a
second signal cuts the wait. Relay only where the terminal could not have
delivered (ttydelivery.go: --new-session, no ctty, or snug not the foreground
group) — on a shared session the tty already delivered and relaying would make
one keypress read as the two that mean force-quit to compose, npm, pytest,
vite. Budget is snug's number: no flag, no profile key.

pasta gets Setpgid so a group ^C cannot kill the network out from under a
payload still inside its window.

MEASURED, real pty, handler doing `sleep 0.5`, 4 trials/cell. Every cell was
0/4 before:

    ^C, offline                     4/4 finished
    ^C, staged @net                 4/4
    kill -INT snug, no tty, both    4/4   (reached the payload on NO host before)
    kill -INT snug, snug fg tty job 0/4   deliberate residual, stated

Negatives: payload ignoring the signal exits 130 at 1.01-1.06s with 0
leftovers; second ^C cuts at 0.27s; payload sees exactly one SIGINT per
keypress; $? is the payload's own 7; SIGKILL at steady state still leaves
nothing.

TWO HAZARDS FIXED IN REVIEW. The relay targets the init's CHILDREN with a
CATCHABLE signal, so a reaped-and-recycled init pid would have snug SIGTERMing
a stranger's children — the init is pidfd-pinned for the run, which stops the
kernel reusing the number. And the payload is COUNTED even where it is not
relayed to, or a terminal-delivered signal spent the whole budget on a
container run whose payload was still PARKED on --block-fd.

#594's CONTAINER STOP NOW RUNS ON THE SIGNALLED PATH, because a payload exiting
inside its grace exits NORMALLY and st.Wait returns an ordinary status. Holding
^C on a container run can cost both budgets, ~2s. Two tests encoded the old
guarantee and are renamed and re-conditioned to assert the surviving negative
(payload ignores -> no stop).

A TEST PINNED THE OLD BEHAVIOUR WITH A FALSE REASON.
TestGroupDeliveredSIGINTTearsDownTheWholeSandboxWithoutForwarding said "this is
the correct, safety-motivated behaviour to pin — a future change that made snug
wait for the payload's own handler would reopen that window", citing #13.
Refuted above. History: the stage commit's own by-hand transcript (2026-08-13)
expected `trap … exit 7` to fire and snug to exit 7; #105's guard (2026-08-17)
flipped it as a side effect; #592 (2026-09-20) turned the stale transcript into
a test with the wrong reason. Nobody decided the current behaviour. The
transcript's expectation is restored and now asserted.

PROSE CORRECTED WHERE IT WAS MEASURABLY FALSE: exec.go's unconditional "Ctrl-C
reaches the payload" (it depends on --new-session, which policy.NewSession()
emits on two independent conditions); INDEX.md's "~40 ms before bwrap arms
--die-with-parent" (teardown.go already says in capitals that number is wrong)
and its "Ctrl-C reaches every stage" (reaching the stage is what killed it);
dryrun.go's claim that confirmTeardown SIGKILLs the stage on a catchable signal
(on a pty the stage died of the tty's own SIGINT first); stage.Wait's "check
dmesg for an OOM kill" (the stage no longer exits on its own).

TESTS: internal/sigseal (with its unsealed arm as the positive control),
test/integration/payloadgrace_test.go (5 tests; 4 of them fail against a build
with grace() neutered — the two upper bounds say in their own comments that
they cannot stand alone), the pidfd-pin regression, and
scripts/0040 for the real-pty ^C path CI structurally cannot run.

Performance attributed rather than assumed: A/B on the nesting alone, same two
engine tests, -count=1, 17.941s disabled vs 17.748s enabled.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CI run 35636253285 failed the UNSEALED control: `plain: SigBlk SIGUSR1 bit =
0x0, want 0x200`. Passed locally every time.

A signal mask is per-THREAD and execve preserves the mask of the thread that
CALLS it. The test child set the mask and then exec'd without locking, so the
Go scheduler was free to move the goroutine in between and the child reported a
thread that had never blocked anything. One scheduler loses that race, another
wins it.

The bug is in the CONTROL, not in the thing under test: Seal() has locked first
since it was written, and that is why the sealed arm passed on CI while the
unsealed one did not. It is still the worse half to get wrong — a control that
silently reports "nothing was blocked" is a control that cannot fail, which is
the exact shape this test exists to keep out of the product.

DIAGNOSED BOTH WAYS rather than assumed, because the failure did not reproduce
locally on its own. Removing the lock and adding goroutine churn between the
mask and the exec reproduces CI's line exactly (`SigBlk SIGUSR1 bit = 0x0, want
0x200`, GOMAXPROCS=8). Putting the lock back with the SAME churn passes 10/10.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… makes (redteam round on 9da8978)

Red-team round found no escape and no boundary crossing. It found two things.

THE RELAY REACHED THE PAYLOAD BUT NOT ITS FOREGROUND CHILD. Fixed here, not
filed.

    trap "echo TRAPPED; exit 7" TERM; sleep 30
    kill -TERM <snug>, no tty

    before   trap never fired, run burned the whole 1.018s, then swept
    after    trap fired at 11-12ms, 3/3

A POSIX shell defers a trap until its foreground child returns, so signalling
the shell and not the `sleep` leaves the handler unrun inside the budget. A
terminal does not have this problem: it signals the whole foreground group.

WHY MY OWN MEASUREMENTS MISSED IT, which is the part worth keeping. Every
payload in the previous commit used `sleep 0.05` in a loop — short enough that
the shell returns and reaches the trap inside the budget. The 4/4 was true and
proved less than it looked: the exact error this branch accuses #595's own
reproduction of making. TestAHandlerBehindALongBlockingChildStillRuns is the
missing case, and it FAILS against the old relay (measured, by disabling the
group path).

THE FIX names the sandbox's process group. Safe because bwrap's init calls
setsid() and LEADS that group, so the group id is the init's own pid — already
pidfd-pinned, so it cannot be recycled. MEASURED, both arms:

    snug  pid=1076120 pgrp=1076120
      bwrap  pid=1076136 pgrp=1076120
        bwrap  pid=1076144 pgrp=1076144   <- the init leads this group
          sleep pid=1076149 pgrp=1076144

Both conditions are CHECKED, never assumed: the init must lead its own group,
and that group must not be snug's. Where either fails there is no
--new-session, which is exactly the case where the terminal already delivered
to everyone. Falls back to per-child pidfd signalling there.

EXIT-CODE SUBSTITUTION, the round's other finding, is KEPT and now stated. A
payload that traps and exits 0 makes a signalled run report 0, so CI gating on
$? alone can be shown success for a run it force-stopped. The payload already
picks the code on every ordinary exit and nothing crosses the boundary; what is
new is that "the operator killed me" can hide behind a zero. Named in grace's
comment and pinned by TestASignalledRunCanReportThePayloadsOwnZero.

TestGroupDeliveredSIGINTTearsDownTheWholeSandboxWithoutForwarding FLIPPED, and
its payload is why: `trap 'echo caught-sigint; exit 7' INT; sleep 30` is the
blocking-child shape, so the group relay makes the trap fire and snug exit 7.
That is what the stage commit's own by-hand transcript expected before #105's
guard flipped it as a side effect and #592 froze the stale version into a test.
Renamed TestAGroupDeliveredSIGINTLetsThePayloadAnswerItFirst; the header keeps
the old name, the old assertions and the false #13 justification verbatim, so
the next reader sees how it got pinned. "Nothing survives" is deliberately NOT
asserted there — orphan_test.go owns it across every signal and offset.

make gate, make verify (6 passed, 0 skipped, 0 failed) and make integration
(sandbox 555.2s, signals 68.3s) all green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ed mask

TestSealClearsWhatExecveWouldOtherwiseCarry/plain failed on CI and passed
locally: `SigBlk SIGUSR1 bit = 0x0, want 0x200`. The unsealed CONTROL, so the
arm whose whole job is to prove the test can fail.

ubuntu-latest's /bin/sh is dash, and dash CLEARS THE INHERITED SIGNAL MASK AT
STARTUP. The child exec'd `/bin/sh -c 'grep -E "^Sig(Blk|Ign):"
/proc/self/status'`, so the program asked to report the mask rewrote it first.
Measured in one ubuntu:24.04 container, same binary, same blocked SIGUSR1:

    dash -c 'grep …'   SigBlk 0000000000000000
    bash -c 'grep …'   SigBlk 0000000000000200
    grep, no shell     SigBlk 0000000000000200
    cat                SigBlk 0000000000000200

SigIgn 0x2 survived all four — a disposition is not a mask, which is why the
failure looked selective.

The reporter is now `cat /proc/self/status`, parsed in Go. Control in both
directions, same container: the 52008b5 binary fails there with CI's line byte
for byte (sigseal_test.go:120, same hex), this one passes.

1e5866e's diagnosis of the same failure was WRONG. It read it as a per-thread
race and added runtime.LockOSThread() to the child; the failure returned
unchanged at 52008b5 with the lock in place. The lock stays — Seal() has always
locked for that reason and the test should match it — but its comment claimed
the race was the cause, and that claim is deleted rather than left in code,
where prose says what is.

The product is unaffected in either reading: a shell clearing MORE than Seal()
does cannot make a payload inherit anything.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vyskocilm
vyskocilm merged commit eb13741 into main Sep 21, 2026
5 checks passed
@vyskocilm
vyskocilm deleted the sig/595-who-answers-the-signal branch September 21, 2026 19:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The payload's own signal handler loses its window to the faster teardown — problem statement, not a design

1 participant