Skip to content

innd: exit on SIGTERM and SIGHUP while waiting for activity - #376

Open
go-while wants to merge 2 commits into
InterNetNews:mainfrom
go-while:fix-sigterm-idle
Open

go-while wants to merge 2 commits into
InterNetNews:mainfrom
go-while:fix-sigterm-idle

Conversation

@go-while

@go-while go-while commented Oct 23, 2026 •

Copy link
Copy Markdown

Made with AI (Claude, via Claude Code) under my direction: found by the end-to-end tests from #375, then checked by a separate review and on real hosts. Please review it with that in mind.

Problem

innd does not shut down when it gets SIGTERM or SIGHUP (or SIGINT with -d) while it waits in select().
It only acts on the signal at the next connection or data, or when the timer interval runs out (ten minutes by default).

ctlinnd shutdown and rc.news stop are not affected.

Stopping innd by signal is: kill, docker stop, and systemd's fallback SIGTERM.

Docker and systemd then send SIGKILL after their stop timeout, so the active file and history are not flushed.

Reproduce: start innd -f, wait a moment, kill -TERM <pid>. innd keeps running until something connects.

Cause

catch_terminate() only sets GotTerminate. A signal that arrives during select() makes it return -1 with EINTR, and CHANreadloop() went straight back into select():

if (count < 0) {
    if (errno != EINTR) { ... }
    continue;                 /* back into select(), GotTerminate not checked */
}
STATUSmainloophook();
if (GotTerminate) { ... }

This order dates from 9b1b651 (5fa1209 in 2.5.4).
That commit moved the GotTerminate check below the EINTR test so that STATUSmainloophook() could no longer overwrite errno first.
Before it, GotTerminate was checked before count < 0.

Fix

Check GotTerminate right after select() returns, before looking at count.
Julien's errno fix is kept: nothing touches errno unless innd is exiting.
Behaviour on a busy server is unchanged.
The second commit only rewords the news.pod entry (2.7.5 section).

Testing

Unpatched and patched innd on Fedora 45, each in a scratch site:

case unpatched patched
idle, SIGTERM still running after 12 s, needed SIGKILL exits 0 after 1.0 s
idle, SIGHUP still running after 8 s exits 0 after 1.0 s
-d, idle, SIGINT still running after 8 s exits 0 after 1.0 s
3 open, idle peer connections, SIGTERM still running after 8 s exits 0 after 1.0 s
burst of IHAVE (about 1800 articles), SIGTERM exits 0 after 1.0 s exits 0 after 1.0 s

The 1.0 s is the sleep(1) in JustCleanup(). All shutdowns were clean ("exiting due to signal", no core dumps).

  • make test passes, and so does the build workflow on my fork.
  • The Go end-to-end tests from e2e Tests #375 (tests/e2e/signal_test.go) on the e2e branch) check idle and loaded shutdown.
  • They pass on Ubuntu 24.04 and Fedora 43, 44 and 45.

Related

A signal that arrives while innd is busy (for instance in a slow Perl filter) is blocked until the unmask right before select().
It then interrupts nothing, and innd waits again. That race is older than this bug.
A small follow-up fix with a test is ready on the fix-sigterm-busy branch;

I'll open it once this one is in, or now if you prefer both together.

go-while and others added 2 commits October 23, 2026 15:09
catch_terminate() only sets GotTerminate, and CHANreadloop() checked it
after the select() result: a signal that arrives while innd waits in
select() makes it return -1 with EINTR, and the loop went straight back
into select().  The signal was only acted on at the next connection or
when select() timed out (the remaining part of the timer, up to several
minutes), so kill or docker stop usually ended in a SIGKILL without the
active file and history being flushed.

Check GotTerminate right after select() returns, whatever its result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The bug is not limited to innd having no connection: it happens whenever
innd waits in select(), with idle connections open too, and also for
SIGINT with -d.  It was introduced by 9b1b651 (5fa1209 in 2.5.4),
which moved the GotTerminate check below the EINTR "continue" so that
STATUSmainloophook() could no longer overwrite errno before the EINTR
test.  The previous commit restores the check before "count < 0" and
keeps that errno fix: nothing touches errno unless innd exits.

Stopping innd with a signal is what kill does, and what docker and
systemd do (followed by SIGKILL after their stop timeout); stopping it
with ctlinnd shutdown, as rc.news stop does, was never affected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Julien-Elie Julien-Elie added bug Something isn't working C: innd Related to innd P: medium Medium priority labels Oct 23, 2026
@Julien-Elie Julien-Elie added this to the 2.7.5 milestone Oct 23, 2026
@go-while

Copy link
Copy Markdown
Author

A small real-world data point for this one, hit by accident today.

On one of our test machines (Ubuntu 26.04), two test runs of a build from current main (6843efc, so without this fix) were killed from outside: the test process died and left its innd -f running, idle, with no clients. An hour or two later I cleaned up:

$ ps -o pid,stat,etime,cmd -p 389720 702453
 389720 S  02:05:29 /opt/inn-e2e/bin/innd -f
 702453 S  01:10:43 /opt/inn-e2e/bin/innd -f
$ kill 389720 702453; sleep 3
$ kill -0 389720 && echo still running      # both still running
$ kill -9 389720 702453

Neither innd reacted to SIGTERM; both needed SIGKILL, so neither flushed the active file or history. That's the case this PR fixes: the signal is only acted on when the next connection, data or timer interval (10 minutes by default) wakes innd up. Here nothing woke them, and I gave up after 3 seconds rather than waiting for the timer. With the fix, innd exits right away on SIGTERM while idle (tested with the e2e test TestSIGTERMWhileIdle).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working C: innd Related to innd P: medium Medium priority

Development

Successfully merging this pull request may close these issues.

2 participants