Repository navigation
[PAC-11] host-install: CI for the bare-metal host packaging contract - #596
Open
physics515 wants to merge 21 commits into
Open
physics515 wants to merge 21 commits into
physics515 wants to merge 21 commits into
Conversation
Implement a dedicated GitHub Actions workflow (host-install.yml) that
proves the host packaging contract on every pull request touching
packaging/ scripts/packaging/ or weft-server/, on every push to main,
nightly at 10:00 UTC, and on workflow_dispatch.
CI runs:
- build: cargo build --release -p weft-server --locked (up to 45 min cold)
- test: scripts/packaging/test-host-install.sh, which runs 7 gates
(manifest match, ExecStart rewrite, operator-env preservation,
env-var registration, invented/Test-tier refusal, systemd-analyze
and systemd-sysusers/tmpfiles verification, transient user-scope
with hardening properties from the installed unit)
- post-check: fail the job if gates 6 or 7 skipped on ubuntu-latest
(systemd-analyze not installed, or no reachable user manager),
because the runner has both — a skip means the test is silently
not guarding against unit drift
Pattern follows container.yml conventions:
- pinned checkout@v7.0.1 SHA
- pinned dtolnay/rust-toolchain SHA
- permissions: contents: read
- concurrency group with cancel-in-progress on PR
- PR path filter scoped to packaging-relevant changes
- periodic cron with reasoning for the offset
Resolves PAC-11: systemd unit, user and env files, and host-install CI.
Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
…op var, quote rc) Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
…ng packaging scripts Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
…t-install.sh caught the missing bit on CI's first run Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
The manifest gate compared the default-prefix --list output against the tree installed at a non-default --prefix, so every dst was absent. List the manifest with the same --prefix as the install; this also keeps the ExecStart-is-in-the- manifest gate meaningful. Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
install-host.sh prints "left $DESTDIR$OPERATOR_ENV alone", which with --destdir includes the scratch root; the gate grepped the bare prefix path and never matched. Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
PAC-5b strict validation (settings.rs value_errors_for) refuses an empty value for every positive/non-negative integer name, so gate 4's contract of exporting all shipped names with no values now dies on the first integer name before the address refusal the gate hunts for. Export the defaults the template actually ships instead — with machine-pointing and auth-enabling defaults redirected into the scratch tree — and pin WEFT_SERVER_ADDR last so the probe address always wins. A NAME=1 fixture keeps gates 5 honest under the same contract. Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
The shipped template documents defaults in two shapes: a commented literal value, and an empty comment whose default lives in prose. Since PAC-5b an empty value on an integer name is a refusal, not a default, so one probe exports the literal defaults (redirected into the scratch tree) and expects the probe's own dead address to be the only refusal named; the other gives every documented name an impossible value and forbids the unknown-name refusal, which is the registry-coverage check itself. Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
With WEFT_AUTH=keys the config stage parses WEFT_KEYS_FILE before it reaches the address, so an empty scratch file ended the probe with a keys-document refusal naming the file instead of the address. auth.rs accepts a versioned document holding at least one key; that is now what the gate writes. Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
127.0.0.1:0 is an address: the config accepted the now-valid keys document, printed the startup banner, and bound an ephemeral port, so the probe served until the runner cancelled it 45 minutes in. A host with no port is refused as invalid configuration naming WEFT_SERVER_ADDR, before anything binds. Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
…something host-install gate 4 installs the unit and its env file, runs the real binary, and recognises one refusal: the address the probe pointed at. It greps for WEFT_SERVER_ADDR in the output, but bind_addr() returned std's bare AddrParseError, so the refusal read "invalid socket address syntax" and named nothing — the gate failed on ubuntu-latest (run 42237464718) for the server's wording, not for the packaging contract it exists to prove. WEFT_SERVER_ADDR and WEFT_ADMIN_ADDR now format the refusal the way flight::flight_addr and the strict-config warnings already do — name, quoted value, reason, an example to copy — so the gate's grep proves the installed env file is the configuration the process read. Exit status stays 78. Pinned twice: a unit test on the formatter, and an end-to-end test that runs the binary and asserts the name and the quoted value on its stderr. Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
… gate Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
Signed-off-by: Justin Icenhour <creative515.justin@gmail.com> # Conflicts: # CHANGELOG.md
Gate 5 exists to prove gate 4 can fail, and it picked the name to probe with `grep -B3 'tier: Tier::Test' settings.rs`. Every VarSpec is one line, so the three context lines above each match also return the entry above the Test entry — the first name out was OTEL_EXPORTER_OTLP_ENDPOINT, which the registry marks Stable and the server is right to accept. On ubuntu-latest the gate read that as a valid variable "refused for some reason other than its tier", with the OTLP TLS notice as the proof, while the server had behaved correctly and strict configuration had been refusing real harness channels the whole time. The name now comes off the matched line itself, so the probe carries WEFT_BENCH_BACKUP_DIR and greps for the refusal the server really prints for a Test-tier channel. Gate 5 is the only gate that had to be wrong for the roll to be red — which is the result it is supposed to be able to produce. Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
…its two causes
Gates 6 and 7 had never executed anywhere: on the host this script was written on they
are skips, and on ubuntu-latest the roll stopped at gate 4 and then gate 5 before
reaching them. Gate 6's first run on a runner is its first run ever, and it failed in
32 seconds with "Unit weft-server.service not found." for a unit gate 1 had just
confirmed on disk.
`--root` resolves an absolute path against the root and a slash-bearing relative path
against the current directory; the gate passed "${PREFIX#/}/lib/systemd/system/...",
i.e. the second kind. Keep the leading slash and name the paths in the failure, and
assert the file exists first so a genuinely missing unit and an unresolvable argument
can never again produce the same message — a gate that cannot tell those apart is a
gate that reports packaging defects as packaging defects only by luck.
Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
Signed-off-by: Justin Icenhour <creative515.justin@gmail.com> # Conflicts: # DECISIONS.md
…fix put it `systemd-analyze verify --root R` resolves a unit by name through R's search paths (R/usr/lib/systemd/system, R/etc/systemd/system). It does not read a unit from a path. Gate 6 passed an absolute path inside the scratch root, so ubuntu-latest answered "Unit weft-server.service not found." for a unit that was plainly installed at /opt/weftdb/lib/systemd/system/weft-server.service — a prefix install is in none of the search paths, and the gate could not tell a broken unit from an unreadable one. Gate 6 now stages the installed bytes into $DEST/usr/lib/systemd/system, proves with `cmp` that the staged copy is the installed file, and verifies the bare name under --root. Same unit, same bytes, one place systemd can actually see. Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
The staging fix worked: gate 6 now reaches verify, and the runner answered weft-server.service: Failed to create weft-server.service/start: Unit sysinit.target not found. systemd injects Requires=sysinit.target into every service and verify builds the real transaction, so verifying a partial tree on systemd 255 fails on a target no install prefix may carry. This host runs systemd 262 and returns 0 for the same bytes, so requiring exit 0 would pass here and fail there for a reason WeftDB does not control. Filter exactly the two essential-target lines, fail on anything else, and name the allowance in the failure text. SYSTEMD_RELAX_ESSENTIALS is not used because no such string exists in this host's systemd-analyze, libsystemd-shared or libsystemd-core, and an environment variable whose effect cannot be observed here would change the gate's meaning on some runners without proving anything on any of them. Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
Gate 6 died between gate 5's "ok" and "exit 1" with no message, because the filter was `grep -v`: grep exits 1 when it removes every line, that is the passing case, and the script runs under `set -euo pipefail`, so the assignment killed the script before the gate could report anything. Delete the two lines with sed, whose status does not depend on whether it deleted anything, and say why in the comment: a reader who "simplifies" this back to grep -v reproduces a failure that cannot be read. Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
… as a variable The [Service] extraction failed to parse on the runner, so the hardening gate read zero properties and failed. gawk accepts a variable named `in` and Ubuntu's mawk treats it as the reserved word it is: awk: cmd. line:1: ^ syntax error. 21 properties survive the drops, so the gate has plenty of margin once the program parses. Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
…ive it a state directory it can keep Two things made gate 7 fail with an empty log: PrivateTmp=yes hides the scratch tree under $TMPDIR and ProtectHome=yes masks /run/user, so the sandbox had nowhere to write; and a systemd-run child is a service, so its output goes to the journal and never to the file the gate polled. Bind the scratch state directory in and read the journal, which is what an operator reads. Signed-off-by: Justin Icenhour <creative515.justin@gmail.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The host-install contract (systemd unit, sysusers/tmpfiles config, env
template, operator-env preservation, ExecStart rewrite) had no CI gate.
A broken unit or a script regression could reach the release without
any automated signal.
What changed
New workflow
.github/workflows/host-install.yml(dedicated,path-filtered, following the same conventions as
container.yml):Triggers: PR on packaging-relevant paths, push to main, nightly
at 10:00 UTC (offset from CI at 07:00 UTC and container at 09:00 UTC
to avoid runner contention), and manual
workflow_dispatch.Build:
cargo build --release -p weft-server --locked(cold: ~30 min; reuses
Swatinem/rust-cache).Test:
test-host-install.sh— 7 gates:--listoutput)--prefixthe installed files
the real server and stops it with SIGTERM
Post-check: fails the job if gates 6 or 7
SKIPPEDonubuntu-latest, because the runner has both systemd and a reachable
user manager after
loginctl enable-linger.Verification done (local, without cargo build)
bash -n scripts/packaging/test-host-install.sh→ syntax OK (exit 0)python3 -c "import yaml; yaml.safe_load(open('.github/workflows/host-install.yml'))"→ YAML parsablecontainer.ymlconventions: pinned SHA actions,permissions: contents: read, concurrency group,PR path filter, nightly cron, workflow_dispatch — all consistent
actionlintnot available on this machine; verified manuallyRisk
The YAML is not byte-identical to a run that completed. The cargo build
step (~30 min cold) has not been executed locally; the workflow's
behaviour depends on the test script accepting the
--binargument andon the runner having the same systemd userspace tools as ubuntu-latest
(shipped since systemd 247). No new dependencies or shared infrastructure changes.
Closes PAC-11.