Repository navigation
composefs: Declare the backend stable - #15
cgwalters-bot wants to merge 34 commits into
Conversation
8d06d99 to
cdf4bf8
Compare
fc2efe2 to
1ee6333
Compare
|
Looks like this needs a rebase |
1ee6333 to
ddca99a
Compare
|
Rebased onto main (f62a0ed); new head ddca99a. bootc-dev#2483 hasn't merged, so its two commits are still carried here, rebased cleanly; bootc-dev#2483 itself needs the same rebase. The only conflict was with bootc-dev#2496 in the hide commit: I kept
Generated-by: https://github.com/cgwalters/#llms |
ddca99a to
b64e028
Compare
b64e028 to
cd88660
Compare
Upstream main moved and this no longer rebases cleanly; needs another look after the conflicts are resolved.
|
Not promoted: |
cd88660 to
7fbd35e
Compare
|
Rebased onto main (b06e0b0); new head 7fbd35e needs your re-approval, since resolving the conflicts changed content: the docs move now targets upstream's Generated-by: https://github.com/cgwalters/#llms |
cgwalters-bot
left a comment
cgwalters-bot
left a comment
There was a problem hiding this comment.
Review guide for head 7fbd35ecd1: where to look closely and what is safe to skim. It is advice from the bot's reviewer and doesn't replace reading the diff; the review app walks it.
Rebase of the approved cd88660 onto main b06e0b0 after upstream#2508 renamed chapters to bootc-*.7.md. The forge PR base is stale, so this diff also shows upstream commits; only the last three (548b19b, 9796fc7, 7fbd35e) are the PR. range-diff against cd88660 shows only path and link renames plus four real changes: redirects retargeted (and one added for the short-lived bootc-experimental-composefs.7.html), bootc-composefs-finalize-staged(8) moved from Experimental to Architecture in SUMMARY, the upstream soft-reboot paragraph in bootc-upgrades(7) relinked, and a new Limitations bullet for --soft-reboot=auto. The soft-reboot claim matches crates/lib/src/bootc_composefs/soft_reboot.rs: Auto returns Ok without rebooting even with --apply. New pages satisfy the 2508 section-7 rules (bootc- prefix, H1, in SUMMARY, mapped links, no wide tables or non-flowchart mermaid). Main thing to decide: whether auto-with-apply silently doing nothing should be documented as a limitation of a stable backend or fixed first.
Hotspots
- look-closely · api —
docs/src/bootc-composefs.7.md:139-141(7fbd35e): Only new claim in the rebase. Accurate per soft_reboot.rs:94-101 (Auto returns Ok before the reboot branch), but --soft-reboot=auto --apply then exits 0 without rebooting, which arguably is a bug to fix rather than a documented limitation of a now-stable backend. - note · api —
docs/src/SUMMARY.md:74-77(7fbd35e): New vs approved: composefs-finalize-staged(8), which commit 2 hides from --help, now sits under Architecture next to the composefs chapter instead of Experimental. Consistent with the loader-entries man pages being there; just confirm you want an internal command in the nav. - note · api —
docs/book.toml:23-48(548b19b): Keys now drop the leading slash to match upstream style; targets are relative to the old page (top level), so building/ prefixes are right. bootc-experimental-composefs.7.html gets only a page redirect, no anchor map; fine since it only existed briefly.
Safe to skim
docs/src/building/bootc-sealed-images.7.md: Verbatim move; identical to the approved sealed-images.md apart from link pathsdocs/src/bootc-sysroot.7.md: Verbatim move plus link renames, unchanged from approveddocs/src/bootc-installation.7.md: Same text as approved bootc-install.md sectiondocs/src/bootc-bootloaders.7.md: Same text as approved, link renames only
7fbd35e to
c4e3983
Compare
|
Rebuilt clean on main (1857859): new head c4e3983 has only the three composefs commits. The extra commits the PR showed before (the man-page series from bootc-dev#2508 and the tmt bump from bootc-dev#2506) were already merged upstream; the fork PR was listing them against a stale base. None of the composefs commits depended on anything unmerged, and their content is unchanged from 7fbd35e (only a rewrapped line in the last commit message). The body is rewritten short, with an empty rationale placeholder for you. Gate, per D1 (cgwalters-forge/tracker#237, answered A): this should merge upstream only after bootc-dev#2516 (GRUB with a separate /boot) and bootc-dev#2515 (install mount ESP/deployment selection) merge. Tested on a 16-core devspace with Approving with a Generated-by: https://github.com/cgwalters/#llms |
|
Gate update: this also depends on bootc-dev#2500 (composefs signaled by Generated-by: https://github.com/cgwalters/#llms |
Before trying to run `bootc install to-disk` manually remove `/run/ostree/deployment-state/...` as ostree checks a file in this directory to test whether the current deployment is unlocked. 5b52216 adds a check in the ostree path to early return if an overlay already exists on top of `/usr`, but the way `systemd-run` works with `Mount=slave` makes it so that no overlay exists on top of `/usr` but the file in `/run/ostree/deployment-state/..` remains intact which causes ostree to throw an error. Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
Older systemd-repart versions (ex. the one in c9s) do not include the partno field in the JSON output. Make the field optional and fall back to the array index when absent Signed-off-by: Pragyan Poudyal <pragyanpoudyal41999@gmail.com>
If `--generic_image` is passed, we won't actually be sure about what to do with non root and non boot partitions, like var,home,swap etc. So, in that case, take their sizes into account when computing the size for root partition, only if an actual root partition is not defined as a systemd repart definition. The root size computation is as follows: Total Disk Size - (Sum of all partition sizes + padding) - 64 Mib The final 64Mib deduction acts as a partition headroom so that we account for GPT headers Signed-off-by: Johan-Liebert1 <pragyanpoudyal41999@gmail.com>
Simulate it by running systemd-repart again on the disk and test whether all partitions are created properly or not Signed-off-by: Johan-Liebert1 <pragyanpoudyal41999@gmail.com>
CentOS 9 ships systemd 252, which predates two capabilities we relied on viz `--include-partitions` and automatic enablement of fs-verity on ext4 - For restricting a generic image to only make the partitions required for a first boot (root, ESP/BOOT), fallback to the older method of copying only those repart definitions to a particular directory and passing `--definitions=` to systemd-repart - Manually enable fs-verity using tune2fs on ext4 systems on older systemd-repart versions Signed-off-by: Johan-Liebert1 <pragyanpoudyal41999@gmail.com>
libtest runs the container tests concurrently as threads of one process, and "printconfig --all" writes and then removes a fragment in /run/bootc/install that "install config" can pick up while scanning the same directory. That made "install config" flake with a bare "No such file or directory" when the fragment vanished between the scan and the read, and it could also see the fragment half-written or get the wrong config. This is a race between the tests, so fix it there: the test that writes config takes a write lock and the ones that read it take a read lock. Generated-by: AI Signed-off-by: Colin Walters <walters@verbum.org>
We already have `install::BOOT`, but many places still spelled out "boot" or "/boot" by hand, which makes it easy to miss one and invites yet more local constants for the same thing. Prep for reusing it in the GRUB separate /boot handling. Generated-by: AI Signed-off-by: Colin Walters <walters@verbum.org>
With GRUB and a separate boot partition, `bootc switch` and `bootc upgrade` fail with "Getting sorted Type1 boot entries: No such file or directory": the boot partition is mounted on /boot (via the systemd.mount-extra karg install adds), and /sysroot/boot is just an empty directory on the physical root. bootc-dev#2440 taught the booted storage to find the right directory for status, rollback, finalization and GC, but staging a new deployment still built its paths from /sysroot, both for BLS entries and for GRUB's UKI menu entries. So have get_boot_dir_for_grub() return the path it picked along with the directory, keep that in Storage, and derive every GRUB boot path while staging from it. The decision itself moves into a pure function so it can be unit tested. That also fixes the BLS entries' kernel and initramfs paths, which must be relative to the boot partition then. Every disk image-builder makes has a separate /boot, so this blocked using those for composefs. Generated-by: AI Signed-off-by: Colin Walters <walters@verbum.org>
Disks from image-builder always have a separate /boot, but the VMs tmt runs in don't, so nothing covered switching, upgrading or rolling back with that layout on composefs. Installing to a loop device inside the VM with a separate /boot would exercise install, but we can't boot that disk. So instead, like the XBOOTLDR test does, give the VM's own disk the layout install would have made: split the ESP into a smaller ESP and an ext4 XBOOTLDR partition, move /sysroot/boot there, make the BLS entries point at the kernels relative to it and add the systemd.mount-extra karg, and point bootupd's bootuuid.cfg at the new filesystem. Then switch, upgrade and roll back, checking that everything stays on /boot and that GC keeps the kernels there in sync with the entries. Generated-by: AI Signed-off-by: Colin Walters <walters@verbum.org>
|
The rebase of this PR onto bootc-dev#2500 (bootc-dev#2500) is prepared on the branch Conflicts resolved: the old
Generated-by: https://github.com/cgwalters/#llms |
Simplify match statement and remove rebase conflict marker Signed-off-by: Johan-Liebert1 <pragyanpoudyal41999@gmail.com>
Signed-off-by: Johan-Liebert1 <pragyanpoudyal41999@gmail.com>
The baseimage-root lint insists on an /ostree -> sysroot/ostree symlink, and baseimage-composefs warns unless ostree's prepare-root.conf enables composefs. Both are meaningless for images that are only ever deployed with the composefs backend, and just force them to carry ostree cruft. Use the presence of /usr/lib/composefs/setup-root-conf.toml (even if empty) as the signal that an image is composefs-native, mirroring how prepare-root.conf signals ostree. If /ostree is present anyway it is still validated, and /sysroot is still required since both backends mount the physical root there. Closes: bootc-dev#2256 Generated-by: AI Signed-off-by: Colin Walters <walters@verbum.org>
Prep for bootc install using the same signal to pick its default backend. Generated-by: AI Signed-off-by: Colin Walters <walters@verbum.org>
Today only a UKI selects the composefs backend automatically. An image built for composefs with a traditional kernel and initramfs (BLS) is installed with ostree unless every caller passes --composefs-backend, which bootc-image-builder and Anaconda don't. The lints already treat /usr/lib/composefs/setup-root-conf.toml as the marker of a composefs-native image. Use the same marker here: when an image ships it and has no ostree prepare-root.conf, it can't be installed with ostree anyway, so default to composefs. An image with both is still installed with ostree by default, since it may be meant for either backend: bootc's own composefs CI images, for example, add setup-root-conf.toml to a stock base image and pass the flag. Both files are read from the root bootc runs in, as the install configuration and prepare-root.conf already were, including with --source-imgref: bootc-image-builder runs bootc from the image it installs, so it gets the same default. Such an image also failed with --composefs-backend, since install required prepare-root.conf regardless of the backend; only the ostree backend reads it, so it's optional for composefs now. --allow-missing-verity and --uki-addon used to require --composefs-backend at the clap level, which would reject them for an image selecting the backend by itself, so check them after the backend is decided. Generated-by: AI Signed-off-by: Colin Walters <walters@verbum.org>
Build a derived image that adds an empty setup-root-conf.toml and drops ostree's prepare-root.conf, and check that `bootc install to-disk` run from it installs the composefs backend without the flag, both as a self-install and with --source-imgref as bootc-image-builder runs it. An image that keeps prepare-root.conf must still get ostree. On the ostree variant, nothing else selects composefs. Generated-by: AI Signed-off-by: Colin Walters <walters@verbum.org>
Image authors who want their image installed with composefs by default, and only with composefs, need one place stating which rules the image has to match; so far that was only spelled out in the composefs and setup-root-conf.toml pages. Generated-by: AI Signed-off-by: Colin Walters <walters@verbum.org>
The composefs test images were only installed with composefs because every CI path passed --composefs-backend, so CI never exercised the way composefs-native images are meant to select the backend. Build them like such an image instead: ship setup-root-conf.toml (empty unless a baseconfig fills it), drop ostree's prepare-root.conf, and let install pick composefs by itself. This covers the sealed and unsealed UKI variants too; a UKI already selected composefs, and the marker doesn't hurt there. bcvk only takes --bootloader together with --composefs-backend, so the images now name their bootloader in an install configuration file. BOOTC_variant=composefs still selects the composefs plans and filesystem in run-tmt. test-upgrade keeps passing the flag, since it installs the published base image first, which isn't composefs-native. Generated-by: AI Signed-off-by: Colin Walters <walters@verbum.org>
The composefs systemd-boot test images drop bootupd, which an ostree install requires, so installing the image that keeps prepare-root.conf failed there with "bootupd is required for ostree-based installs". Only run that case where bootupd is present; the composefs cases still run. Generated-by: AI Signed-off-by: Colin Walters <walters@verbum.org>
This test formats its own ext4 and runs `bootc install to-existing-root` without a backend flag. Now that the composefs test images default to the composefs backend, bootc infers verity support from the configured fstype (ext4 counts as supported) and creates the repository in strict mode, which fails on a plain mkfs.ext4 with "Filesystem does not support fs-verity". Create the filesystems with -O verity. Generated-by: AI Signed-off-by: Colin Walters <walters@verbum.org>
Prep for declaring the composefs backend stable. Its single page mixed architecture, image building, bootloader and install material, each of which has a natural home in the production docs, so move each section there verbatim: the storage layout into bootc-sysroot(7), sealed image building into a new bootc-sealed-images(7) under Building images, the bootloader and install notes into bootc-bootloaders(7) and bootc-installation(7), and the rest into a new bootc-composefs(7) under Architecture. Only headings and intra-page links change here, plus dropping a sentence in the bootloader section that pointed at bootc-bootloaders(7) itself (`git diff --color-moved` shows the rest as moves). The wording, including the "experimental" status text, is updated in the next commit. Redirects keep the old experimental-composefs.html URL, and its main anchors, working, as well as the short-lived bootc-experimental-composefs.7.html. Generated-by: AI
This is an internal entry point run by bootc-finalize-staged.service and bootc-finalize-staged-hold.service, not something users should invoke, so it shouldn't be listed in `bootc --help` or bootc(8) once the composefs backend is declared stable. Its man page stays, since it documents what the services do. Hidden commands are left out of the CLI JSON that the man page generator reads, so the page's OPTIONS section is now maintained by hand rather than between the generated markers. This takes just the CLI change from bootc-dev#2168 by cgwalters, which also documents the service; that part can still land from there. Generated-by: AI
We already committed in bootc-dev#2298 to in-place upgrades for every composefs system installed since 1.16.0, the V1/V2 EROFS transition has shipped with its bridge for 1.16 UKIs, and `bootc install mount` (bootc-dev#2483) gives post-install tooling a supported way into a deployment instead of poking at backend paths. What was left was docs and help text calling the backend experimental, which no longer matches what we promise. Stable now, under the same RELEASES.md guarantees as the ostree backend: the backend itself (install, upgrade, switch, rollback, status, usr-overlay, soft reboot, and `bootc install mount`), its on-disk state, BLS installs with GRUB or systemd-boot, UKIs with systemd-boot including sealed ones, fs-verity optional on filesystems without it, the composefs.digest= and bare composefs= kernel arguments, bootc-root-setup.service and setup-root-conf.toml, and the image build commands `bootc container ukify`, `split-kernel-and-rootfs` and `compute-composefs-digest`. The last was hidden only because it belonged to the experimental feature set; it is documented as part of the sealed image flow, so it is now visible and gets a man page. The composefs part of the status JSON (BootEntryComposefs) becomes stable API along with it. Still experimental: the grub-cc bootloader (CI covers it only on Fedora 44, with a GRUB binary side-loaded from ELN), UKI addons (they are only installed at install time, never updated on upgrade, garbage collected or reverted on rollback) and unified storage. compute-composefs-digest-from-storage stays hidden as a debugging aid. Known gaps users can hit are listed as limitations rather than as reasons to keep the whole backend experimental: bootc edit, install reset, boot counting, ostree to composefs transitions (planned), --soft-reboot=auto not falling back to a regular reboot, a single ESP without XBOOTLDR, and fsck coverage. Test coverage gaps are tracked separately rather than documented. Generated-by: AI
2ed2ce3 to
852bc0c
Compare
|
@cgwalters Rebased onto main now that bootc-dev#2500 and bootc-dev#2516 are in; ready for your review — approve with |
|
I did some tweaks and pushed bootc-dev#2530 myself, let's drive that one to completion |
4 participants
Declare the composefs backend stable, under the same RELEASES.md guarantees as the ostree backend.
Stable: the backend itself (install, upgrade, switch, rollback, status, usr-overlay, soft reboot,
bootc install mount); its on-disk state and the composefs part of the status JSON (BootEntryComposefs); BLS installs with GRUB or systemd-boot; UKIs with systemd-boot, sealed or not; running without fs-verity; thecomposefs.digest=and barecomposefs=kernel arguments;bootc-root-setup.serviceandsetup-root-conf.toml; andcontainer ukify,split-kernel-and-rootfsandcompute-composefs-digest(now unhidden, with a man page).Still experimental:
--bootloader=grub-cc,--uki-addonand unified storage.compute-composefs-digest-from-storagestays hidden. Known gaps users can hit (no boot counting, nobootc edit/install reset, no ostree-to-composefs transition yet, single ESP, partialfsckcoverage) are listed as limitations in the new bootc-composefs(7). Test coverage gaps and signature enforcement are tracked in cgwalters-forge/tracker#259, bootc-dev#260 and bootc-dev#261 instead.Commits:
docs: Split the composefs backend docs out of the experimental chapter: a verbatim move ofbootc-experimental-composefs.7.mdintobootc-composefs.7.md,building/bootc-sealed-images.7.mdand the existing sysroot, bootloader and installation chapters, with redirects from the old URLs. Best read withgit show --color-moved.cli: Hide composefs-finalize-staged: the CLI part of docs: Update composefs finalization docs bootc-dev/bootc#2168; it's an internal entry point for the finalize services.composefs: Declare the backend stable: wording, help text, the unhidden command, and regenerated man pages and schema.Merge gate: satisfied. bootc-dev#2500 (composefs signaled by
setup-root-conf.toml) and bootc-dev#2516 (GRUB with a separate /boot on upgrade) have merged, per cgwalters-forge/tracker#237, and this is rebased onto main. (bootc-dev#2515,install mountdeployment selection, was moved off this gate: it stays a draft until bootloader conformance dispatch exists.)Post-stabilization follow-ups (boot counting and the other P1 items in cgwalters-forge/tracker#160) are out of scope here.
Tested at this head, 852bc0c, on main at 7d6df9a, on a 16-core devspace (cgwalters-devspace-36991593056) with
BOOTC_base=quay.io/fedora/fedora-bootc:44:just validate: passed;just build-mdbook-to: passed, the book builds.This rework changed only docs, plus a rebase onto main; unit tests and tmt were not rerun. c4e3983 on 1857859 also passed
just unit-testsand composefs tmt plan-01-readonly (GRUB/BLS and systemd-boot/UKI, ext4, unsealed).Generated-by: https://github.com/cgwalters/#llms
Review draft in cgwalters-forge, not upstream yet. This section is removed when the PR is opened upstream.
bootc-dev/bootc, basemainPVTI_lADOE9oHIs4BlJLczg9kEQ0To review:
/promoteon a line of its own, to open it upstream, ready for review. Either covers only the commits pushed so far.Signed-off-by: Colin Walters <walters@verbum.org>to the commits lacking it (the bot's and yours; anyone else's only if you ask), with you as committer./draftline (in the same comment or before) to open it upstream as a draft (/readyundoes that).Bot run: session 929c7a64, agent a6ed81c1 · claude-opus-5-5 6.6M in (91% cached) / 25k out · ~$4.69 inference + ~$1.20 compute (est., list prices) · 34m wall · 7.6 core-h, Actions run 36596019513