Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,38 @@ docstring gives: it is a record, not a claim about now.

## 2026-11-16

- **`eden` is in OpenTofu's state, and #1443 is done.** The first
untargeted `tofu -chdir=tofu plan` on `phoenix` since the 2026-11-12
rebuild refreshed every guest, `eden` (141) and `ragnarok` (130) among
them, and said *No changes*. So the replace that rebuilt `eden` went through
the pipeline, and plans no longer need `-target` to step around it.
`ragnarok`'s half was done on 2026-11-12 under #1464, and proved by
`finish-attack-vm.sh`. The roadmap entry and the one-time step in
`deploy-merged-changes.md` leave. `build-the-bloodhound-guest.md` §1 now
rebuilds `eden` with `-replace` instead of the import, which has been
done. A rebuild still loses the graph and the key, as ADR-0081 accepts.

- **Correction: `eden`'s admin password was not changed until 08:43 UTC on
2026-11-16, and is now.** This corrects the entry below, *`eden`'s admin
password changed, and X11 forwarding turned off by hand*, which said it had
been changed in the UI. A live check found the `admin` row of
`auth_secrets` still as created at 03:28 on 2026-11-13, and BloodHound's
audit log showing no password change and no activity at all after
05:25 on 2026-11-15. The change was made again and checked in the database:
`UpdateAuthSecret` succeeded at 08:43:33, and the new secret expires on
2027-02-14. BloodHound expires a password after 90 days, so the next change
is due by then. Checking the database, not the UI, is what tells.

- **Template 901 takes `autoinstall` off the kernel line.** Subiquity copies
what follows `---` in the boot command into `GRUB_CMDLINE_LINUX_DEFAULT`, so
every clone, `eden` included, booted with `autoinstall`. It does nothing on
an installed system. `packer/ubuntu/prune.sh` now clears it before
regenerating GRUB, and fails the build if it is still in `grub.cfg`. On
`eden` it was cleared by hand the same way; it takes effect at the next
boot, as does the crash-kernel reservation the purge below dropped, since
`eden` had not rebooted since 2026-11-12. Not built yet: the 901 rebuilt
earlier today still carries it.

- **A wiki drift check that cannot run now alerts**
([#1531](https://github.com/Gerrrt/HomeLab/issues/1531)). On 2026-11-15
the run on `oracle` failed at the wiki fetch, exit 128. The checker never
Expand Down
5 changes: 0 additions & 5 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,11 +199,6 @@ Gated on the domain, on a household observation, or on something to publish.
The name is
[ADR-0106](adr/0106-name-the-range-host-gandalf-and-leave-ifrit-to-its-disabled-amt.md)'s.
→ [runbook](runbooks/build-the-playground.md)
- **[#1443](https://github.com/Gerrrt/HomeLab/issues/1443) Bring `eden` into
`tofu/`.** `ragnarok` is in, and `eden` is declared but still the hand-built
guest: importing it plans a replace, which rebuilds it and its key
([`build-the-bloodhound-guest.md`](runbooks/build-the-bloodhound-guest.md)).
Until then every plan and apply keeps `-target`.
- **[#1176](https://github.com/Gerrrt/HomeLab/issues/1176) Move the SOC to
Wazuh 5.0.** Gated on a stable 5.0 release. It retires filebeat and the
certificate generator, and it is the only path that closes the CVE issues
Expand Down
39 changes: 14 additions & 25 deletions docs/runbooks/build-the-bloodhound-guest.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,10 +101,12 @@ Three things have to be in place first:
| grep ssh-ed25519
```

**If `eden` exists already** (VMID 141 is in `qm list` on `Saruman`), stop
here and go to *Once: bring the hand-built `eden` into the pipeline* below.
The commands next are for a first build only: an apply with 141 taken fails
on the clone, after it has made the pool.
**If `eden` exists already** (VMID 141 is in `qm list` on `Saruman`), it is
in the state: the hand-built guest was imported and replaced on 2026-11-12
under [#1443](https://github.com/Gerrrt/HomeLab/issues/1443), and the first
untargeted plan after it, on 2026-11-16, said *No changes*. The commands
below then only refresh it: the plan should say *No changes*, and anything
else is a stop. To rebuild it on purpose, use *Rebuild: replace it* below.

For a first build, on `phoenix`:

Expand All @@ -119,34 +121,21 @@ The plan should add the guest and the `bloodhound` pool, and nothing else.
Proxmox warns that the sum of thin volumes exceeds the pool. That is the
allocation ADR-0081 accepts, and the warning is expected.

### Once: bring the hand-built `eden` into the pipeline
### Rebuild: replace it

`eden` was first built by hand, with `qm clone 901 141` and `qm set`
(2026-10-07), so the state does not know it. Import it once, as the domain's
six were ([`build-the-lab-domain.md`](build-the-lab-domain.md#rebuild-from-the-pipeline)):

```bash
tofu -chdir=tofu import 'module.guest["eden"].proxmox_virtual_environment_vm.this' Saruman/141
tofu -chdir=tofu plan -target='module.guest["eden"]'
```

The plan should **replace** `eden` and **create** the `bloodhound` pool, and
nothing else. A guest built by hand has no `clone` to read back, so this is the
one plan where a replace is expected. Anything else is a stop.

The replace destroys both disks, the graph and the guest's age key with them.
A replace destroys both disks, the graph and the guest's age key with them.
ADR-0081 accepts that: the graph is one collection run from rebuilt, and the
key is replaced by issuing new values. So take it out of the stack first, then
let the pipeline make it again:
key is replaced by issuing new values.

```bash
tofu -chdir=tofu destroy -target='module.guest["eden"]'
tofu -chdir=tofu plan -target='module.guest["eden"]' -out=eden.tfplan
tofu -chdir=tofu plan -replace='module.guest["eden"].proxmox_virtual_environment_vm.this' \
-target='module.guest["eden"]' -out=eden.tfplan
tofu -chdir=tofu apply eden.tfplan && rm tofu/eden.tfplan
```

Then continue from §2. At §3, the guest is new and so is its key: follow §11
for the `.sops.yaml` rule and the secrets file.
The plan should replace `eden` and nothing else. Then continue from §2. At
§3, the guest is new and so is its key: follow §11 for the `.sops.yaml` rule
and the secrets file.

## 2. First boot, and the data disk

Expand Down
4 changes: 2 additions & 2 deletions docs/runbooks/build-the-lab-templates.md
Original file line number Diff line number Diff line change
Expand Up @@ -366,8 +366,8 @@ What each one does, so a stall can be placed:
finds `cidata`, installs unattended, and reboots. Packer waits for SSH as
`packer` with `phoenix`'s key. `packer/ubuntu/prune.sh` purges the
server seed's services no clone uses (VMware tools, ModemManager, snapd,
thermald, multipath, iSCSI, kdump, udisks2) and fails the build if one
survives. Packer then upgrades, turns off X11 forwarding, and wipes
thermald, multipath, iSCSI, kdump, udisks2), takes the installer's
`autoinstall` off the kernel line, and fails the build if any of that is left. Packer then upgrades, turns off X11 forwarding, and wipes
cloud-init's state, machine-id and host keys. It then deletes the `packer`
user.
- **Fedora** picks "Install Fedora" from the GRUB menu, whose default is
Expand Down
14 changes: 1 addition & 13 deletions docs/runbooks/deploy-merged-changes.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,19 +181,7 @@ Some merges come with a step that is run once and is then done. Each one is
listed here while its issue is open, and leaves this page when the issue
closes.

- **`eden` and `ragnarok` into the pipeline ([#1443](https://github.com/Gerrrt/HomeLab/issues/1443)).**
Before the first plan on `phoenix`:
1. Add `TF_VAR_gandalf_endpoint` and `TF_VAR_gandalf_api_token` to
`phoenix.env` ([`build-the-playground.md`](build-the-playground.md), §5).
2. Add both guests' MACs to `tofu/identities.auto.tfvars`, and re-escrow it
([`provision-lab-guests.md`](provision-lab-guests.md), §3).

Until both guests are rebuilt, the plan shows changes for `eden` and
`ragnarok`, and for nothing else. Then:
- `eden`: [`build-the-bloodhound-guest.md`](build-the-bloodhound-guest.md),
*Once: bring the hand-built `eden` into the pipeline*.
- `ragnarok`: [`build-the-playground.md`](build-the-playground.md), §5,
ending in `finish-attack-vm.sh`.
None is open.

## If something goes wrong

Expand Down
10 changes: 9 additions & 1 deletion packer/ubuntu/prune.sh
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,11 @@ if [ -n "$installed" ]; then
fi

# kdump-tools reserved crash-kernel memory on the kernel line; without it the
# reservation is dead weight on every clone.
# reservation is dead weight on every clone. The installer also copies the
# boot command's `autoinstall` (packer/ubuntu.pkr.hcl) into
# GRUB_CMDLINE_LINUX_DEFAULT, so every clone booted with it. It does nothing
# on an installed system, but it is not a choice anyone made.
sed -i 's/^GRUB_CMDLINE_LINUX_DEFAULT="autoinstall"$/GRUB_CMDLINE_LINUX_DEFAULT=""/' /etc/default/grub
update-grub

# Fail the build rather than ship a template that still carries one.
Expand All @@ -55,3 +59,7 @@ if grep -q crashkernel /boot/grub/grub.cfg; then
echo "prune.sh: crashkernel= is still on the kernel line" >&2
exit 1
fi
if grep -q autoinstall /boot/grub/grub.cfg; then
echo "prune.sh: autoinstall is still on the kernel line" >&2
exit 1
fi
Loading