Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
99abad6
fix(api): ask to set up tailscale when it makes no auth key
fransallen Nov 16, 2026
0e142c0
feat(dash): link tailscale in the add sandbox server error
fransallen Nov 16, 2026
2042b90
fix(api): drop characters tailscale rejects from auth key descriptions
fransallen Nov 16, 2026
bd2708e
feat(dash): show integrations as cards with filters and search
fransallen Nov 16, 2026
248e786
feat(dash): link straight to the hetzner and registries cards
fransallen Nov 16, 2026
c245b15
feat(scripts): offer to set up the host from pyo-up.sh
fransallen Nov 16, 2026
66adc01
chore: merge feat/integrations-cards into dev
fransallen Nov 16, 2026
380d58d
feat(dash): show real logos on the integrations cards
fransallen Nov 16, 2026
f64262a
refactor(dash): drop the clusters card from integrations
fransallen Nov 16, 2026
11bc199
feat(dash): move pyo's own backups to a tab on backups
fransallen Nov 16, 2026
0ffff01
feat(sandbox): run sandboxes under gvisor on servers without kvm
fransallen Nov 16, 2026
86bee8b
build(sandbox): ship runsc beside the sandbox bundle
fransallen Nov 16, 2026
7c30ca1
test(sandbox): run the gvisor driver end to end in ci
fransallen Nov 16, 2026
b35745b
feat: open every integration's settings in its modal
fransallen Nov 16, 2026
ce4942e
chore: merge feat/integrations-logos-backups into dev
fransallen Nov 16, 2026
45f5549
feat(api): make templates on gvisor sandbox servers and show runtime
fransallen Nov 16, 2026
f17bc7f
style(dash): show integrations in three columns on a wider page
fransallen Nov 16, 2026
2ff15ad
feat(api): buy sandbox servers at hetzner
fransallen Nov 16, 2026
97ccb61
feat(cli): buy a sandbox server at hetzner
fransallen Nov 16, 2026
3490f2f
fix: reserve the root settings pages as team slugs
fransallen Nov 16, 2026
0447581
feat(dash): move installation settings to root paths
fransallen Nov 16, 2026
0badd85
feat: move the github app back to settings
fransallen Nov 16, 2026
e9b01a7
chore: merge feat/root-backups into dev
fransallen Nov 16, 2026
cda4b24
feat(dash): add sandbox servers cloud provider first
fransallen Nov 16, 2026
a9b5ada
style(dash): size the integration cards' text like the rest of the da…
fransallen Nov 16, 2026
0f90d0b
style(dash): pull the integration card menu closer to its badge
fransallen Nov 16, 2026
fbf1df5
docs(api): say gvisor servers start sandboxes from a template's disk
fransallen Nov 16, 2026
a30e943
style(dash): drop the tags from integration cards
fransallen Nov 16, 2026
db2fedd
style(dash): give every integration card the same height
fransallen Nov 16, 2026
3bf1544
style(dash): widen the templates page like integrations
fransallen Nov 16, 2026
a6385f8
chore: merge feat/sandbox-cloud-first into dev
fransallen Nov 16, 2026
43434ed
style(dash): show hetzner's logo when buying a sandbox server
fransallen Nov 16, 2026
02d6179
style(dash): give template cards more room
fransallen Nov 16, 2026
40095b9
style(dash): set template descriptions at the bottom of their cards
fransallen Nov 16, 2026
0a77fb2
style(dash): say connected once on integration cards
fransallen Nov 16, 2026
44558a2
feat(dash): ask for a sandbox server before new sandbox
fransallen Nov 16, 2026
8583695
chore: merge feat/sandbox-server-simple into dev
fransallen Nov 16, 2026
e59e446
fix(sandbox): give gvisor sandboxes their address back on resume
fransallen Nov 16, 2026
109f27b
fix(deploy): list gvisor sandboxes with runsc list --quiet on uninstall
fransallen Nov 16, 2026
73f62bd
chore: merge fix/gvisor-first-run into dev
fransallen Nov 16, 2026
1f371d2
fix(api): fetch join scripts off the tailnet for new machines
fransallen Nov 16, 2026
ba4f2f6
chore: merge fix/join-scripts-before-tailnet into dev
fransallen Nov 16, 2026
7f1e350
fix(sandbox): read image references with a tag and a digest
fransallen Nov 16, 2026
88899b9
feat(images): add the base sandbox template image
fransallen Nov 16, 2026
cb62495
build(release): publish sandbox-base and pin it in release.json
fransallen Nov 16, 2026
5ba89d7
feat(sandbox): add the built-in base template and keep it on the release
fransallen Nov 16, 2026
ff1ec20
feat(cli): leave the sandbox template out to use base
fransallen Nov 16, 2026
ba342ac
ci: build the base template image and boot it on kvm
fransallen Nov 16, 2026
16c9bf9
feat(dash): show the base template and start new sandboxes on it
fransallen Nov 16, 2026
83b3f23
chore: merge feat/sandbox-base-template into dev
fransallen Nov 16, 2026
1e1ae77
chore(release): 0.4.1
fransallen Nov 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 107 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,10 @@ jobs:
--health-interval 2s
--health-timeout 5s
--health-retries 30
# Where this branch's base template image goes, for the daemon to pull.
registry:
image: registry:3.0.0
ports: ["5000:5000"]
env:
PYO_SANDBOX_ENVD: ${{ github.workspace }}/target/x86_64-unknown-linux-musl/release/pyo-envd
steps:
Expand Down Expand Up @@ -137,9 +141,16 @@ jobs:
run: |
cargo build --release --locked -p pyo-envd --target x86_64-unknown-linux-musl
cargo build --locked -p pyo-sandbox
# Pyo's built-in template `base`, from this branch's Dockerfile.
- name: Build the base template image
run: |
docker build -f deploy/images/sandbox-base.Dockerfile -t localhost:5000/pyo/sandbox-base:e2e deploy/images
docker push localhost:5000/pyo/sandbox-base:e2e
docker image ls localhost:5000/pyo/sandbox-base:e2e
- name: pyo-sandbox e2e
env:
PYO_SANDBOX_E2E: "1"
PYO_SANDBOX_E2E_BASE_IMAGE: 127.0.0.1:5000/pyo/sandbox-base:e2e
run: cargo test --locked -p pyo-sandbox --test e2e --test template_e2e -- --nocapture
- name: pyo-api sandbox e2e
env:
Expand All @@ -164,6 +175,102 @@ jobs:
sudo nft list ruleset | head -80 || true
ls -la .cache/sandbox-e2e || true

# Pyo Sandbox on gVisor, end to end: runsc as root, as on a server
# without KVM (Hetzner Cloud). Firecracker is left out, so detection
# falls back to gVisor although the runner has KVM. It mounts disks and
# changes the host's network, which the runner is for.
gvisor:
name: Sandbox on gVisor
runs-on: ubuntu-24.04
timeout-minutes: 30
env:
PYO_SANDBOX_ENVD: ${{ github.workspace }}/target/x86_64-unknown-linux-musl/release/pyo-envd
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Packages
run: |
sudo apt-get update
sudo apt-get install -y --no-install-recommends busybox-static e2fsprogs nftables musl-tools zstd
# runsc, the release pinned in gvisor.env, checked against its sha512.
- uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
id: runsc
with:
path: .cache/sandbox-gvisor
key: sandbox-gvisor-${{ hashFiles('crates/pyo-sandbox/scripts/gvisor.env', 'crates/pyo-sandbox/scripts/gvisor-assets.sh') }}
- name: runsc
run: |
set -euo pipefail
crates/pyo-sandbox/scripts/gvisor-assets.sh | tee -a "$GITHUB_ENV"
- uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
if: steps.runsc.outputs.cache-hit != 'true'
with:
path: .cache/sandbox-gvisor
key: ${{ steps.runsc.outputs.cache-primary-key }}
- uses: dtolnay/rust-toolchain@683b588c487a69d01f6c6ddf40f036ff682cc2c8 # v1
with:
toolchain: 1.99.1
targets: x86_64-unknown-linux-musl
- uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3
with:
key: sandbox-gvisor
- name: Build pyo-envd (musl)
env:
CC_x86_64_unknown_linux_musl: musl-gcc
run: cargo build --release --locked -p pyo-envd --target x86_64-unknown-linux-musl
# As root. cargo isn't on root's PATH, so the runner user builds the
# test binary and sudo -E runs only it.
- name: gVisor e2e (root)
env:
PYO_SANDBOX_GVISOR_E2E: "1"
run: |
set -euo pipefail
bin=$(cargo test --locked -p pyo-sandbox --test gvisor_e2e --no-run --message-format=json \
| jq -r 'select(.reason == "compiler-artifact" and .target.name == "gvisor_e2e" and .executable != null) | .executable')
[ -x "$bin" ] || { echo "::error::No gvisor_e2e test binary."; exit 1; }
sudo -E "$bin" --nocapture
- name: What went wrong
if: failure()
run: |
sudo find /tmp -name runsc.log -path '*pyo-gvisor-e2e*' -exec tail -n 40 {} \; || true
sudo nft list ruleset | head -80 || true
ip netns list || true

# The base sandbox template's image for both architectures, built
# without pushing, when its Dockerfile changes. The release publishes it.
sandbox-base:
name: Base sandbox template image
runs-on: ubuntu-24.04
timeout-minutes: 45
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
- name: Did the Dockerfile change
id: changed
env:
PR_BASE: ${{ github.event.pull_request.base.sha }}
BEFORE: ${{ github.event.before }}
run: |
base=${PR_BASE:-$BEFORE}
if [ -z "$base" ] || ! git cat-file -e "$base^{commit}" 2>/dev/null ||
git diff --name-only "$base" HEAD | grep -qx deploy/images/sandbox-base.Dockerfile; then
echo "build=true" >> "$GITHUB_OUTPUT"
else
echo "Its Dockerfile didn't change."
fi
- uses: docker/setup-qemu-action@99012661954931238ded8c8b007157a8430204e1 # v4.4.0
if: steps.changed.outputs.build == 'true'
- uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
if: steps.changed.outputs.build == 'true'
- name: Build
if: steps.changed.outputs.build == 'true'
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7.4.0
with:
context: deploy/images
file: deploy/images/sandbox-base.Dockerfile
platforms: linux/amd64,linux/arm64
push: false

waf:
name: WAF
runs-on: ubuntu-24.04
Expand Down
38 changes: 33 additions & 5 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ jobs:
strategy:
fail-fast: true
matrix:
image: [pyo, agent, builder, flows, waf, geoip, db-controller, kv, static]
image: [pyo, agent, builder, flows, waf, geoip, db-controller, kv, static, sandbox-base]
# Pyo's Postgres, one image per version, from one Dockerfile.
# One repository, tagged <version>-<release>: CloudNativePG reads
# the version from the tag.
Expand All @@ -111,8 +111,13 @@ jobs:
# (deploy/images/moco), one Dockerfile with a target each.
- { image: mysql-operator, file: moco/moco, context: deploy/images/moco, target: operator }
- { image: mysql-backup, file: moco/moco, context: deploy/images/moco, target: backup }
# Pyo Sandbox's built-in template `base`. Sandbox servers run on
# both architectures, so it's built for both (arm64 under QEMU).
- { image: sandbox-base, context: deploy/images, platforms: "linux/amd64,linux/arm64" }
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: docker/setup-qemu-action@99012661954931238ded8c8b007157a8430204e1 # v4.4.0
if: contains(matrix.platforms, 'arm64')
- uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
- uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
with:
Expand All @@ -128,7 +133,7 @@ jobs:
file: deploy/images/${{ matrix.file || matrix.image }}.Dockerfile
build-args: ${{ matrix.args }}
target: ${{ matrix.target }}
platforms: linux/amd64
platforms: ${{ matrix.platforms || 'linux/amd64' }}
cache-from: ${{ matrix.rust && format('type=gha,scope=rust-{0}', github.run_id) || '' }}
push: true
tags: ${{ env.IMAGES }}/${{ matrix.repo || matrix.image }}:${{ matrix.tag }}${{ needs.verify.outputs.version }}
Expand All @@ -138,6 +143,22 @@ jobs:
env:
IMAGE: ${{ env.IMAGES }}/${{ matrix.repo || matrix.image }}@${{ steps.build.outputs.digest }}
run: cosign sign --yes "$IMAGE"
# Installations and sandbox servers pull without logging in. A new
# package on ghcr.io may start private: make it public in its
# package settings, then run the job again.
- name: Anyone can pull it
env:
REPO: upsetdev/pyo/${{ matrix.repo || matrix.image }}
DIGEST: ${{ steps.build.outputs.digest }}
run: |
token=$(curl -fsS "https://ghcr.io/token?scope=repository:$REPO:pull" | jq -r .token)
code=$(curl -sS -o /dev/null -w '%{http_code}' -H "Authorization: Bearer $token" \
-H 'Accept: application/vnd.oci.image.index.v1+json,application/vnd.oci.image.manifest.v1+json' \
"https://ghcr.io/v2/$REPO/manifests/$DIGEST")
if [ "$code" != 200 ]; then
echo "::error::ghcr.io/$REPO can't be pulled without logging in (HTTP $code). Make the package public in its settings on GitHub, then run this job again."
exit 1
fi

release:
name: GitHub release
Expand Down Expand Up @@ -176,6 +197,7 @@ jobs:
cd sandbox
for a in x86_64 aarch64; do
sha256sum -c "pyo-sandbox-linux-$a.tar.gz.sha256"
sha256sum -c "pyo-sandbox-gvisor-linux-$a.tar.gz.sha256"
done
# What Pyo needs to update itself from the dashboard: the images
# pinned by the digests that were just signed, and the manifests.
Expand All @@ -187,13 +209,15 @@ jobs:
tagged() { docker buildx imagetools inspect "$IMAGES/$1:$2$VERSION" --format '{{json .Manifest}}' | jq -r .digest; }
bun run scripts/release-manifest.ts --version "$VERSION" --images "$IMAGES" \
--pyo "$(digest pyo)" --agent "$(digest agent)" --builder "$(digest builder)" --flows "$(digest flows)" --waf "$(digest waf)" --geoip "$(digest geoip)" \
--db-controller "$(digest db-controller)" --kv "$(digest kv)" --static "$(digest static)" \
--db-controller "$(digest db-controller)" --kv "$(digest kv)" --static "$(digest static)" --sandbox-base "$(digest sandbox-base)" \
--mysql-operator "$(digest mysql-operator)" --mysql-backup "$(digest mysql-backup)" \
--postgres-16 "$(tagged postgres 16-)" --postgres-17 "$(tagged postgres 17-)" --postgres-18 "$(tagged postgres 18-)" \
--agent-access "$(docker run --rm "$IMAGES/agent:$VERSION@$(digest agent)" access-hash)" \
--sandbox-x86_64 "$(cut -d' ' -f1 sandbox/pyo-sandbox-linux-x86_64.tar.gz.sha256)" \
--sandbox-aarch64 "$(cut -d' ' -f1 sandbox/pyo-sandbox-linux-aarch64.tar.gz.sha256)" > release.json
jq '{version, images, k3s, minFrom, manualOnly, beforeYouUpdate, agentAccess, sandbox, migrations: (.migrations | length)}' release.json
--sandbox-aarch64 "$(cut -d' ' -f1 sandbox/pyo-sandbox-linux-aarch64.tar.gz.sha256)" \
--sandbox-gvisor-x86_64 "$(cut -d' ' -f1 sandbox/pyo-sandbox-gvisor-linux-x86_64.tar.gz.sha256)" \
--sandbox-gvisor-aarch64 "$(cut -d' ' -f1 sandbox/pyo-sandbox-gvisor-linux-aarch64.tar.gz.sha256)" > release.json
jq '{version, images, k3s, minFrom, manualOnly, beforeYouUpdate, agentAccess, sandbox, sandboxGvisor, migrations: (.migrations | length)}' release.json
# The release's own upgrader reads it before it's signed: an image
# it doesn't accept, or an agentAccess that isn't its agent's roles,
# fails the release here, not on installations.
Expand Down Expand Up @@ -276,3 +300,7 @@ jobs:
sandbox/pyo-sandbox-linux-x86_64.tar.gz.sha256
sandbox/pyo-sandbox-linux-aarch64.tar.gz
sandbox/pyo-sandbox-linux-aarch64.tar.gz.sha256
sandbox/pyo-sandbox-gvisor-linux-x86_64.tar.gz
sandbox/pyo-sandbox-gvisor-linux-x86_64.tar.gz.sha256
sandbox/pyo-sandbox-gvisor-linux-aarch64.tar.gz
sandbox/pyo-sandbox-gvisor-linux-aarch64.tar.gz.sha256
6 changes: 5 additions & 1 deletion .github/workflows/sandbox.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ name: Sandbox
# A sandbox server's files for each architecture, as
# pyo-sandbox-linux-<arch>.tar.gz with its .sha256: the daemon and
# pyo-envd (static, musl), Firecracker and its jailer, and Pyo's guest
# kernel. deploy/sandbox.sh installs them. The Release workflow runs it and
# kernel. Beside it, pyo-sandbox-gvisor-linux-<arch>.tar.gz: runsc for
# servers without KVM. deploy/sandbox.sh installs them. The Release workflow runs it and
# puts the tarballs in the GitHub release. Run it by hand (Actions ›
# Sandbox › Run workflow) to try the builds on a branch: it publishes
# nothing, the tarballs stay in the run's artifacts.
Expand Down Expand Up @@ -97,6 +98,9 @@ jobs:
check/pyo-envd --version
check/firecracker --version
check/jailer --version
mkdir check-gvisor
tar -xzf "out/pyo-sandbox-gvisor-linux-$ARCH.tar.gz" -C check-gvisor
check-gvisor/runsc --version
file check/pyo-sandbox check/pyo-envd | tee /dev/stderr | grep -c 'static' | grep -qx 2
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
Expand Down
2 changes: 2 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ Read the script before you run it. It changes the machine:

To undo it, run `/usr/local/bin/k3s-uninstall.sh`.

If you skip this step, `scripts/pyo-up.sh` asks to run it for you.

**2. Build and deploy (as your normal user):**

```bash
Expand Down
24 changes: 12 additions & 12 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ resolver = "3"
members = ["crates/pyo-proto", "crates/pyo-api", "crates/pyo-agent", "crates/pyo-builder", "crates/pyo-cli", "crates/pyo-flows", "crates/pyo-resp", "crates/pyo-kv", "crates/pyo-db-controller", "crates/pyo-static", "crates/pyo-envd", "crates/pyo-sandbox"]

[workspace.package]
version = "0.4.0"
version = "0.4.1"
edition = "2024"
license = "Apache-2.0"
publish = false
Expand Down
10 changes: 9 additions & 1 deletion RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ A version is one number for everything: `version` in the root `Cargo.toml`.

`.github/workflows/release.yml` then checks that the tag matches `Cargo.toml`,
runs the tests (the WAF's Go tests in `waf/` too), publishes signed images
to `ghcr.io/upsetdev/pyo/{pyo,agent,builder,flows,waf,geoip,db-controller,kv,postgres,mysql-operator,mysql-backup}`
to `ghcr.io/upsetdev/pyo/{pyo,agent,builder,flows,waf,geoip,db-controller,kv,postgres,mysql-operator,mysql-backup,sandbox-base}`
(all from `deploy/images/`, at the same version. The Rust binaries are
compiled once, by the `binaries` job, and the Rust images' jobs take them
from that run's build cache),
Expand Down Expand Up @@ -53,6 +53,14 @@ crates/pyo-sandbox/scripts/bundle.sh --arch x86_64 \
`sandbox.sh --bundle dist/pyo-sandbox-linux-x86_64.tar.gz` installs such a
build, with `--bundle-sha256 <its sum>` or, unchecked, `--dev`.

`sandbox-base` is Pyo Sandbox's built-in template `base`
(`deploy/images/sandbox-base.Dockerfile`), built for amd64 and arm64.
`release.json` pins it as `images.sandboxBase`, and Pyo seeds the template
from it. Sandbox servers pull it without logging in, so the release checks
that every image can be pulled anonymously. The first time a new image is
published, ghcr.io may make its package private: make it public in the
package's settings, then run the failed job again.

`release.json` is signed with
cosign keyless as `release.json.sigstore.json` (the job needs `id-token:
write`, and `.github/release-signing-config.json` pins Sigstore's public
Expand Down
2 changes: 1 addition & 1 deletion crates/pyo-agent/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ publish.workspace = true

[dependencies]
anyhow = "1.0.104"
pyo-proto = { version = "0.4.0", path = "../pyo-proto" }
pyo-proto = { version = "0.4.1", path = "../pyo-proto" }
futures = "0.3.34"
http = "1.5.0"
k8s-openapi = { version = "0.28.0", features = ["latest"] }
Expand Down
Loading
Loading