Skip to content

e2e: add multi-node rollout test on EKS - #240

Open
ptalgulk01 wants to merge 1 commit into
bootc-dev:mainfrom
ptalgulk01:scale-test-multi-node
Open

ptalgulk01 wants to merge 1 commit into
bootc-dev:mainfrom
ptalgulk01:scale-test-multi-node

Conversation

@ptalgulk01

@ptalgulk01 ptalgulk01 commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Coverage so far only exercises a single node per pool, so staged rollout across a fleet has never been tested. TestMidRolloutImageChange does provision two nodes, but it runs on bink only.

TestMultiNodeRollout provisions three nodes in one pool, rolls the pool to a new image, and asserts every node boots the new digest and the pool reports all nodes updated. It declares both bink and eks; the GitHub runners cannot host three nodes, so in practice this provides coverage on EKS.

maxUnavailable is asserted inside the same polling loop that waits for convergence, so the invariant is sampled throughout the rollout. Asserting it once up front would pass trivially, as UpdatingCount is zero before the rollout starts. A violation aborts immediately instead of retrying, which would let a later poll mask it.

A node that stays Degraded past a grace period aborts the wait and reports the daemon's own error. Degraded is cleared on the next successful reconcile, so a retried transient error must not fail the test; without the early abort, a failure such as an unauthorized registry pull surfaced only as an opaque matcher timeout minutes later.

Each node's daemon pod is also checked for Running before waiting for Idle, so a pod stuck in ImagePullBackOff is named directly.

Signed-off-by: Prachiti Talgulkar <ptalgulk01@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant