Skip to content

fix(uat): mark RBAC-05 and RBAC-06 flaky in the matrix gate - #97

Merged
lex00 merged 1 commit into
mainfrom
fix/uat-flaky-rbac
Nov 18, 2026
Merged

lex00 merged 1 commit into
mainfrom
fix/uat-flaky-rbac

Conversation

@lex00

@lex00 lex00 commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

Closes #96

Evidence: in the last 17 uat runs, 4 failed the matrix gate with "Expected failures that now PASS" naming exactly RBAC-05 and RBAC-06 (runs 44190279679, 44232635312, 44233018360, 44400202011). The other 13 matched the list (11 failures). Merge_group run 44399609125 and push run 44400202011 on the same commit disagree for the same reason.

Cause: RBAC-05 waits 60s for .status.endpointUrl, which the operator sets at its next resync (measured 61 to 65s), so it wins that race on some runs. RBAC-06 reuses the pod RBAC-05 creates: with no pod its result is empty ('' does not contain 'not authorized'); with the pod, the operator rejects other-vm and it passes. The two flip together. The suite is upstream's, pinned and unmodified, so there is nothing small to fix in m80's setup.

Change: flaky: lines in uat/expected-failures.txt may pass or fail; the result is printed, not fatal. Every other expected failure is still strict in both directions, and a flaky case that did not run at all still fails the gate. Adds uat/test_assert_matrix.py (5 cases). Other race-group cases (QS-07, NET-01, NET-04, MEM-07) did not flip in these 17 runs and stay strict.

Gate: go build, go vet, gofmt, go test -race, python3 -m unittest uat/test_assert_matrix.py.

@lex00
lex00 enabled auto-merge November 18, 2026 21:43
@lex00
lex00 added this pull request to the merge queue Nov 18, 2026
Merged via the queue into main with commit ae4b6e2 Nov 18, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

uat: RBAC-05 and RBAC-06 flip between runs and keep main red

1 participant