Repository navigation
Conversation
This was referenced Nov 21, 2026
ci(visual): run the matrix's independent cells in parallel so the gate is not one serial sweep
#1208
Open
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Written-by: HugoXK
Author: Fledge Platform Dev-3
Fixes #787
What a user sees differently
Nothing on a console page. The change is inside CI: the
visualjob stops downloading Chromium on every run, and its job summary now names every measured case with its own result instead of one line for the whole matrix. A reader of a red run sees the page, viewport and scheme that failed; a reader of a green run sees the full table.What changed
.github/workflows/ci.yml: inside the existingvisualjob (no job added, none moved, the graph is unchanged), a step reads the installed Playwright version fromplaywright/package.json, andactions/cache(0057852…, v4.3.0) restores~/.cache/ms-playwrightunder${{ runner.os }}-playwright-${{ steps.playwright.outputs.version }}. There is deliberately norestore-keys: a Playwright bump has to miss and reinstall rather than launch an older browser under a newer lockfile. Thenpx playwright install --with-deps chromiumstep andnpm run visual:matrixstay unconditional, so a hit skips the download only.scripts/visual-matrix.ts: records one case per page/viewport/scheme as it runs (record()takes the failure count around the case), refuses a case whose key was already recorded — two cases share one<width>-<slug>.png, so the second would pass on the first one's picture — and after both locales compares the recorded keys with the promised set (checkCoverage()). A dropped page, width or scheme is a failure with the missing keys named, not a shorter table. The table reachesGITHUB_STEP_SUMMARYin thefinally, before the exit code is decided.src/lib/visualMatrix.ts:VISUAL_LOCALES,caseKey,expectedCaseKeys,caseCoverageandsummaryMarkdownas pure functions, so the matrix's promise can be tested without a browser.test/:visualJobCeiling.test.ts(new) reads the workflow and the runner;visualMatrix.test.tspins the promised set and the negatives that are pure functions..github/agent/action-allowlist.json: addsactions/cache, the action the job now uses.docs/autonomous-dev.md: thevisualbullet records the cache, the summary and the coverage refusal.The three negatives
ci-ok. The matrix step is unguarded, the runner setsprocess.exitCode = 1on any failure, andvisualis in the aggregateci-okneeds:list inci.yml, whichtest/ciOk.test.tsalready pins againstREQUIRED_JOBS.~/.cache/ms-playwrightand nothing else; the install and matrix steps are unconditional; there is nocontinue-on-error; and a summary row'sokisfailures.length === from— the assertion count taken around that case in this process, not a cache result.checkCoverage()compares recorded keys againstexpectedCaseKeys(): a missing promised case fails with "promised case(s) never ran", a key recorded twice fails with "case key reused", andcaseCoverage()reports missing and duplicated at once, so a duplicate row cannot paper over a dropped page.Red first
At
f1c48ed3, in a clean worktree, with the committed test files byte-identical (sha256082c9b5548d0bc0c5809af65b62fa7ddcf07edf006bbbfe5d893397d2edb6d4dandcac23dbb69cd099965e53996bb89a730ea9b9d555273a8afa4ab4f82bde16554):The ten failures are the new behaviour: no cache step, no summary, no promised set, no coverage check. Four assertions pass on the base by design and are the controls this change must not break: the matrix steps are already unguarded, the runner already sets a red exit code,
visualis already inci-ok's needs, and the base has norestore-keysto remove.At the tip the same two files are 43/43.
Measurement
Filled from this pull request's own runs (the cold number is the first
visualrun at this tip, the warm number a re-run of the same job at the same tip):visualcold: pending this run — gating, ≤ 75 s.visualwarm: pending this run — gating, ≤ 45 s. Baseline: run44251917876at 68 s cold / 55 s.dockeron the same runs: pending — recorded, not gating (target ≤ 60 s cold / ≤ 40 s warm).Ran:
npm run checkat the tip — exit 0, 3277 tests / 3277 pass, build cleannpx tsx --test test/visualJobCeiling.test.ts test/visualMatrix.test.tsat the tip — 43/43f1c48ed3— 33/43, the red run abovenode scripts/risk-gate.mjs --base f1c48ed3— T3, 7 changed filesNot run:
visualmatrix on this host: no Playwright Chromium is installed here; the job's own runs above are the measurement.dockerlocally: the image build, scan and start run in CI.Risk and rollback
runner.osplus the installed Playwright version. Two lockfile states that resolve to the same Playwright version share one browser build, which is what the key promises; a bump misses and downloads (the cold cap). A miss is a full download because there is norestore-keys.actions/cacheis pinned by sha and added to the action allowlist;action-pinsandworkflow-lintjudge it like any other action.expectedCaseKeysin the same change, not to soften the check.Docs and changelog
docs/autonomous-dev.mdupdated (thevisualbullet).Changelog-not-needed: this is CI plumbing users cannot see.