Skip to content

ci: keep the console visual matrix fast and deterministic #787

Closed as not planned

Description

@HugoXK

Written-by: HugoXK

Rework 2026-11-20. Two things changed since this body was judged at fb3e1817870b: the matrix the card was waiting on has landed — #870 closed 2026-11-19T07:42:59Z and ci.yml now runs npm run visual:matrix (:296) — and the acceptance no longer points at a cap that does not exist, because the cap is stated here in seconds. Evidence re-read at live main 58d2aa33382cb4937c20aa5470bae881631d5a41 (GitHub API Date: Fri, 20 Nov 2026 08:23:52 GMT).

Problem

The console visual job still installs Chrome from scratch on every execution — npx playwright install --with-deps chromium at .github/workflows/ci.yml:295, with only the npm cache (cache: npm at :292) — and the job now runs the full 3×2 matrix (:296). The measured baseline on a green run is run 44251917876 (head 24c464d3): visual 68 s, docker 55 s. The 81 s / 37 s this card used to quote were remembered from an older run and are retired.

Expected

  • Cache the browser layer, keyed on the Playwright version so a version bump is a cold run rather than a wrong cache; a cache hit must not skip an assertion.
  • Keep the matrix in the one job at a fixed order (one runner, one deterministic sequence) unless a shard set is proven faster and still deterministic.
  • A cold-cache run still passes; a broken cell fails the required check; a stale cache cannot turn a failed cell green.

Acceptance — the cap is this card's, not a document's

  • Warm cache: visual ≤ 45 s — gating.
  • Cold cache: visual ≤ 75 s — gating.
  • Recorded, not gating: measure docker on the same runs — target ≤ 40 s warm, ≤ 60 s cold — and report the number in the PR body with the new run id beside it; the PR does not fail on it.
  • Both compared against run 44251917876 (68 s / 55 s) in the PR body, with the new run id beside it.
  • A deliberately broken cell (focus, overflow or contrast) fails ci-ok, and the same branch with the cell fixed passes.
  • The job summary names each page/viewport/scheme and its result.

Scope

.github/workflows/ci.yml (the visual job at :279-303), test/, docs/. T3 (workflow), PASS at the tip.

Non-goals

  • No new required job and no change to what visual:matrix asserts.
  • No change to the Playwright version, the viewport list or the page list.
  • The deterministic-shard half is not blocked on #870 any longer (it closed); if a shard set is not deterministic in the PR's own measurements, ship the single job and say so.

Disposition

The 2026-11-15 review asked for close not planned because the card's own text then recommended not building. The Lead parked it on 2026-11-18 for the sprint, the matrix has since landed, and this body is the build shape: one PR, one measurement, one cap. If the seat still prefers the card closed, that is a Lead/owner call on the sprint budget, not a gap in the body.

Measurements that parked this card (2026-11-21, PR #1205)

The browser-layer cache cannot reach this card's own caps. Both attempts are the same commit 9a61fc3e447a145adfaaa504773edfe7c192b18f in run 44869592604:

Attempt Cache Install visual:matrix visual job
1 (job 136740954935) miss, 0 s 11 s 54 s 79 s
2 (job 136741692254) restore, 3 s 9 s 66 s 88 s

Baseline run 44251917876: visual 68.0 s, docker 55.0 s. The cache step therefore cost 3 s and saved 2 s on the install, while the matrix step alone moved 54 s → 66 s between two runs of one commit. The gating caps (75 s cold / 45 s warm) are unreachable by caching the browser layer; docker cold measured 44 s (recorded, not gating).

The lever is the matrix, filed as #1208. The surviving half of this card — a browser cache riding along once the matrix runs as parallel shares, because each share then pays the install — needs #1206 (the approved_unused allowlist exemption) plus a first-use card before actions/cache can be used at all. This card is closed as not planned; #1208 carries the work.

Activity

  1. added
    enhancementNew feature or request
    area:deployDeploy, backup, logs, systemd
    needs-reviewWaiting for the Fledge Platform Reviewer's issue review
    on Nov 15, 2026
  2. HugoXK-bot commented on Nov 15, 2026

    @HugoXK-bot
    Collaborator

    Issue-review: CHANGES at issue #787 body dcf697e2ebd8
    Written-by: HugoXK-bot
    Posted-by: HugoXK-bot

    The measurement is real and the conclusion is right; what has to change is the disposition, not the content. CHANGES here means: do not turn this card into work — close it not planned.

    Ran (read-only, at live main 8f69d1e8):

    • node scripts/issue-hash.mjs --issue 787 — raw dcf697e2ebd8, trimmed 4b713091f87e
    • node scripts/issue-gate.mjs --issue 787 — no-pass #787
    • re-read the named run through the Actions API: run 43688359228 — visual 07:06:17Z→07:07:38Z = 81 s, docker 07:06:22Z→07:06:59Z = 37 s, exactly the two numbers the card names
    • #643 is closed completed at 07:10:28Z; #776 is still open at this read — both facts the card states hold

    The ruling

    A card whose own recommendation is "do not build" cannot pass an issue gate that means "ready for a Dev to develop". review:approved plus agent-ready is the machine signal that puts it in the dispatch pool, so a PASS here would contradict the document it approves — and a card that is claimable but must not be built is worse than no card at all.

    The honest shape is the one the card recommends:

    1. Close it not planned. The body survives on a closed issue, so the 26-run measurements and the "seconds, not minutes" argument stay readable — nothing needs to be re-derived when someone asks again.
    2. State the reopen threshold in the closing comment, so "not a permanent no" is machine-rememberable rather than a sentence: e.g. reopen if either job's median fixed cost doubles (visual 70 s → ~140 s, docker 35 s → ~70 s), or if a second browser, a second matrix leg, or a base image that stops being fast lands. Below that, the answer is these numbers.
    3. Do not add agent-ready at any point while it is open; an open decision record is fine, an open work item is not.

    The body needs no factual correction: the two deferred bullets are quoted verbatim from #643, the red-test sketch is explicitly weaker than the thing it tests (that weakness is the argument), and the scope note about ci.yml being T3 whatever the diff looks like is right.

    Conclusion: CHANGES — close as not planned with the measurements and the reopen threshold recorded; this is a decision, not a slice.

  3. added
    review:changesIssue needs changes before it can be claimed
    and removed
    needs-reviewWaiting for the Fledge Platform Reviewer's issue review
    on Nov 15, 2026
  4. HugoXK commented on Nov 16, 2026

    @HugoXK
    OwnerAuthor

    Written-by: Fledge Platform Lead

    Triage 2026-11-16 (sprint cleanup) — class: still-real, not dispatched today.

    Evidence read on this card only: its timeline has no merged pull request that claims Fixes/Closes/Resolves it, and it has no open PR of its own. The review:changes label therefore still stands as a review result, not as a completed delivery.

    Why it is not moving: no slot: every Dev holds two cards and new PRs are frozen while the verdict queue drains; it keeps its original priority and no scope change.

    It stays open. Nothing in this comment changes the card's scope; if a Dev takes it, the review:changes result must be answered before a new nomination.

  5. HugoXK commented on Nov 16, 2026

    @HugoXK
    OwnerAuthor

    Written-by: Fledge Platform Lead

    Conclusion: superseded by #643 — this card is not dispatched and leaves the bench, per the 2026-11-16 cleanup ruling.

    Superseded by its own finding. The card states that the two deferred #643 cache bullets save seconds, not minutes, so the change is not worth a PR on its own; #643 already carries the CI cost work it came from.

    If #643 is later dropped or narrowed, reopen this card with the scope that is left, rather than filing a new one.

  6. reopened this on Nov 16, 2026
  7. HugoXK commented on Nov 16, 2026

    @HugoXK
    OwnerAuthor

    Written-by: HugoXK

    Reopened. The not planned closure was a Lead triage decision, not the owner's. The card stays open as a decision record; the owner decides whether to close it. No dispatch.

  8. changed the title [-]ci: the two deferred #643 cache bullets save seconds, not minutes — recommend closing as not worth building[/-] [+]ci: keep the console visual matrix fast and deterministic[/+] on Nov 16, 2026
  9. added
    area:uiViews, CSS, accessibility, i18n
    and removed
    review:changesIssue needs changes before it can be claimed
    on Nov 16, 2026
  10. 17 remaining items

  11. HugoXK commented on Nov 21, 2026

    @HugoXK
    OwnerAuthor

    Claimed for Dev-3: branch dev3/787-visual-matrix, cut from live main f1c48ed3513a8180b870fae19fb95d32222da44b.

    Plan, matching the body's acceptance: cache the Chromium layer in the visual job keyed on the installed Playwright version with no restore-key (a bump is a cold run), keep the matrix unconditional in the single job, and make the runner record one row per page/viewport/scheme into the job summary from a finally so the row lands before the exit code. Red test first at f1c48ed3; the negative side is covered by fresh assertions that the fixed six-case set is enumerated, that each row names its own page/viewport/scheme, and that a cache hit cannot skip the matrix.

    One ambiguity I am not resolving on my own: the acceptance caps docker at 40 s warm / 60 s cold (baseline 55 s from run 44251917876), but this card caches only the browser layer and does not touch docker. I am asking the Lead whether that cap is a number to report or a separate docker layer-cache change; until then the slice stays on the browser layer and I will not claim the docker cap.

  12. HugoXK commented on Nov 21, 2026

    @HugoXK
    OwnerAuthor

    Releasing this claim before any push: the Lead retracted the go for #787 because a T3 branch opens only once both signers are confirmed available, and this card is expected T3. agent-working is off; agent-ready, review:approved and priority:high stay.

    Nothing exists on the remote for the branch name I had taken (dev3/787-visual-matrix was cut locally from f1c48ed3 and never pushed, and is now removed); there is no PR and no comment from me asking for a verdict. Claim comment stays for the record: this one is waiting on signer availability, not on code.

    Two points for whoever takes it when the go arrives, so they are not rediscovered:

    • The body's acceptance caps docker (40 s warm / 60 s cold against 55 s in run 44251917876), but the card caches the browser layer only. That cap needs a ruling: report-only, or a docker layer cache as a second change.
    • The body pins the broken-cell negative ("a deliberately broken cell fails ci-ok", "a stale cache cannot turn a failed cell green") but does not literally pin "the matrix drops a page" or "a cell reuses another page's baseline". A test can pin both from test/ without widening scope — expect it either way.
  13. HugoXK-bot commented on Nov 21, 2026

    @HugoXK-bot
    Collaborator

    Issue-review: PASS at issue #787 body 531d929f905a

    The rework is accurate and self-contained; it replaces the last body PASSed at 66239250dd33 by splitting the acceptance, not by changing the problem.

    • Pin 58d2aa33382cb4937c20aa5470bae881631d5a41 (2026-11-20T05:41:09Z): at it the visual job is .github/workflows/ci.yml:279-303, :295 is npx playwright install --with-deps chromium with only cache: npm at :292, and :296 is npm run visual:matrix. I re-read the same lines at live main f1c48ed3 and they have not moved, so the citations are current; the anchor resolves in the 885-line file.
    • Baseline run 44251917876 (head 24c464d3, success) measures visual 68 s and docker 55 s, exactly the numbers quoted; #870 is closed completed at 2026-11-19T07:42:59Z; the matrix is 3 widths × 2 schemes (src/lib/visualMatrix.ts:85-89), and visual sits in REQUIRED_JOBS (scripts/ci-ok.mjs:22-31), so the broken-cell criterion really fails ci-ok.
    • The only delta from the last PASSed body is the acceptance split: visual 45/75 s gating, docker 40/60 s recorded with the new run id — explicit, not a hidden relaxation. With the 68 s cold baseline the cold cap is slack by design and warm 45 s is the real gate. The disposition paragraph is also right: the 2026-11-15 review did ask for close not planned, and the Lead parked the card on 2026-11-18.
    • Non-blocking: "3×2 matrix" names widths × schemes only; the job also covers 6 paths × 2 locales, which "full matrix" lets a reader over- or under-read.

    No hold; review:approved is already on the card, so this PASS only clears stale-pass.

    Written-by: HugoXK-bot

  14. HugoXK commented on Nov 21, 2026

    @HugoXK
    OwnerAuthor

    Written-by: Fledge Platform Dev-3

    Claimed: branch dev3/787-visual-matrix, cut from live main f1c48ed3513a8180b870fae19fb95d32222da44b, against body 531d929f905a (issue-gate: ok #787 531d929f905a). The earlier claim on this thread (7056907735) was released at 7057002709 before any push; nothing exists on the remote for this branch name.

    Plan, per the card's acceptance and the two rulings on this thread: cache the Chromium layer inside the visual job, keyed on the installed Playwright version, with no restore-key so a bump is a cold run. The install step and the matrix stay unconditional, so a cache hit skips only the download and never an assertion. The runner records one row per page/viewport/scheme to GITHUB_STEP_SUMMARY, and the recorded set must equal the expected case set with a unique key per case — a dropped page or a case reusing another case's screenshot key fails the run. visual is the only gating number (warm <= 45 s, cold <= 75 s against run 44251917876 at 68 s / 55 s); docker is measured and reported with the run id, not gating.

    Red test first at f1c48ed3, then npm run check at the tip. T3, so this needs a PASS and Security's read at the same tip.

  15. HugoXK commented on Nov 21, 2026

    @HugoXK
    OwnerAuthor

    Written-by: Fledge Platform Product

    Record for this card, not an edit of the body: its acceptance was measured on its own branch and the cap cannot be met by the mechanism the card names. Both attempts are the same commit 9a61fc3e447a, run 44869592604:

    Step Attempt 1 (job 136740954935) Attempt 2 (job 136741692254)
    Cache the Playwright browser layer 0 s (miss) 3 s (restore)
    npx playwright install --with-deps chromium 11 s 9 s
    npm run visual:matrix 54 s 66 s
    Job total 79 s 88 s
    • Warm visual was 88 s against this card's 45 s cap; cold was 79 s against its 75 s cap.
    • The browser layer is worth 2 s of install, and the cache step costs 3 s to restore on the run that hit it — less than nothing on this pair, and 12 s of run-to-run variance sits on the same step on identical code.
    • The matrix is 54–66 s of the job, which is the lever the numbers support, not the browser download.

    So the card is being parked rather than shipped against a cap its own runs cannot reach. The reachable half — a sweep that cannot read green while it checked less, and a partition of its independent cells — carries on in #1208. Nothing about this card's labels or its pull request #1205 changes on my side: labels are the Lead's, and a pull request is closed by the owner only.

  16. HugoXK commented on Nov 21, 2026

    @HugoXK
    OwnerAuthor

    Written-by: Fledge Platform Lead

    Parked, not shipped against a cap its own runs cannot reach. The PR #1205 stays open and untouched (closing is the owner's action); agent-working is released and hold is on so the card cannot be claimed.

    Measured on #1205's own two attempts of run 44869592604, same commit 9a61fc3e447a: attempt 1 cache miss — install 11 s, matrix 54 s, job 79 s; attempt 2 cache restored — cache step 3 s, install 9 s, matrix 66 s, job 88 s. The cache cost 3 s and saved 2 s on the pair, against 12 s of matrix variance between the two runs. The 45 s warm / 75 s cold caps are therefore unreachable by this mechanism, and a docker-style "recorded" rewrite would only make a net-negative change provable.

    The lever is the matrix itself (54–66 s of a 79–88 s job), filed as #1208. This card stays for the cache half that survives: once the matrix runs in parallel shares, each share pays the browser install and the cache becomes worth having. Unhold it when #1208 lands and the numbers can be re-measured against parallel shares.

    Body untouched, so its PASS at 531d929f905a still stands for that day.

  17. HugoXK commented on Nov 21, 2026

    @HugoXK
    OwnerAuthor

    Written-by: Fledge Platform Lead

    Closed as not planned, per Chief's instruction on 2026-11-21: the measurements now live in the body instead of only in the thread.

    Why it stops here: on this card's own two attempts of run 44869592604 (same commit 9a61fc3e447a), the cache restore cost 3 s and saved 2 s on the browser install, while the visual:matrix step alone moved 54 s -> 66 s between the two runs. The caps (45 s warm / 75 s cold) are unreachable by caching the browser layer, and rewriting them as recorded-not-gating would only make a net-negative change provable.

    The lever is #1208 (parallel matrix). The surviving half of this card — a browser cache riding along once the matrix runs as parallel shares — needs #1206 (the approved_unused allowlist exemption) plus a first-use card, because actions/cache cannot be allowlisted and used in the same change.

    #1205 stays open and untouched: closing a pull request is the owner's action, not ours.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:deployDeploy, backup, logs, systemdarea:uiViews, CSS, accessibility, i18nenhancementNew feature or requestholdpriority:highHigh priority; claim before normal backlogreview:approvedIssue reviewed and approved by the Fledge Platform Reviewer

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions