Repository navigation
ci(e2e): run e2e specs with 3 workers on CI - #1486
Merged
Merged
Conversation
The install-deps step spent ~2m40s on apt every run because the browser cache only stores binaries, not OS packages. The mcr.microsoft.com/playwright image ships both, so the cache, install-deps and install steps go away. --user 1001 matches the runner's uid, which Firefox (nightly) needs. The image tag must match @playwright/test in example/package.json; a mismatch fails fast with Playwright's "Executable doesn't exist" error. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
3 tasks
Contributor
|
Deploy preview for remote-flows ready!
Deployed with vercel-action |
Contributor
|
Deploy preview for adp-cost-calculator ready!
Deployed with vercel-action |
Contributor
📊 Coverage Report⚪ Coverage unchanged
Detailed BreakdownLines Coverage
Statements Coverage
Functions Coverage
Branches Coverage
✅ Coverage check passed |
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The container saved nothing: apt install-deps normally takes 11-33s (the 2m41s seen once was an outlier), while initializing the container costs ~28s, and it adds a manually-bumped image tag. Keep only the worker change. This reverts commit 0db255b. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
jordividaller
approved these changes
Oct 9, 2026
gabrielseco
added a commit
that referenced
this pull request
Oct 9, 2026
* ci(e2e): only surface failures in Playwright CI output Drop DEBUG=pw:api from the PR, main and nightly e2e jobs: it logged every Playwright action (~4.8k lines per run), including expected waitFor timeouts that read like failures. On CI, use the dot reporter for a compact log plus the github reporter so failures show as PR annotations. The html report is still uploaded, and trace: on-first-retry already records a trace for every failing test since CI retries twice. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * ci(e2e): use the list reporter so each test is named in CI output Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * ci(e2e): run e2e specs with 3 workers on CI (#1486) * ci(e2e): run e2e jobs in the official Playwright container The install-deps step spent ~2m40s on apt every run because the browser cache only stores binaries, not OS packages. The mcr.microsoft.com/playwright image ships both, so the cache, install-deps and install steps go away. --user 1001 matches the runner's uid, which Firefox (nightly) needs. The image tag must match @playwright/test in example/package.json; a mismatch fails fast with Playwright's "Executable doesn't exist" error. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * ci(e2e): run e2e specs with 3 workers on CI (#1487) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * revert: run e2e jobs in the official Playwright container The container saved nothing: apt install-deps normally takes 11-33s (the 2m41s seen once was an outlier), while initializing the container costs ~28s, and it adds a manually-bumped image tag. Keep only the worker change. This reverts commit 0db255b. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
E2E specs run 3 at a time on CI instead of one after another. The e2e job drops from about 2.5 minutes to about 1.5.
Why
workers: process.env.CI ? 1 : undefinedis the line Playwright's scaffold generates, and it has been there since the original "move to playwright" commit. It doesn't look like a deliberate choice. The setup already supports parallel runs:fullyParallel: trueis set.Date.now()names and emails, so specs don't obviously share data.This PR first tried the official Playwright Docker image as well, to skip the apt install. It didn't help. Across 23 recent runs the apt step took 11–33s (the 2m41s that prompted it was a one-off), and starting the container costs about 28s. On top of that, someone would have to bump the image tag by hand. It's reverted here, so only the worker change remains.
What changed
Toggle details
example/playwright.config.ts: CI workers 1 → 3 (ubuntu-latesthas 4 vCPUs). Removed the scaffold's "Opt out of parallel tests on CI" comment, which would now be wrong.undefined= Playwright's default).retry #lines in thelistoutput, not as clean failures.Screenshots
N/A
Related Resources
Testing
Measured E2E job timings:
🤖 Generated with Claude Code
Note
Low Risk
Only affects CI test execution parallelism in the example app; no production or SDK behavior changes.
Overview
CI E2E parallelism: On CI, Playwright now runs with 3 workers instead of 1, so test files can run in parallel again rather than being forced serial on the runner.
The old comment about opting out of parallel tests on CI was removed to match the new setting. Local runs are unchanged (
workersstaysundefinedoutside CI).Reviewed by Cursor Bugbot for commit 0949451. Bugbot is set up for automated code reviews on this repo. Configure here.