Skip to content

upstream: playwright-cli c2d8d19 conflicts with our test port #18

Open

Description

@github-actions

microsoft/playwright-cli b85c7a7 -> c2d8d19

Verdict: blocked

  • tests/integration/core.spec.ts: 3 conflict(s) between our changes and upstream changes

Commits

Mapped files (three-way merge)

file result
tests/integration/core.spec.ts conflict (3)

Files we rewrote (upstream patch, for reading)

.github/workflows/ci.yml (modified)

@@ -16,7 +16,7 @@ jobs:
     steps:
     - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
     - name: Use Node.js 20
-      uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
+      uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
       with:
         node-version: '20'
         cache: 'npm'

CLAUDE.md (modified)

@@ -1,3 +1,22 @@
+## Build and test
+
+The CLI source lives in `microsoft/playwright`, in `packages/playwright-core/src/tools/`. This repo
+ships `@playwright/cli`: `playwright-cli.js` runs the CLI of the `playwright-core` alpha that
+`package.json` pins, and adds the update check, with `skillCheck.js`. `skills/playwright-cli/` is a
+copy of the skill that `playwright-cli install --skills` writes, and `node scripts/update.js`
+refreshes it.
+
+```bash
+npm ci
+npm test                                 # tests/integration.spec.ts
+npx playwright test -g "open data URL"
+```
+
+There is no build step and no lint. Tests are in `tests/integration.spec.ts`. They run
+`playwright-cli.js` with `runCli()` in the output directory of the test, and the `npmRegistry`
+fixture serves a fake registry for the update check. Add a case to the test whose setup is closest
+instead of a new test.
+
 ## Commit Convention
 
 Semantic commit messages: `label(scope): description`
@@ -28,3 +47,19 @@ EOF
 
 Never add Co-Authored-By agents in commit message.
 Branch naming for issue fixes: `fix-<issue-number>`
+
+## Rolling Playwright and preparing a release
+
+See [.claude/skills/dev/roll.md](.claude/skills/dev/roll.md) and [.claude/skills/dev/release.md](.claude/skills/dev/release.md).
+
+## Bots
+
+Labels start GitHub Actions workflows that run the Copilot CLI and open pull requests as the
+`microsoft-playwright-automation` app:
+
+- `needs-triage`, `needs-triage-mac`, `needs-triage-win` on an issue: [triage.yml](.github/workflows/triage.yml) reproduces and fixes it, with the skill in [.claude/skills/playwright-triage/](.claude/skills/playwright-triage/SKILL.md). A fix in `microsoft/playwright` opens as a draft PR there.
+- `needs-revision`, `needs-revision-mac`, `needs-revision-win` on a PR that triage opened here: [revision.yml](.github/workflows/revision.yml) addresses the maintainers' review comments.
+- `needs-roll` on an issue: [roll.yml](.github/workflows/roll.yml) rolls Playwright to `playwright@next`, as the roll skill says.
+- `needs-release` on an issue: [release.yml](.github/workflows/release.yml) opens the `chore: mark v<version>` PR with the release notes as its description, as the release skill says. Copy the notes into the release draft.
+
+Apply a label again to start another run. [bots.md](https://github.com/microsoft/playwright/blob/main/.github/workflows/bots.md) in `microsoft/playwright` describes triage and revision.

README.md (modified)

@@ -185,9 +185,11 @@ playwright-cli keyup <key>              # press a key up on the keyboard
 playwright-cli mousemove <x> <y>        # move mouse to a given position
 playwright-cli mousedown [button]       # press mouse down
 playwright-cli mouseup [button]         # press mouse up
-playwright-cli mousewheel <dx> <dy>     # scroll mouse wheel
+playwright-cli mousewheel <dx> <dy>     # scroll by pixels; negative dy scrolls up

+The element under the mouse pointer scrolls, so use mousemove to move over it first.
+

Save as

@@ -203,6 +205,7 @@ playwright-cli pdf --filename=page.pdf  # save pdf with specific filename
```bash
playwright-cli tab-list                 # list all tabs
playwright-cli tab-new [url]            # create a new tab
+playwright-cli tab-new https://example.com --isolated-context=user-b # use an isolated context
playwright-cli tab-close [index]        # close a browser tab
playwright-cli tab-select <index>       # select a browser tab

@@ -218,6 +221,7 @@ playwright-cli cookie-list [--domain] # list cookies
playwright-cli cookie-get # get a cookie
playwright-cli cookie-set # set a cookie
playwright-cli cookie-delete # delete a cookie
+playwright-cli cookie-delete --domain=example.com # delete only on this domain
playwright-cli cookie-clear # clear all cookies

LocalStorage

@@ -286,6 +290,16 @@ playwright-cli highlight --hide # hide highlight on a specific element
playwright-cli highlight --hide # hide all page highlights


+Tracing requires an open browser. Open it before starting the trace, then use `goto` to
+record navigation:
+
+```bash
+playwright-cli open
+playwright-cli tracing-start
+playwright-cli goto https://example.com
+playwright-cli tracing-stop
+```
+
### WebMCP

Some pages register their own tools for agents through the experimental WebMCP API. When available,
@@ -357,6 +371,16 @@ playwright-cli find --regex "\\$[0-9]+\\.[0-9]{2}"
playwright-cli find --regex "/sign (in|up)/i"

+### Raw output
+
+The global --raw option strips page status and generated code from the output. It keeps
+the result value, errors, and the page snapshot (or the link to the snapshot file).
+Commands that don't produce output return nothing.
+
+bash +playwright-cli --raw eval "JSON.stringify(performance.timing)" | jq '.loadEventEnd - .navigationStart' +
+

Targeting elements

By default, use refs from the snapshot to interact with page elements.


### package.json (modified)

```diff
@@ -1,6 +1,6 @@
 {
   "name": "@playwright/cli",
-  "version": "0.1.22",
+  "version": "0.1.23",
   "description": "Playwright CLI",
   "repository": {
     "type": "git",
@@ -18,12 +18,12 @@
     "test": "playwright test"
   },
   "devDependencies": {
-    "@playwright/test": "1.64.0-alpha-1790635538000",
+    "@playwright/test": "1.65.0-alpha-2026-10-30",
     "@types/node": "^25.2.1"
   },
   "dependencies": {
-    "playwright": "1.64.0-alpha-1790635538000",
-    "playwright-core": "1.64.0-alpha-1790635538000"
+    "playwright": "1.65.0-alpha-2026-10-30",
+    "playwright-core": "1.65.0-alpha-2026-10-30"
   },
   "bin": {
     "playwright-cli": "playwright-cli.js"

playwright-cli.js (modified)

@@ -19,6 +19,7 @@
 
 const fs = require('fs');
 const path = require('path');
+const { spawn } = require('child_process');
 
 const { program } = require('playwright-core/lib/tools/cli-client/program');
 const coreBundle = require('playwright-core/lib/coreBundle');
@@ -28,6 +29,7 @@ const { checkInstalledSkills, frame } = require('./skillCheck');
 const packageJson = require('./package.json');
 
 const ONE_DAY_MS = 24 * 60 * 60 * 1000;
+const UPDATE_CHECK_COMMANDS = new Set(['open', 'attach', 'list', 'install', 'install-browser']);
 
 main();
 
@@ -40,13 +42,17 @@ async function checkForUpdates() {
   if (process.env.NO_UPDATE_NOTIFIER || process.env.CI)
     return;
 
+  // The check spawns npm, so keep its cost to the commands that start a workflow.
+  const command = process.argv.slice(2).find(arg => !arg.startsWith('-'));
+  if (!command || !UPDATE_CHECK_COMMANDS.has(command))
+    return;
+
   const cache = readCache();
   const stale = !cache || (Date.now() - cache.lastCheck) > ONE_DAY_MS;
   if (!stale)
     return;
   writeCache({ lastCheck: Date.now() });
 
-  const command = process.argv.slice(2).find(arg => !arg.startsWith('-'));
   if (command !== 'install')
     checkInstalledSkills();
 
@@ -55,22 +61,38 @@ async function checkForUpdates() {
     printNotice(packageJson.version, latest);
 }
 
-async function fetchLatestVersion() {
-  try {
-    const controller = new AbortController();
-    const timeout = setTimeout(() => controller.abort(), 1500);
-    try {
-      const res = await fetch(`https://registry.npmjs.org/${packageJson.name}/latest`, { signal: controller.signal });
-      if (!res.ok)
-        return undefined;
-      const json = await res.json();
-      return typeof json.version === 'string' ? json.version : undefined;
-    } finally {
+// Asks npm, so that the user's registry, proxy and auth configuration is respected.
+/** @returns {Promise<string | undefined>} */
+function fetchLatestVersion() {
+  return new Promise(resolve => {
+    const child = spawn(`npm view ${packageJson.name} version --json --no-update-notifier`, {
+      shell: true,
+      stdio: ['ignore', 'pipe', 'ignore'],
+    });
+    let stdout = '';
+    child.stdout.on('data', data => stdout += data);
+    // On Windows, killing the shell leaves npm running: stop listening to it
+    // so that the CLI does not wait for npm before exiting.
+    const timeout = setTimeout(() => {
+      child.kill();
+      child.unref();
+      child.stdout.destroy();
+      resolve(undefined);
+    }, 1500);
+    child.on('error', () => {
       clearTimeout(timeout);
-    }
-  } catch {
-    return undefined;
-  }
+      resolve(undefined);
+    });
+    child.on('close', code => {
+      clearTimeout(timeout);
+      try {
+        const version = JSON.parse(stdout);

Other upstream files

  • .claude/skills/playwright-triage/SKILL.md (added)
  • .github/actions/setup-triage-runner/action.yml (added)
  • .github/workflows/publish.yml (modified)
  • .github/workflows/release.yml (added)
  • .github/workflows/revision.yml (added)
  • .github/workflows/roll.yml (added)
  • .github/workflows/triage.yml (added)
  • package-lock.json (modified)
  • skills/playwright-cli/SKILL.md (modified)
  • skills/playwright-cli/references/storage-state.md (modified)
  • skills/playwright-cli/references/test-generation.md (modified)
  • skills/playwright-cli/references/tracing.md (modified)

core.spec.ts.merged

/**
 * Copyright (c) Microsoft Corporation.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

import fs from 'fs';
import http from 'http';
import path from 'path';
<<<<<<< tests/integration/core.spec.ts
import { test, expect } from 'patchright/test';

import { runCli } from '../fixtures';
=======
import { spawn } from 'child_process';
import { test as baseTest, expect } from '@playwright/test';
import type { AddressInfo } from 'net';

type CliResult = {
  output: string;
  error: string;
  exitCode: number | null;
};

type NpmRegistry = {
  url: string;
  status: number;
  hang: boolean;
  requests: string[];
};

const test = baseTest.extend<{ npmRegistry: NpmRegistry }>({
  npmRegistry: async ({}, use) => {
    const registry: NpmRegistry = { url: '', status: 200, hang: false, requests: [] };
    const server = http.createServer((req, res) => {
      registry.requests.push(req.url!);
      if (registry.hang)
        return;
      res.statusCode = registry.status;
      res.setHeader('content-type', 'application/json');
      res.end(JSON.stringify({
        'name': '@playwright/cli',
        'dist-tags': { latest: '99.0.0' },
        'versions': { '99.0.0': { name: '@playwright/cli', version: '99.0.0' } },
      }));
    });
    await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve));
    registry.url = `http://127.0.0.1:${(server.address() as AddressInfo).port}`;
    await use(registry);
    server.closeAllConnections();
    await new Promise(resolve => server.close(resolve));
  },
});

// Enables the update check and points npm at the test registry, isolated from
// the npm config of the machine running the tests.
function updateCheckEnv(npmRegistry: NpmRegistry): Record<string, string> {
  return {
    'CI': '',
    'NO_UPDATE_NOTIFIER': '',
    'npm_config_registry': npmRegistry.url,
    'npm_config_@playwright:registry': '',
    'npm_config_userconfig': test.info().outputPath('user-npmrc'),
    'npm_config_globalconfig': test.info().outputPath('global-npmrc'),
    'npm_config_cache': test.info().outputPath('npm-cache'),
  };
}

async function runCli(args: string[], env: Record<string, string> = {}): Promise<CliResult> {
  const cliPath = path.join(__dirname, '../playwright-cli.js');

  return new Promise<CliResult>((resolve, reject) => {
    let stdout = '';
    let stderr = '';

    const childProcess = spawn(process.execPath, [cliPath, ...args], {
      env: {
        ...process.env,
        PLAYWRIGHT_CLI_INSTALLATION_FOR_TEST: test.info().outputPath(),
        ...env,
      },
      cwd: test.info().outputPath(),
    });

    childProcess.stdout?.on('data', (data) => {
      stdout += data.toString();
    });

    childProcess.stderr?.on('data', (data) => {
      stderr += data.toString();
    });

    childProcess.on('close', (code) => {
      resolve({
        output: stdout.trim(),
        error: stderr.trim(),
        exitCode: code,
      });
    });

    childProcess.on('error', reject);
  });
}
>>>>>>> upstream tests/integration.spec.ts

test('open data URL', async ({}) => {
  expect(await runCli(['open', 'data:text/html,hello', '--persistent'])).toEqual(expect.objectContaining({
    output: expect.stringContaining('hello'),
    exitCode: 0,
  }));

  expect(await runCli(['delete-data'])).toEqual(expect.objectContaining({
    output: expect.stringContaining('Deleted user data for'),
    exitCode: 0,
  }));
});

test('warns when installed skill is out of date', async ({ npmRegistry }) => {
  expect(await runCli(['install', '--skills'], { NO_UPDATE_NOTIFIER: '1' })).toEqual(expect.objectContaining({
    exitCode: 0,
  }));

  const skillFile = path.join(test.info().outputPath(), '.claude', 'skills', 'patchright-cli', 'SKILL.md');
  fs.appendFileSync(skillFile, 'x');

  const env = updateCheckEnv(npmRegistry);
  expect(await runCli(['list'], env)).toEqual(expect.objectContaining({
    error: expect.stringContaining('does not match the tool version'),
  }));

  expect(await runCli(['list'], env)).toEqual(expect.objectContaining({
    error: expect.not.stringContaining('does not match the tool version'),
  }));
});

<<<<<<< tests/integration/core.spec.ts
test('warns when an installed reference file drifts', async ({}) => {
  expect(await runCli(['install', '--skills'], { NO_UPDATE_NOTIFIER: '1' })).toEqual(expect.objectContaining({
    exitCode: 0,
  }));

  const reference = path.join(test.info().outputPath(), '.claude', 'skills', 'patchright-cli', 'references', 'stealth.md');
  fs.appendFileSync(reference, 'x');

  expect(await runCli(['--help'], { CI: '', NO_UPDATE_NOTIFIER: '' })).toEqual(expect.objectContaining({
    error: expect.stringContaining('does not match the tool version'),
  }));
});

test('does not warn when installed skill only differs in line endings', async ({}) => {
=======
test('does not warn when installed skill only differs in line endings', async ({ npmRegistry }) => {
>>>>>>> upstream tests/integration.spec.ts
  expect(await runCli(['install', '--skills'], { NO_UPDATE_NOTIFIER: '1' })).toEqual(expect.objectContaining({
    exitCode: 0,
  }));

  const skillFile = path.join(test.info().outputPath(), '.claude', 'skills', 'patchright-cli', 'SKILL.md');
  fs.writeFileSync(skillFile, fs.readFileSync(skillFile, 'utf8').replace(/\n/g, '\r\n'));

  expect(await runCli(['list'], updateCheckEnv(npmRegistry))).toEqual(expect.objectContaining({
    error: expect.not.stringContaining('does not match the tool version'),
  }));
});

test('caches the update check in the default registry directory', async ({ npmRegistry }) => {
  // Redirect the home/cache directories so the real user cache is untouched, and
  // leave PATCHRIGHT_CLI_HOME empty so the default path is used.
  const home = test.info().outputPath('home');
  fs.mkdirSync(home, { recursive: true });
  const env = {
<<<<<<< tests/integration/core.spec.ts
    CI: '',
    NO_UPDATE_NOTIFIER: '',
    PATCHRIGHT_CLI_HOME: '',
=======
    ...updateCheckEnv(npmRegistry),
    PLAYWRIGHT_CLI_INSTALLATION_FOR_TEST: '',
>>>>>>> upstream tests/integration.spec.ts
    HOME: home,
    USERPROFILE: home,
    XDG_CACHE_HOME: path.join(home, '.cache'),
    LOCALAPPDATA: path.join(home, 'AppData', 'Local'),
  };

  expect(await runCli(['list'], env)).toEqual(expect.objectContaining({ exitCode: 0 }));

  const found: string[] = [];
  const walk = (dir: string) => {
    for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
      const full = path.join(dir, entry.name);
      if (entry.isDirectory())
        walk(full);
      else if (entry.name === 'cli-update-check.json')
        found.push(full);
    }
  };
  walk(home);

  expect(found).toHaveLength(1);
  expect(JSON.parse(fs.readFileSync(found[0], 'utf8')).lastCheck).toEqual(expect.any(Number));
});

test('update check asks npm for the latest version on setup commands only', async ({ npmRegistry }) => {
  const env = updateCheckEnv(npmRegistry);
  expect(await runCli(['--version'], env)).toEqual(expect.objectContaining({
    error: '',
    exitCode: 0,
  }));
  expect(npmRegistry.requests).toEqual([]);

  expect(await runCli(['list'], env)).toEqual(expect.objectContaining({
    error: expect.stringContaining('Update available for @playwright/cli'),
    exitCode: 0,
  }));
  expect(npmRegistry.requests).toEqual(['/@playwright%2fcli']);
});

test('update check on install does not warn about the skills being installed', async ({ npmRegistry }) => {
  expect(await runCli(['install', '--skills'], { NO_UPDATE_NOTIFIER: '1' })).toEqual(expect.objectContaining({
    exitCode: 0,
  }));

  const skillFile = path.join(test.info().outputPath(), '.claude', 'skills', 'playwright-cli', 'SKILL.md');
  fs.appendFileSync(skillFile, 'x');

  const result = await runCli(['install', '--skills'], updateCheckEnv(npmRegistry));
  expect(result.exitCode).toBe(0);
  expect(result.error).toContain('Update available for @playwright/cli');
  expect(result.error).not.toContain('does not match the tool version');
  expect(npmRegistry.requests).toEqual(['/@playwright%2fcli']);
});

test('update check uses the scoped registry from project .npmrc', async ({ npmRegistry }) => {
  fs.writeFileSync(test.info().outputPath('package.json'), '{}');
  fs.writeFileSync(test.info().outputPath('.npmrc'), `@playwright:registry=${npmRegistry.url}/scoped/`);

  expect(await runCli(['list'], updateCheckEnv(npmRegistry))).toEqual(expect.objectContaining({
    error: expect.stringContaining('Update available for @playwright/cli'),
    exitCode: 0,
  }));
  expect(npmRegistry.requests).toEqual(['/scoped/@playwright%2fcli']);
});

test('update check failure is silent', async ({ npmRegistry }) => {
  npmRegistry.status = 401;
  expect(await runCli(['list'], updateCheckEnv(npmRegistry))).toEqual(expect.objectContaining({
    error: '',
    exitCode: 0,
  }));
  expect(npmRegistry.requests).toEqual(['/@playwright%2fcli']);
});

test('update check does not wait for a hanging registry', async ({ npmRegistry }) => {
  npmRegistry.hang = true;
  const start = Date.now();
  expect(await runCli(['list'], updateCheckEnv(npmRegistry))).toEqual(expect.objectContaining({
    error: '',
    exitCode: 0,
  }));
  expect(Date.now() - start).toBeLessThan(10000);
});
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    upstream-blockedupstream roll needs a manual port

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions