Playwright 1.65 Quietly Changed What Agents See

Same page, same modal open, both labeled “Playwright 1.65”. The MCP server bundled with my tests’ Playwright 1.65.0 gave the agent 23 elements it could act on. Playwright MCP, installed the way its README says, gave it 3.

That gap came from a pre-release Playwright build inside the MCP server, and it landed in the same week as the Playwright 1.65 aria snapshot changes. Both alter what a test, and every AI agent reading Playwright’s output, perceives on a page that didn’t change. Pin the Playwright version your agent runs on the same way you pin the model, and diff the agent’s view of your key pages before you take any upgrade.

Playwright 1.65.0 shipped on November 5, 2026. Its breaking-changes section is honest about the two big items. What it can’t tell you is which of your pages they touch, and it doesn’t mention that the default Playwright MCP install shipped on a different build than the one your tests run. In an earlier post I argued that you should anchor your agents to the accessibility tree because the tree only changes when the meaning of the UI changes. That still holds for your app. But the tree an agent reads is Playwright’s rendering of it, and the renderer is a dependency with a version number.

What are the Playwright 1.65 aria snapshot changes?

There are two documented changes and one undocumented one. Text the browser doesn’t render no longer counts toward accessible names or aria snapshots. Actions now wait for elements to stop being inert. And AI-mode snapshots now print container names they used to drop. On my fixture page, that changed 2 lines of the default snapshot and 5 lines of the AI-mode snapshot an agent reads.

Unrendered text leaves accessible names

The release notes say text “that is not assigned to a slot or text directly inside a closed <details>, is no longer part of accessible names and aria snapshots.” The fix is PR #43150, and it covers exactly those two cases. Here is the default-mode diff for my page:

    - listitem:
      - text: Priya Raman
      - button "RemoveRemove member"
      - button "Remove member"
- region "FAQ":
  - heading "FAQ" [level=2]
  - group: Can I restore a deleted workspace? Yes, within 30 days from the audit log.
  - group: Can I restore a deleted workspace?

The button is a custom element with role="button". Its shadow root renders “Remove member”, and its light-DOM text “Remove” is a no-JS fallback that never gets slotted. Users see “Remove member”. Through 1.64, Playwright glued both strings together. Any web-component design system that keeps light-DOM fallback text for no-JS rendering hits this, so check yours.

The <details> line matters more for agents than for tests. Through 1.64, an agent “saw” the answer to a collapsed FAQ that no human could read without clicking. An agent asked to verify the restore policy could report it as visible without ever expanding the accordion.

Actions wait for elements to stop being inert

The second change (PR #43851) adds a Not Inert actionability check to click, check, tap, fill, clear, and selectOption. The worst pre-1.65 case was fill(): on 1.64.1 it returned success in a few milliseconds on the inert textbox and typed nothing, leaving the value "". On 1.65.0 it waits, and with the demo’s 3-second action timeout it fails with element is inert in the call log. For click(), the error text is the interesting part, because for an agent the error text is the next prompt:

# 1.64.1
2 × waiting for element to be visible, enabled and stable
  - element is visible, enabled and stable
  - scrolling into view if needed
  - done scrolling
  - <section aria-labelledby="danger-h">…</section> intercepts pointer events
- retrying click action

# 1.65.0
2 × waiting for element to be visible, enabled and stable
  - element is inert
- retrying click action

On 1.64, the error blamed a <section> for intercepting pointer events. A self-healing agent reads that and reaches for force: true or a different selector. On 1.65 the error names the cause, so the agent has a fair chance of looking for whatever un-inerts the region, which here is the “I understand this is permanent” checkbox.

Changes the release notes don’t list

Diffing the AI-mode snapshot also turned up region → region "Members" on all three landmarks. That’s PR #43810, “preserve identifying container names in AI snapshots”, merged three days before the release and not listed under breaking changes. It’s an improvement. It’s also three changed lines in every agent transcript that touches a named landmark, so any prompt, saved plan, or golden transcript that quotes the old snapshot is now stale. The compare between 1.64.1 and 1.65.0 holds eight more fix(aria) commits, covering whitespace-only name sources, reflected label references, and YAML quoting of ~, .inf, and .nan. Expect diffs on pages I didn’t test.

Why does getByRole stop matching after a Playwright upgrade?

Because the accessible name Playwright computes is the name your locator has to match, and 1.65 computes some names differently. If a human or an agent copied the name from a 1.64 snapshot, the locator encodes Playwright’s old computation, not your UI:

for (const name of ['RemoveRemove member', 'Remove member']) {
  const count = await page.getByRole('button', { name, exact: true }).count();
  console.log(`${name}: ${count}`);
}
// 1.64.1 → RemoveRemove member: 1, Remove member: 0
// 1.65.0 → RemoveRemove member: 0, Remove member: 1

Agents make this worse in a specific way. A human writing role-based locators types the label they see on screen, “Remove member”, and was simply wrong on 1.64. A generator agent copies the string from the snapshot it was handed, so it was “right” on 1.64 and breaks on 1.65. Every agent-generated locator that captured a Playwright naming quirk breaks the day you upgrade, and in CI that looks like a wave of element(s) not found failures in exactly the code nobody wrote by hand.

Does a Playwright upgrade change what an AI agent sees?

Yes, and in November 2026 it changed it in the opposite direction from your tests. The Playwright MCP README installs @playwright/mcp@latest. The current latest, 0.0.84 (October 31), doesn’t run on Playwright 1.65.0. Its package.json pins playwright-core to 1.65.0-alpha-2026-10-30:

$ npm view @playwright/mcp@0.0.84 dependencies
{
  playwright: '1.65.0-alpha-2026-10-30',
  'playwright-core': '1.65.0-alpha-2026-10-30'
}

That alpha still contains a change that was reverted on November 2, three days before 1.65.0 shipped. The change treated inert elements, and everything outside an open modal <dialog>, as hidden. So the agent sees controls without refs it can act on, while your 1.65.0 tests see the full page. I counted actionable refs in the real browser_snapshot output from both servers:

With the onboarding dialog open, @playwright/mcp@latest gave the agent three things it could act on: the dialog, its paragraph, and “Skip tour”. The rest of the page shows up as unnamed textbox and checkbox nodes without refs. An agent told to “fill in the workspace name” has nothing to target and will improvise. Your test suite on 1.65.0, meanwhile, can still reach every control and passes.

Expect a later MCP release to fix it, since a new MCP release on a final core is the normal cadence. That’s the point, though: nobody announced the mismatch, and the only way to see it was to compare the two views. It’s the same thing that happened when WebMCP tools appeared in the agent’s tool list by default: teams on @latest picked up a behavior change they never reviewed.

How do you pin Playwright for tests, MCP, and CLI together?

Install an exact version once and run every agent surface from that install. Since 1.62, Playwright bundles the MCP server and CLI, so npx playwright mcp and npx playwright cli run on whatever version your package.json resolves. Your agent and your tests can’t drift apart unless you let them.

{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": ["playwright", "mcp"]
    }
  }
}

I’m opinionated about the second step. @latest in an agent config is a dependency that upgrades itself at session start, outside your lockfile, so no pull request ever shows the change. You’d never let a model version float like that. The Playwright build is the part of the prompt that describes the page, so it deserves the same discipline.

How do you diff aria snapshots across Playwright versions?

Install both versions side by side, capture the AI-mode snapshot of the same pages with each, strip the refs, and git diff the results. Refs come from a per-snapshot counter and aren’t stable, so the script removes them and records how many there were. Losing refs is how you spot an agent going blind on part of a page.

// node agent-view.mjs <playwright-install-dir> <out-dir> <url>...
import { createRequire } from 'node:module';
import { mkdirSync, writeFileSync } from 'node:fs';
import path from 'node:path';

const [installDir, outDir, ...urls] = process.argv.slice(2);
const { chromium } = createRequire(path.resolve(installDir, 'package.json'))('playwright');
const browser = await chromium.launch();
mkdirSync(outDir, { recursive: true });
for (const url of urls) {
  const page = await browser.newPage();
  await page.goto(url);
  const yaml = await page.ariaSnapshot({ mode: 'ai' });
  const refs = (yaml.match(/\[ref=/g) || []).length;
  const body = `# actionable refs: ${refs}\n` + yaml.replace(/ \[ref=\w+\]/g, '') + '\n';
  const { pathname, hash } = new URL(url);
  writeFileSync(path.join(outDir, pathname.split('/').pop() + hash.replace('#', '_') + '.yml'), body);
  await page.close();
}
await browser.close();

The wrapper installs each version into its own folder, so your project’s node_modules stays untouched:

#!/usr/bin/env bash
# usage: ./agent-view-diff.sh 1.64.1 1.65.0 <url>...
set -euo pipefail
OLD=$1; NEW=$2; shift 2
for v in "$OLD" "$NEW"; do
  npm install --silent --prefix ".pw/$v" "playwright@$v"
  (cd ".pw/$v" && npx playwright install chromium >/dev/null)
  node agent-view.mjs ".pw/$v" "agent-view/$v" "$@"
done
git diff --no-index "agent-view/$OLD" "agent-view/$NEW"

git diff --no-index exits 1 when anything differs, so the script doubles as a CI gate on the upgrade PR. Against my two fixture URLs, git diff --stat on the 1.64.1 → 1.65.0 output reported 2 files changed, 8 insertions(+), 10 deletions(-). Point it at the version your MCP server actually runs and it catches the other problem too. ./agent-view-diff.sh 1.65.0 1.65.0-alpha-2026-10-30 flagged # actionable refs: 23 → 3 at the top of the modal page’s diff.

Should you run —update-snapshots after a Playwright upgrade?

Not as one blanket -u. When toMatchAriaSnapshot fails after an upgrade, re-baseline only after you’ve classified every hunk in the agent-view diff, because each kind of hunk calls for a different fix, and -u applies the same one to all of them:

The first three are Playwright getting more correct, and they still break things, because tests and agents learned the old behavior. The last row is the one that hurts an agent rollout. A test that can’t find a button fails loudly. An agent that can’t see a button finds another way to “complete” the task. The Playwright aria snapshot docs cover the toMatchAriaSnapshot and .aria.yml mechanics. This table is the review step those mechanics assume someone does.

Common Playwright 1.65 upgrade errors

• locator.fill: Timeout … exceeded with element is inert in the call log. The target sits inside an inert container. On 1.64 the same call could “succeed” without typing anything. Find the interaction that removes inert and do it first. Never reach for force.
• <section …> intercepts pointer events on 1.64. Often an inert region in disguise. Check for an inert ancestor before you blame overlays.
• getByRole with an exact name finds 0 elements, only on web components or inside <details>. The name lost unslotted or collapsed text. Hidden text still counts when referenced through aria-labelledby, per PR #43150.
• Firefox page crash on getByRole with a name, when aria-labelledby is also read in the page’s main world (the repro reads ariaLabelledByElements). Open regression #43967, first bad in 1.65.0. Hold Firefox projects on 1.64.1 until it’s fixed.

Diff your agents’ busiest pages this week

Run agent-view-diff.sh against the five pages your agents drive most, with your test suite’s Playwright version as the old one and the version your agent’s MCP server actually runs as the new one (npm view @playwright/mcp@<version> dependencies tells you). If that diff isn’t empty, you’ve just found what your agents would have started improvising around next sprint. For how the agent turns that snapshot into actions, see how Playwright MCP feeds the agent an accessibility snapshot.

FAQ