Playwright Test Agents in Java: Plan, Generate, Heal

“The agents are part of the Node.js test runner (npx playwright init-agents) and aren’t part of the driver shipped with the Java package.” That’s the reason the Playwright team gave when it closed the Java request for Playwright Test Agents as not planned on October 14, 2026. So Playwright Test Agents for Java aren’t coming. The planner, generator and healer workflow still runs against a Java suite, though, because the parts it’s built from are not tied to Node.

In my demo, a coding agent planned 23 scenarios and wrote 25 TestNG tests. 24 of them passed, and the one failure was a real bug it found. Then I broke the app twice. The agent healed a renamed button with a one-line change, and it refused to “heal” an off-by-one regression. The project has a pom.xml and no package.json of its own. Here is the setup, the numbers, and the four places where it still falls short.

Why don’t Playwright Test Agents work with Java?

Because the agents are prompt files that call tools only the Node test runner provides. npx playwright init-agents writes three agent definitions plus an MCP config that starts playwright run-test-mcp-server. That server’s test_list, test_run and test_debug tools load @playwright/test specs. Your TestNG classes are invisible to it.

The request has been made for every binding. #38610 asked for Python and was closed on August 11, 2026. The .NET request (#40560) was closed as not planned on October 31, when the team retired its P3-collecting-feedback label. The useful line is in that .NET thread, from Pavel Feldman in May: “Agents are just skills / specs and MCP server / CLI commands. Skills are text, MCP and CLI are cross-framework, so technically everything is there.”

So I put that claim to work. First, the control: I ran the official setup inside the Java project.

$ npx playwright init-agents --loop=claude
 🌱 seed.spec.ts - default environment seed file
 🤖 .claude/agents/playwright-test-generator.md - agent definition
 🤖 .claude/agents/playwright-test-healer.md - agent definition
 🤖 .claude/agents/playwright-test-planner.md - agent definition
 📝 specs/README.md - directory for test plans
 🔧 .mcp.json - mcp configuration

# then, calling the healer's own test_list tool over MCP:
Error: Cannot find module '@playwright/test'
   at seed.spec.ts:1
Listing tests:
Total: 0 tests in 0 files

The command succeeds and drops a TypeScript seed into a Java repo. The healer’s first tool call then reports zero tests, while mvn test in the same folder runs one. The healer prompt also tells the model to mark stubborn tests test.fixme(), which has no TestNG equivalent. Nothing here is broken. It just isn’t for you.

What can a Playwright Java suite use today?

The tools under the agents: Playwright MCP, playwright-cli, and two Java APIs. Every row in this table is something I ran against Playwright 1.65.0 from npm (its bundled npx playwright mcp and npx playwright cli) and Playwright Java 1.64.0, the latest release on Maven Central. The agent runs below called the standalone playwright-cli binary from @playwright/cli 0.1.23, which is built on a 1.65.0 alpha of the same core.

The table is the post in miniature. The browser half of the agent workflow speaks Java now. The test-runner half doesn’t, and you replace it with Maven plus about 20 lines of instructions.

How do you make Playwright MCP and playwright-cli emit Java?

Set the code-generation language once and each action the agent takes comes back as a Java statement it can paste into a test. Playwright MCP has had --codegen java since v0.0.79. playwright-cli reads the same setting from its default config file.

{ "codegen": "java" }

With that file in the project root, a CLI action prints Java instead of the TypeScript the skill docs promise:

$ playwright-cli -s=leave-desk select e5 Sick
### Ran Playwright code
```java
page.getByLabel("Leave type").selectOption("Sick");
```

Java locators as MCP targets

Since MCP v0.0.84, element tools “accept JavaScript, Python, Java and C# locators in target”. An agent that already knows your page objects can pass the Java locator it read from them. There is one catch the release note doesn’t mention: drop the page. receiver.

target: page.getByRole(AriaRole.BUTTON, new Page.GetByRoleOptions().setName("Submit request"))
Error: "page.getByRole(AriaRole.BUTTON, new Page.GetByRoleOptions().setName("Submit request"))" is not a valid target.
Use a ref from the page snapshot (e.g. "e5"), a CSS selector, or a locator (e.g. "getByRole('button', { name: 'Submit' })").

target: getByRole(AriaRole.BUTTON, new Page.GetByRoleOptions().setName("Submit request"))
page.getByRole(AriaRole.BUTTON, new Page.GetByRoleOptions().setName("Submit request")).click();

Python’s get_by_label("...") and JavaScript’s getByRole('button', { name: '...' }) were accepted too, and the emitted code was normalized to Java either way. Locators in Java page objects start with page., so an agent that copies them will hit this error. Put the rule in your agent instructions.

Planner and generator: what replaces init-agents?

A short CLAUDE.md (or AGENTS.md) in the Java project, plus the official skills. I ran npx playwright init-skills --loop claude to install the playwright-cli skill. Its references/test-generation.md describes plan, generate and heal step by step, so the workflow isn’t lost. Only the commands are wrong for Java. The instructions file translates them:

This is a Java suite. The playwright-cli skill's examples are TypeScript; translate
them with these rules instead of creating any `.ts` file or `package.json`.

- Run tests: `mvn -q test` (one class: `mvn -q test -Dtest=LeaveRequestTest`).
  Never run `npx playwright test`; there is no JS runner here.
- Seed: every test extends `BaseTest`, which opens a fresh context at BASE_URL.
- Debug a failing test (replaces `--debug=cli`):
  `mvn -q test -Dtest=<Class> -Dagent.debug=true` in the background, wait for the
  `AGENT:` line, run `playwright-cli attach leave-desk`, inspect, then
  `touch target/agent-release` and `playwright-cli detach`.
- `generate-locator` still prints JavaScript. Convert it to the Java form
  (`page.getByRole(AriaRole.BUTTON, new Page.GetByRoleOptions().setName("..."))`).
- If a failure looks like an app bug rather than test drift, do not change the
  assertion to match. Stop and report it.

Then I gave Claude Code (headless, claude -p) two prompts against a small leave-request app: one to plan, one to generate. These are the real numbers from both runs:

Both runs used Claude Code 2.1.319 on Claude Opus 5.5. The cost is what Claude Code reports for the run, not a bill, so treat it as an order of magnitude. When I reran the suite myself (the 25 new tests plus my seed test), 25 of 26 passed in 23 seconds.

What the plan and the generated tests look like

The plan is the artifact worth reviewing first, because everything downstream inherits its mistakes. The planner followed the skill’s Markdown format but adapted it to Java without being asked twice: each scenario group maps to one TestNG class, each kebab-case scenario name becomes a camelCase @Test method, and every scenario starts from BaseTest. It also wrote a locator table where each entry is the Java that playwright-cli codegen printed, so the generator never had to guess a selector.

It was honest about its limits, too. Two behaviours, calendar days that count weekends and a table that clears on reload, went into the plan as open questions for product rather than silent assumptions. It also noted that it hadn’t tried the Friday-to-Monday and daylight-saving cases in the browser, and that their expected values came from reading the page’s day-count code. That’s exactly the kind of note a reviewer needs and rarely gets from a human-written plan.

The jump from 23 scenarios to 25 tests isn’t padding. One happy-path scenario says “submit each leave type”, and the generator wrote it as a single method driven by a TestNG @DataProvider over Vacation, Sick and Bereavement, which Surefire counts as three tests. The generator also ran the full suite twice before reporting, and got the same 25-of-26 result both times.

The failure is the interesting part. While exploring, the planner found that double-clicking Submit request adds two identical rows, because the button stays enabled during a 400 ms save. I hadn’t planted that bug. It was just how I’d written the demo app. The planner recorded it as a likely bug and kept the expectation at one row. The generator reproduced it in the browser, then wrote a test that installs Playwright’s fake clock so the second save can’t slip past the assertion:

@Test(description = "KNOWN FAILURE: suspected app bug, double submit adds duplicate rows")
public void shouldNotDuplicateRequestOnDoubleSubmit() {
  // Control the 400 ms save timer so both pending saves settle before asserting.
  page.clock().install();
  fillForm("Bereavement", "2026-12-28", "2027-01-03");
  submit().dblclick();
  page.clock().runFor(2000);

  assertThat(status()).hasText("Request submitted: 7 day(s) of Bereavement");
  assertThat(rows()).hasCount(1);
}

It fails with Expected: 1, Received: 2, which is the right result. It used locators that the CLI had printed in Java, put them in one shared base class, and never wrote a fixed sleep. I’d still review it like any pull request. What saved the time was the exploration and the Java boilerplate.

How does a Java test hand a failing page to an agent?

With Browser.bind(), which is in the Java API (since 1.59). It publishes the running browser under a name, and playwright-cli attach <name> connects to it. That pair replaces the Node runner’s --debug=cli (the headless CLI debugger), which is what the official skill uses to pause a test and let the agent look around. I put it behind a system property in the base class:

private static final boolean AGENT_DEBUG = Boolean.getBoolean("agent.debug");

@BeforeClass
void launchBrowser() {
  playwright = Playwright.create();
  browser = playwright.chromium().launch();
  if (AGENT_DEBUG) {
    browser.bind("leave-desk",
        new Browser.BindOptions().setWorkspaceDir(System.getProperty("user.dir")));
  }
}

@AfterMethod
void closeContext(ITestResult result) throws InterruptedException {
  if (AGENT_DEBUG && result.getStatus() == ITestResult.FAILURE) {
    holdForAgent(result.getMethod().getMethodName()); // polls for target/agent-release
  }
  context.close();
}

holdForAgent prints an AGENT: line and waits up to five minutes for a marker file. While it waits, the failed test’s page is still open, and the agent can snapshot, click and eval against the exact state where the assertion failed. Without the property, the hook does nothing.

That default matters more than the hook itself. Your CI pipeline, your nightly regression job and every developer running mvn test see exactly the behaviour they saw before, because nobody sets agent.debug by accident. The pause only exists when an agent (or a person) asks for it on a single test class. If you run TestNG classes in parallel, give each bound browser a distinct name, for example by appending the class name, so the agent attaches to the browser that actually failed.

Two breakages, two different right answers

I renamed the Submit request button to Send request and asked the agent to heal. Then, with the rename healed, I introduced an off-by-one in the app’s day count and asked again with the same instructions.

For the rename, the agent ran one failing test with -Dagent.debug=true, attached, saw the new label in the snapshot, and confirmed that clicking Send request still saved the row. Then it made the fix. For the regression, it attached to the single-day test, saw “0 day(s)” on the live page, found the missing + 1 in the page script, and left all 16 failing assertions alone.

That second run is the one I care about. The official healer prompt’s fallback is test.fixme(). My rule (“if it looks like an app bug, stop and report”, in CLAUDE.md and repeated in the prompt) produced a bug report instead. If you’ve read about the healer skipping tests it couldn’t fix, that’s the behaviour you want to write down for your own agent before you run it.

Notice the 725 seconds as well. One renamed button made 24 tests each wait out the 30-second default timeout. Scale that to a 1,000-test suite that shares one submit-button locator, run serially, and a single full rerun costs over eight hours of timeouts. A healer that has to rerun the full suite to confirm its diagnosis spends most of its time waiting. Debugging one failing test first, as my prompt asked, is the step that makes this affordable.

The rename heal was a one-line change for a reason that has nothing to do with AI. The generator had put every locator in one shared base class, so every test reached the button through a single submit() helper. That’s the same discipline you’d want from a human: a page object or shared helper turns a UI rename into one diff instead of twenty-four. If your suite has locators copied into each test method, an agent heal will be exactly as large and as risky as a manual one.

The reports are worth reading in full, because both runs did more than the table shows. After the rename, the agent asked for confirmation from the app team that “Send request” was intentional, and pointed out that the plan in specs/ still said “Submit request”. After the off-by-one, it explained why 16 tests failed when only some of them check day counts: every test that submits a valid request breaks at the confirmation message, so the form-reset and validation-recovery tests went down with them. It proposed the one-line app fix, and it said plainly that it couldn’t confirm the double-submit test that day, because the day-count error failed that test first.

Where does getByRef fit?

Page.getByRef() (1.64) turns a ref from an AI-mode aria snapshot into a locator. It’s the same e9 an MCP or CLI agent sees. In Java:

String snapshot = page.ariaSnapshot(
    new Page.AriaSnapshotOptions().setMode(AriaSnapshotMode.AI));
String ref = snapshot.lines()
    .filter(line -> line.contains("- button \""))
    .map(line -> line.replaceAll(".*\\[ref=(e\\d+)].*", "$1"))
    .findFirst().orElseThrow(); // "e9", same ref an MCP/CLI agent sees
page.getByRef(ref).click();

That passed. After a page.reload(), though, the same getByRef("e9").click() failed with a plain Timeout 2000ms exceeded (I’d set a 2-second timeout), not a stale-ref error. Refs belong to a single snapshot. Use getByRef in Java code that has its own agent loop (snapshot, decide, act). Don’t use it in committed tests. Committed tests should use the getByRole line that codegen printed.

What still doesn’t work for Java teams?

Four gaps remain: generate-locator prints JavaScript, MCP waits leak JavaScript into Java output, there are no runner tools for Maven, and playwright-cli list doesn’t show a bound Java browser until you attach. I hit each one in the demo, and none of them blocks the workflow once you know about it.

1. generate-locator ignores the codegen setting. With codegen: java set, playwright-cli --raw generate-locator e9 still printed getByRole('button', { name: 'Submit request' }). The skill sends agents to this command when they write assertions, so they get JavaScript.
2. browser_wait_for leaks JavaScript into Java output. In --codegen java mode, MCP printed await page.getByText("Request submitted").first().waitFor({ state: 'visible' });. That doesn’t compile. Agents that paste generated code verbatim will hand you a red build. Tell them to turn waits into assertThat(...) calls.
3. No runner tools. Nothing replaces test_run or test_debug inside MCP. The agent runs mvn through its shell, and failure output comes back as Surefire text, not structured results.
4. playwright-cli list didn’t show the bound Java browser. It printed (no browsers) before I attached, yet attach leave-desk worked, and only then did list show it. Attach by the name you passed to bind().

Where to start on Monday

Add two things to your Java framework before you add any agent: .playwright/cli.config.json with "codegen": "java", and a Browser.bind() hook in your base test behind a system property. Those two changes give any coding agent a Java-speaking browser and a way into a failing test, and they cost nothing if nobody uses them. Then write the 20-line instructions file for your own repo. Your agent can’t guess your page objects, your data setup or how you run Maven.

Treat that instructions file the way you treat pom.xml: it lives in the repository, changes go through review, and a rule only gets added after an agent run shows it’s needed. Every rule in my excerpt traces back to something I saw in the demo, from the page. prefix to the JavaScript that generate-locator prints. Keep agent output on a branch as well, and review it like any pull request. The agent told me what it changed and what it left alone in every run, and that report is what makes the review fast.

For the browser half, the Playwright MCP vs CLI vs Agents comparison has re-measured token numbers for both. The workflow here assumes the CLI, because a coding agent with a shell already has everything else it needs.

FAQ