Selenium to Playwright Migration: 1,400 Never Made It

We moved 4,000 Selenium tests to Playwright over six months. Only 2,600 arrived. We deleted 1,120 outright and left 280 behind on Selenium, permanently. The suite went from 190 minutes to 41 — and when I did the arithmetic honestly afterwards, close to half of that speedup had nothing to do with Playwright. It came from deleting tests we should have deleted years earlier. That is the line item every enterprise Selenium to Playwright migration plan leaves out.

Every published guide on this prescribes. Almost none of them report. So here is the report.

The Suite Nobody Wanted to Touch

The client was a national lottery and gaming operator — a regulated shop, which matters more than it sounds. Nine SDETs across five squads. 4,000 UI tests in Java and TestNG, driving Selenium Grid across 20 nodes, accumulated over roughly nine years by people who had mostly left the company.

A full regression run took 190 minutes. Nobody ran it on a pull request. It ran nightly, and the first thing the team did every morning was triage a wall of red that they had learned to read like weather.

The part that made it a genuinely hard migration, rather than a tedious one, was that nobody could tell you what the suite covered. Not “nobody had documented it” — nobody knew. There was no map from test to requirement. There were tests named testFlow_2b_final_FIXED. Coverage was a number in a dashboard, and the number was going up, and everyone had agreed to stop asking what it meant.

What the Audit Found: 4,000 Tests, 1,400 Dead

Three weeks of work, two engineers, one spreadsheet. Here is what an audit output actually looks like, since every article recommends one and none of them show you theirs.

Twenty-eight percent of a suite that leadership believed was its safety net had not asserted anything meaningful in years. The 340 @Ignore’d tests are the ones I think about. Somebody disabled each one, in a hurry, intending to come back. Nobody ever did, and the count kept getting reported as coverage.

Who fights you when you delete 1,120 tests

The QA director’s quarterly objective was automated test count. I want to be fair to him: he did not invent that metric, he inherited it, and it had been rewarded for three years running. But it meant that the single most valuable thing we could do for the suite, removing more than a quarter of it, read on his scorecard as catastrophic failure.

What broke the deadlock wasn’t an argument about quality. It was changing the unit. We pulled twelve months of run history and sorted every test by one question: has it ever failed? Not flaked. Failed, once, on a real regression. Of the 1,120 we wanted to cut, none had. They’d passed every night for years, because most of them asserted that a page had loaded.

A test that has never failed is not proof that the code works. It’s an unpaid subscription. That reframing got us the sign-off in one meeting, after two months of losing the argument in every other meeting.

The one we deleted that we shouldn’t have

Of those 295 “redundant parameterized” tests, one wasn’t. Provincial withholding thresholds on prize payouts differ, and the Ontario and Quebec cases looked identical to us at a glance because the test names differed by a single digit. We cut the Quebec case.

It surfaced six weeks later in UAT, not production, and the fix took an afternoon. That was luck, not process. If you delete 1,120 tests, you will delete at least one you needed. Budget for that instead of pretending your audit is perfect.

Why We Killed the Big-Bang Migration in Week Three

We started with a big-bang plan, and I still think it was the right thing to try. The logic was sound: a strangler-fig migration means running two frameworks, two CI configs, two flake profiles, and two mental models side by side for months.

So we froze the Selenium suite and pointed everyone at conversion for three weeks. The freeze meant the app kept shipping while the tests didn’t, so by week three we were writing Playwright tests for screens that no longer existed. A regulated shop ships on a compliance calendar. You do not get to pause it because QA is busy.

We switched to a squad-by-squad strangler fig for the remaining five months, and paid the tax we’d been trying to avoid: both suites ran in CI for four months, and the runner bill went from roughly $2,400 to $4,100 a month. About $6,800 of pure duplication. It was the cheapest line item in the project and the one finance asked about most.

Can You Convert Selenium Tests to Playwright Automatically?

Not usefully — and the failure mode is worse than it not working. An LLM converter’s job is to preserve the semantics of the code it’s given, and Selenium’s semantics are precisely the thing you are trying to escape. A converter that works perfectly hands you a Playwright suite that thinks like Selenium.

We piloted one on 400 tests. The headline numbers looked like a win: 92% compiled, 71% passed on the first run. Then we read the output.

// Selenium — the original, already ported to the Duration constructor.
// The old (driver, long seconds) form was deprecated in Selenium 4.0.0
// and removed outright in 4.1.3, so a 2019 suite won't even compile
// against a modern Selenium until someone fixes this line first.
WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10));
wait.until(ExpectedConditions.visibilityOf(confirmButton));
confirmButton.click();

Here is what the converter produced:

// The converter's output. It compiles. It passes. It is wrong.
await page.waitForTimeout(10000);
await page.getByRole('button', { name: 'Confirm' }).click();

It translated an explicit wait into a hard sleep. Mechanically, that is a defensible reading of WebDriverWait — wait ten seconds, then proceed. Semantically it is the single worst thing you can write in Playwright, and the docs say so in as many words: “Never wait for timeout in production. Tests that wait for time are inherently flaky.”

Those tests averaged about four such waits each, so across the 400 the converter added roughly four and a half hours of cumulative sleep to a suite whose entire business case was speed. What it should have produced is a web-first assertion, which waits exactly as long as it needs to and not one millisecond more:

// Playwright's auto-waiting does the job the WebDriverWait was doing.
await expect(page.getByRole('button', { name: 'Confirm' })).toBeVisible();
await page.getByRole('button', { name: 'Confirm' }).click();

We spent three weeks reviewing converted output before admitting that reviewing a bad conversion was slower than writing the test fresh. We threw it away — the converter cost us three weeks and produced nothing we kept. Writing fresh was less work than it sounds, because most of our Selenium wrapper classes collapsed straight into Playwright locators: the abstraction we’d hand-built over years ships in the box.

Should You Migrate If Your Team Is on Java and TestNG?

Yes, but probably not on Playwright’s Java binding. That binding is a browser automation library, not a test framework: there is no test runner, no fullyParallel, and no worker model, and fixtures exist only through an experimental JUnit 5 integration. You keep TestNG or JUnit and rebuild the parallel execution machinery yourself, which is most of what you were hoping to delete. The vendor guides skip this because they are all written TypeScript-first.

We moved the UI suite to TypeScript and kept Java and TestNG for the API layer. That call cost us about two months of ramp across nine engineers, and it was still correct. Staying on Playwright Java would have meant carrying the ThreadLocal scaffolding our Selenium suite was built on into the new framework: the exact machinery we were migrating to get rid of.

Nine Java engineers writing their first TypeScript are not fast. Budget the dip.

The Three Things That Actually Broke

Everything else in the migration was mechanical. These three were not, and all three are the same failure underneath: we brought Selenium’s model with us.

The engineers who were too good at waits

The most expensive habit in the migration belonged to our best people. Two of our strongest engineers had spent a decade becoming genuinely excellent at Selenium’s wait model — custom ExpectedConditions, tuned polling, the whole craft. In Playwright, that expertise is a liability, and it does not feel like one from the inside.

You get waitForTimeout(2000) sprinkled through a framework designed to make waits unnecessary, defended in code review by someone senior enough that juniors don’t push back. This is written up everywhere as a syntax problem. It is a people problem. We fixed it with a lint rule that made waitForTimeout a build failure outside of a debugging branch, because a rule in a linter is an argument you only have to win once. The async mistakes that made our ported tests look flaky are the code-level version of this same mental-model gap.

Dialogs stopped being something we handled

Selenium treats an alert as blocking session state that you go and fetch. Playwright fires an event, and if no listener is registered it dismisses the dialog automatically. Our ported tests failed because they still went looking: code that polled for an alert the Selenium way found nothing there, because Playwright had already dismissed it. The dialog is not in the DOM and it is not sitting around waiting for you. There is nothing to go and get.

The suite got simpler once we stopped trying. Most of our dialog-handling code deleted itself. The blocking alert model that used to poison entire worker threads has no equivalent failure in Playwright, because the failure is structurally impossible rather than merely handled.

Grid nodes became workers, and we sized them wrong

The two tools do not scale on the same unit, and if you size the new one with the old one’s instincts you will get it wrong. We did.

Playwright workers are OS processes that each start their own browser, and per-test isolation comes from BrowserContexts rather than fresh browsers. We knew that and still got it wrong: for the first month we launched a browser per test, faithfully reproducing Grid’s heavyweight isolation inside a framework built for the opposite, and wondered why CI hadn’t gotten faster. That mistake, reproducing Grid’s browser-per-test isolation inside Playwright, cost us enough to deserve its own write-up.

Once we moved to context-per-test and settled at 24 workers, the 20-node Grid went away entirely. So did its maintenance.

Results

Now the honest arithmetic, which is the reason I wanted to write this up at all.

Deleting the 1,120 dead tests and carving the 280 IE-mode tests into their own Selenium job — changing nothing else, migrating nothing — would have brought the 190-minute run down to roughly 124 minutes on its own. Of the 149 minutes we saved in total, about 66 came from that pruning and about 83 from Playwright’s execution model. Forty-four percent of our headline win was available without migrating anything at all.

How Long Does an Enterprise Selenium to Playwright Migration Take?

Ours took six months for 4,000 tests, with about three full-time engineers out of a nine-person team. The published effort tables say that should have been impossible: ThinkSys puts 1,000+ tests at 6–12 months, and Autonoma puts a 1,000+ suite at a phased 6–12 months. We did four times their test count at the bottom of their range, for exactly one reason: we refused to port two-fifths of it.

That is not a brag about velocity. Those tables price the cost of moving everything, and moving everything is the mistake. Playwright was still the right call: the flake rate fell, and the team stopped triaging weather every morning. But a migration is a rare, sanctioned excuse to delete code. That is most of the value. The new framework is the rest.

When You Shouldn’t Do This Migration

If your Selenium suite is stable, fast enough, and your team is good at it, don’t. The most common objection I hear is some version of “I see no point migrating my 3,000 tests,” and that objection is usually correct. A WebDriver setup nobody complains about is not worth a six-month project, and Grid is free in a way that CI minutes are not.

We migrated because 190 minutes meant nobody ran the suite before merging, and a suite nobody runs before merging is not a safety net. It’s a nightly report about the past. If your regression takes 20 minutes and your engineers trust it, the transition cost buys you almost nothing. Migrate when the suite has stopped doing its job, not when a framework gets popular. If you’re still deciding, the same test built in both frameworks is a much cheaper way to form an opinion than a migration is.

What I’d Do Differently

Run the audit before you pitch the migration, not after. We sold the project on Playwright and then discovered the deletions, which meant relitigating scope with a director who had already been promised a number. Had we walked in with the 1,400-dead-tests spreadsheet first, the migration would have been the conclusion of the pitch rather than a renegotiation halfway through.

Skip the converter entirely. Three weeks, nothing kept. I would not pilot one again, and I would be suspicious of anyone quoting a percentage of effort saved without saying what fraction of the output survived code review. Ours was zero.

Cut the dual-run window to two months, not four. We let squads finish at their own pace, which was kind and which doubled the period where every failure had to be triaged in two frameworks. A hard cutover date per squad — with the Selenium tests deleted, not disabled, on that date — would have cost less than the four months of context-switching did.

If you’re weighing this migration, don’t start with a proof-of-concept in Playwright. Start with a spreadsheet. Pull twelve months of run history and sort every test by one column: has it ever failed? The tests that never have are not protecting you, and you are about to pay to move them. Find out how many you have before you promise anyone a date, because that number, not the framework, is what your timeline is actually made of.

More war stories with the numbers left in — including the ones that didn’t go this well — are in the case studies archive.