Quick Tip: Diff Passing vs Failing Playwright Traces

The most useful trace in a flaky-test investigation is the one Playwright throws away. retain-on-failure keeps the failing attempt — but discards the passing retry, your only record of what correct timing looks like. Playwright’s retain-on-failure-and-retries trace mode, shipped in 1.59, keeps both.

That pair is for the flake you can’t reproduce locally: green 50 times on your laptop, red once a day in CI. Stop trying to reproduce it — CI already did, twice, minutes apart. The trace pair is the reproduction.

How does retain-on-failure-and-retries work?

It records a trace for every attempt and, once any attempt fails, retains all of them — including the passing retry every older mode discarded. Teams used to flip between on-first-retry and retain-on-failure and hope (the feature request reads like group therapy). Now it’s one line:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  retries: process.env.CI ? 2 : 0,
  use: {
    trace: 'retain-on-failure-and-retries',
  },
});

Open the two traces side by side

Each attempt gets its own folder under test-results/ — the retry carries a -retry1 suffix:

npx playwright show-trace test-results/checkout-promo-code/trace.zip &
npx playwright show-trace test-results/checkout-promo-code-retry1/trace.zip

Two windows, one screen — the trace viewer has no built-in compare mode. Find the last action both runs share, then read the network panels from there.

The passing trace is the baseline you were missing

A failing trace alone is ambiguous. At a large retail platform we stared at one for three sprints: a cart-totals request took 2.4 seconds and the test clicked a still-disabled button. Slow API, or bad wait? The passing retry’s trace settled it in ten minutes — same request at 310ms, same click, but only after a totals refresh the failing attempt never awaited. The fix was one await, the classic async trap behind flaky Playwright tests.

Two caveats. The mode traces every attempt, so scope it to the project you’re investigating — tracing every retry tripled one team’s artifact bill. And a test that passes on retry is usually a race condition — the retry isn’t the fix, but now it’s evidence.

Next step: find the test marked “flaky” in your last CI run, set trace: 'retain-on-failure-and-retries' on its project only, and re-run until it flips. Diff the pair at the first action where the timelines disagree. That’s your bug — no local reproduction required.