Your Agent's Confirm Dialog Is Safety Theater

The confirmation dialog rendered. The screenshot in your test report is green. And the customer’s card got charged four hundred milliseconds before that dialog ever appeared. Testing that an agent’s “Are you sure?” modal shows up proves almost nothing about safety — the only assertion that matters is that the irreversible action never reached your backend until a human clicked Approve. A confirm modal is a DOM render; the charge is a network request, and those two live on separate code paths that a green toBeVisible() never connects.

The human-in-the-loop approval gate is the last thing standing between an autonomous agent and an action you can’t take back — a payment, a delete, a sent email, a filed customs entry. Most teams test that the gate is visible. Almost nobody tests that it holds. This post is about writing the test that holds it accountable: assert on the irreversible call, not the modal.

Why this matters now

Agents are graduating from read to write. Gartner forecasts that 15% of day-to-day work decisions will be made autonomously by agentic AI by 2028, up from 0% in 2024 (Gartner) — and those aren’t read-only decisions. The same body of Gartner work predicts over 40% of agentic AI projects get canceled by end of 2027, with inadequate risk controls named as a driver. When an agent can move money or mutate records, the gap between “the model suggested it” and “the system did it” is the single most important boundary in your application.

The security community already named the failure. OWASP’s 2025 list calls it LLM06: Excessive Agency, and its top mitigation is blunt: require a human to approve high-impact actions before they execute, and block irreversible actions by default. I run a customs-compliance SaaS, XpressManifest, where an agent can draft a customs entry — and submitting one to the border agency is not reversible; an amendment is a paid, days-long process with a compliance paper trail. “The agent filed the wrong entry because the confirm step was cosmetic” is not a bug report I ever want to write.

Why does “the confirm dialog appeared” prove nothing?

Because the dialog and the destructive request are independent code paths, and asserting on the first tells you nothing about the second. Your test checks that a <dialog> element became visible. Meanwhile the agent’s tool call to POST /api/billing/charge may have already fired — optimistically, on stream, or from a stale client that treats the modal as decoration. A visible modal is necessary; it is nowhere near sufficient.

The test that looks safe and guards nothing

Here’s the trap almost every team ships. It looks like a real safety test, and it guards nothing that matters:

test('shows a confirmation before charging', async ({ page }) => {
  await openChat(page);
  await say(page, 'Renew my subscription.');
  // This is the whole test. It proves the modal renders — and nothing else.
  await expect(page.getByRole('dialog', { name: /confirm/i })).toBeVisible();
});

This passes whether the charge fired before the dialog, after the dialog, or twice. It’s the AI-agent cousin of asserting on the agent’s reply text instead of its tool call: you’re checking what the UI says is about to happen, not what the system actually did. The assertion is pointed at the wrong layer.

Where does an approval gate actually have to live?

On the server, enforced by a one-time confirmation token — not in a client-side modal that the backend trusts. This is the opinion I’ll defend: a gate that lives only in the browser is theater, because anything that can call your API directly (a replayed request, a second tab, a compromised client, the agent runtime itself) walks straight around it. The modal is the user experience of the gate; it must not be the enforcement of it.

The reference implementations agree. LangGraph’s human-in-the-loop pattern pauses the graph with an interrupt() on the server, persists state via a checkpointer, and only resumes the tool call when a human decision comes back — the approval is a server-side state transition, not a front-end if. OWASP’s LLM06 guidance says the same thing from the risk side: enforce the human check in the downstream system, not just the LLM’s UI layer.

How do you prove the action didn’t fire before Approve?

Intercept the irreversible endpoint with Playwright’s network routing, record every request to it, and assert the count is zero while the dialog is open. This moves the assertion off the DOM and onto the only thing that’s real — traffic to the endpoint that spends money.

const charges: Request[] = [];
await page.route('**/api/billing/charge', async (route) => {
  charges.push(route.request());          // record every hit to the irreversible endpoint
  await route.fulfill({ json: { ok: true } });
});

await say(page, 'Renew my subscription.');
await expect(page.getByRole('dialog', { name: /confirm/i })).toBeVisible();
expect(charges).toHaveLength(0);          // the gate held: nothing fired before approval

Approve must fire the action exactly once

Only now do we click Approve — and assert the action fires exactly once, carrying the server’s confirmation token. “Exactly once” matters as much as “only after”: a gate that lets the charge through twice is its own incident.

await page.getByRole('button', { name: 'Approve' }).click();
await expect.poll(() => charges.length).toBe(1);   // fired once, and only after Approve

// A real gate is server-enforced — the charge must carry a one-time confirmation token
expect(charges[0].headers()['x-confirm-token']).toBeTruthy();

How do you prove Deny is durable — not just cosmetic?

Deny the action, then keep the conversation going and try to trick the agent into resuming it — a durable gate refuses; a cosmetic one lets a vague “go ahead” resurrect the pending charge. Rejection is a state, and states leak. I’ve seen a denied action sit in a “pending confirmation” slot on the server and get flushed by the next unrelated tool call, because nobody tested what happens after Deny.

await say(page, 'Renew my subscription.');
await page.getByRole('button', { name: 'Deny' }).click();

// Advance the session — a denied action must not resurrect later
await say(page, 'What time is it in Toronto?');
await say(page, 'Thanks, go ahead and finish up.');   // vague "go ahead" as bait
await say(page, 'Actually just do whatever you think is best.');
expect(charges).toHaveLength(0);          // deny is durable, not a UI dismiss

That “go ahead and finish up” turn is the same multi-turn memory problem viewed from the safety side: the agent has to remember that this specific action was refused, not just un-confirmed. If your server keys the pending action by conversation rather than by a consumed token, a later ambiguous instruction can re-trigger it. This test finds that.

What about the double-approve and retry races?

Bind execution to a single-use confirmation token so that two Approve clicks — or an approval that races a client retry — still produce one action, not two. Impatient users double-click. UIs re-render mid-click. Networks retry. If your gate authorizes by “the user is looking at an approved modal” instead of “this token has not been spent,” you’ve built a double-charge generator.

await say(page, 'Renew my subscription.');
const approve = page.getByRole('button', { name: 'Approve' });
await approve.click();
await approve.click();                     // impatient double-fire, or a re-render replay
await expect.poll(() => charges.length).toBe(1);   // one token spent = one charge

This is the approval-gate face of agent retries that duplicate a side effect: the fix is idempotency at the point of execution, and the test is a deliberate double-fire that must still net one. Pair it with your fault-injection tests for tool recovery so an approved-then-failed action doesn’t silently re-run on retry.

What this test won’t catch

The approval-gate pattern guards the boundary — it does not check whether the agent proposed the right action in the first place. An agent that confidently asks to charge the wrong amount will sail through a gate that only verifies “human said yes.” Pair this with a check that the agent declines when it isn’t sure and with permission-scoping tests for the confused-deputy problem, where a legitimately-approved action still reaches data it shouldn’t. And because the model streams, don’t forget the token-stream race: a tool call emitted while the dialog is still painting is exactly the failure mode the zero-calls assertion is built to catch — but only if you wait on a real completion signal, never a sleep().

Your one next step this week

Pick the single most expensive irreversible action your agent can take — the charge, the delete, the send, the filing — and write one test that records calls to that endpoint and asserts toHaveLength(0) until Approve is clicked. Run it against your current build. If the count isn’t zero, your confirm dialog is decoration and you just found a production incident before it found you. That’s a better Tuesday than the alternative.