Your Agent Acts on Half-Streamed Tool-Call JSON

When your agent decides to call a tool, its arguments don’t arrive as an object — they arrive as a string, streamed one fragment at a time, and that string is invalid JSON until the very last chunk lands. If any part of your app treats the half-arrived buffer as a real decision — an optimistic preview, an eager useEffect, a confirm button that lights up the moment the panel populates — your agent is acting on half a decision. The fix is one invariant, and it’s testable: nothing commits until the stream says it’s done.

Every major provider works this way. Anthropic streams tool inputs as input_json_delta events carrying partial_json fragments, and the Messages streaming docs are blunt about it: “the deltas are partial JSON strings, whereas the final tool_use.input is always an object.” OpenAI does the same with response.function_call_arguments.delta and warns in the function-calling guide to “treat partial JSON as display-only” and “execute only after arguments are complete.” The providers are telling you exactly where the bug is. Most test suites still don’t have an assertion on it.

Why do streamed tool calls arrive as broken JSON?

Because the model emits the arguments object token by token, and a prefix of a JSON object is almost never itself valid JSON. Anthropic notes that current models emit “one complete key and value property at a time” — so mid-stream your buffer is a dangling fragment like {"amountCents": 50 with no closing brace, no closing quote, sometimes a key sliced in half. It only becomes a parseable object on the terminal event (content_block_stop for Anthropic, stream end for OpenAI).

This isn’t a rare edge case you can wish away — tool-call correctness is a benchmarked, first-class failure axis. The Berkeley Function-Calling Leaderboard V4 (updated April 2026) devotes a full category to hallucination and weights agentic behavior at 40% of the score precisely because models still get which tool and what arguments wrong. And with Gartner projecting 40% of enterprise apps will embed task-specific agents by the end of 2026, the surface where “half-parsed arguments” turns into a real side effect is now most of the product.

What actually breaks when your UI trusts the buffer?

Three failure modes, in ascending order of how much they’ll cost you. First, the parse crash: naive code runs JSON.parse(buffer) on every delta, throws on every fragment until the last, and either takes down the render or hides behind a swallowing try/catch that leaves the preview flickering.

onDelta((partialJson) => {
  buffer += partialJson;              // partial_json fragments, in arrival order
  const args = JSON.parse(buffer);    // throws on every chunk but the last
  setPreview(args);                   // ...so it never runs, or a catch swallows it
});

Second — and this is the one that looks like a fix and is actually worse — the best-effort parser. Someone discovers a library like partial-json that happily closes the dangling braces for you. The flicker disappears. But now the preview is populated with a valid object built from a truncated prefix, and any control gated on “preview is non-null” goes live while the arguments are still wrong.

import { parse, Allow } from 'partial-json';
onDelta((partialJson) => {
  buffer += partialJson;
  setPreview(parse(buffer, Allow.ALL));   // valid object from a partial buffer
});
// <ConfirmButton disabled={preview == null} />  ← now clickable mid-stream

Third, the late field. In OpenAI-compatible streams, function.name and the call id are only set on the first delta; a consumer that reads the name off the current chunk gets undefined on every subsequent one and mis-dispatches — a real, filed bug (opencode #24137). LangChain shipped the mirror image: streaming that reported empty tool-call args despite valid partial_json arriving on the wire.

I watched this exact chain play out at a support-tooling client. Their agent could issue refunds, and the “Confirm refund” button enabled the instant the optimistic preview had a non-null value — which, after the team swapped in a best-effort parser to kill a flicker, happened several deltas before the amountCents and destination account finished streaming. Over one weekend, before anyone connected the dots, fast agents hammering Enter confirmed 14 refunds whose amount was still mid-stream — each posting a truncated prefix of the intended value. The model was correct every time. The app committed early every time. No text assertion, no live-model eval, and no happy-path E2E test caught it, because on a fast local network the stream completed before the click. The bug only exists in the gap between the first populated frame and the last.

Why can’t you reproduce this with a real backend in CI?

Because the chunk boundaries that trigger the bug are nondeterministic in production and invisible in a fast test environment. Where a partial_json fragment gets split depends on the model’s token boundaries, TLS record sizes, proxy buffering, and your gateway’s flush behavior — none of which you control and none of which reproduce on demand. Against a live backend over localhost, the whole stream often lands in a single tick, so the mid-stream window never opens and your test goes green on a bug that ships.

So you stop hoping and start controlling the split. The senior move is to make the boundary an input to the test: feed the arguments JSON to your client fragmented at adversarial points, and release the fragments on your schedule. This is the same reason I argue for controlling the token stream directly when testing races against streamed AI text — you can’t assert on a window you can’t reliably open.

How do you test the content boundaries in Playwright?

Fulfill the streaming request with a hand-built SSE body whose partial_json fragments split at the boundaries that hurt — mid-number, mid-key, mid-value — and assert two things: the UI never surfaces a parse error, and the committed value equals the full object, not a prefix. Playwright’s request interception lets you stand in for the model with a fixture you fully control.

Encode the adversarial split

Start by encoding the adversarial split. The full object is {"amountCents": 5000, "to": "acc_88f2"}; the fragments below slice a key (amount|Cents), a number (50|00), and a string value (acc|_88f2):

const event = (type: string, data: object) =>
  `event: ${type}\ndata: ${JSON.stringify({ type, ...data })}\n\n`;

function sseToolCall(name: string, frags: string[]): string {
  const start = event('content_block_start', { content_block: { type: 'tool_use', name } });
  const deltas = frags.map((partial_json) =>
    event('content_block_delta', { delta: { type: 'input_json_delta', partial_json } }));
  return [start, ...deltas, event('content_block_stop', {})].join('');
}

// Boundaries chosen to break naive parsers: mid-key, mid-number, mid-value.
const frags = ['{"amount', 'Cents": 50', '00, "to": "acc', '_88f2"}'];

Serve the fragments and assert on behavior

Now serve it and assert on behavior, not prose:

test('preview survives adversarial JSON boundaries', async ({ page }) => {
  await page.route('**/api/agent/chat', (route) =>
    route.fulfill({ contentType: 'text/event-stream',
                    body: sseToolCall('create_refund', frags) }));

  await page.goto('/support');
  await sendPrompt(page, 'refund $50 to acc_88f2');

  await expect(page.getByTestId('preview-error')).toHaveCount(0); // never parses a partial buffer
  await expect(page.getByTestId('preview-amount')).toHaveText('$50.00'); // full value, not "$0.50"
  await expect(page.getByTestId('preview-account')).toHaveText('acc_88f2');
});

This is a close cousin of asserting on the call itself rather than the reply, which I covered in testing what your agent does, not what it says. There the point was to assert on the tool call instead of the chat bubble; here it’s to assert the tool call is complete before anything reads it.

How do you prove the side effect waits for the stream to finish?

Give the test frame-accurate control over when each event is released, then assert the side effect fires exactly once — after the terminal event, never before. Content-boundary tests catch bad parsing; this timing test catches the eager-commit bug that cost my client 14 refunds. Install a stepped stream by overriding fetch before the app loads:

import type { Page } from '@playwright/test';

export async function installSteppedStream(page: Page, url: string, chunks: string[]) {
  await page.addInitScript(({ url, chunks }) => {
    let release: () => void;
    (window as any).__next = () => release?.();          // test drives this
    const real = window.fetch;
    window.fetch = (input: any, init?: any) =>
      String(input).includes(url)
        ? Promise.resolve(new Response(new ReadableStream({
            async start(c) {
              for (const chunk of chunks) {
                await new Promise<void>((r) => (release = r)); // wait for __next()
                c.enqueue(new TextEncoder().encode(chunk));
              }
              c.close();
            },
          }), { headers: { 'Content-Type': 'text/event-stream' } }))
        : real(input, init);
  }, { url, chunks });
}

test('side effect waits for the terminal event', async ({ page }) => {
  let refunds = 0;
  await page.route('**/api/refunds', (r) => { refunds++; return r.fulfill({ json: { ok: true } }); });

  await installSteppedStream(page, '/api/agent/chat', chunks); // start + deltas + stop
  await page.goto('/support');
  await sendPrompt(page, 'refund $50 to acc_88f2');

  for (let i = 0; i < chunks.length - 1; i++)                  // everything but content_block_stop
    await page.evaluate(() => (window as any).__next());
  await expect(page.getByRole('button', { name: 'Confirm refund' })).toBeDisabled();
  expect(refunds).toBe(0);                                     // nothing fired on partial args

  await page.evaluate(() => (window as any).__next());         // release the terminal event
  await page.getByRole('button', { name: 'Confirm refund' }).click();
  await expect.poll(() => refunds).toBe(1);                    // exactly once, full args
});

That last assertion — fires exactly once — is doing double duty. It fails if the action never fires, and it fails if a retry or a re-render fires it twice, which is the same idempotency concern behind an agent that retries and charges the card twice.

Which split boundaries actually matter?

Not all of them are equal — a few reliably expose distinct bugs, and they map to failure classes people have already hit in the wild. Parameterize your fixture over this set rather than trusting one lucky split:

The mid-UTF-8 case is the one teams forget: a customer named “José” or a note with an emoji means a multi-byte character can be split across two partial_json fragments, and a decoder that runs per fragment garbles it. Add one fixture with a non-ASCII value and you’ll find out whether your accumulation is byte-safe or character-naive. This is a different concern from tool ordering — for that failure class, see when your agent’s tool calls come out of order.

What ships on every PR versus nightly?

The stepped-stream and boundary tests are pure, deterministic, and fast — they belong on every PR, because they test your accumulation and commit logic, which is code you own and can break in any refactor. They never call a model, so they never flake. Keep at least one real end-to-end run against a live model in a nightly lane to catch the decisions your fixtures can’t anticipate, but don’t put a live stream on the critical path of a merge. The provider guidance is the same guardrail stated as a rule: parse the display copy however you like, but execute only after arguments are complete — so make “complete” an explicit, asserted state in your app, and pin it with a test that controls the stream.

Your one next step this week

Take the single highest-stakes tool your agent can call — the one that moves money, deletes data, or emails a customer — and write one test that streams its arguments split mid-number, then asserts the side effect fires exactly once, only after the terminal event. Don’t build the whole harness yet; just open the mid-stream window on that one action and see whether your confirm button was already clickable inside it. On most agent codebases I’ve touched, it was.