A Correct RAG Answer Can Still Be Hallucinated

Your product shipped a RAG feature — a support bot, a docs Q&A box, a “chat with your contracts” panel. It answers a question correctly in the demo, your Playwright test asserts the answer contains the right phrase, the test goes green, and everyone moves on. That green check is lying to you. It proves the model produced the right words; it proves nothing about where the words came from. A model that ignored every retrieved document and answered from its own training data would pass that exact assertion — right up until the input changes and it invents a citation to a document that doesn’t exist.

Here’s the whole argument: for a RAG feature, correctness and grounding are two different properties, and the one that predicts production failures is grounding, not correctness. An answer is grounded when every claim in it is supported by what the system actually retrieved. Your text assertion measures neither retrieval nor support — it measures string overlap with an answer you happened to see once. This post is about testing the property that matters, in Playwright, without turning your suite into a flake machine.

Why can a correct RAG answer still be a hallucination?

Because “correct” and “grounded” are independent. An answer can be factually right and completely ungrounded — the model knew the fact from pre-training and never used your documents — and an answer can cite a real-looking source that says nothing of the kind. The Stanford RegLab study on legal research tools measured exactly this gap: purpose-built RAG systems from LexisNexis and Thomson Reuters, marketed as hallucination-free because they used retrieval, still hallucinated on 17% to 33% of queries, with Westlaw’s tool wrong roughly a third of the time. Retrieval reduced hallucination; it did not eliminate it, and the failures were often “misgrounded” — a confident answer attached to a source that didn’t support it.

That is the failure mode a text assertion is structurally blind to. expect(answer).toContain("30 days") passes whether the model read your refund policy, guessed, or quoted a competitor’s terms it saw in training. The assertion never looks at the retrieved context, so it cannot tell a grounded answer from a lucky one — and lucky answers stop being lucky the moment a customer asks something your demo didn’t.

What does “grounding” actually mean for a test?

Grounding is a measurable relationship between the answer and the retrieved context, not a vibe. The eval community has a precise name for it: faithfulness — the fraction of claims in the answer that are supported by the retrieved documents. RAGAS defines the faithfulness score as the ratio of supported claims to total claims: decompose the answer into individual statements, check each one against the context, and score 1.0 when every claim is supported and 0.5 when half are inventions. DeepEval implements the same claim-by-claim verification as its faithfulness metric. This is the number your test should be asserting on, not string containment.

Why good retrieval isn’t enough

Crucially, a good retrieval step does not guarantee a grounded answer. Better retrieval improves the ceiling, but the generator can still ignore the context it was handed — which is why faithfulness is scored on the answer-versus-context relationship, downstream of retrieval, rather than on retrieval quality alone. Independent eval tooling like Deepchecks treats faithfulness as its own metric precisely because retrieval quality and answer grounding fail separately. So a RAG grounding test has three distinct layers, and each fails differently:

Two layers you check exactly, one you score

The first two layers are cheap, exact, and never flaky — they compare ID sets and never read the model’s prose. The third is fuzzy, needs a judge, and belongs in a different suite. Most teams skip straight to asserting the answer prose and get none of the three.

How do you capture what the model actually retrieved?

You intercept the retrieval call. The whole technique hinges on one thing Playwright is unusually good at: watching the network while driving the UI. When the user asks a question, your app calls a retrieval endpoint before it calls the model — so wait for that response, capture the chunk IDs it returned, and you now have ground truth for what the answer was allowed to be built from.

test('enterprise refund answer is grounded in retrieved policy', async ({ page }) => {
  const chat = new ChatPanel(page);
  await chat.goto();

  // The retrieval call fires before the model call — grab it.
  const retrievalPromise = page.waitForResponse(r =>
    r.url().includes('/api/retrieve') && r.ok());
  await chat.ask('What is the refund window for enterprise plans?');

  const { chunks } = await (await retrievalPromise).json();
  const retrievedIds = new Set(chunks.map((c) => c.id));
});

How do you assert attribution without flaking?

Check that every citation the answer displays maps to a chunk that was actually retrieved. This is the highest-value assertion in the whole post and it is completely deterministic — it never touches the model’s prose, so it never flakes. If the answer renders a citation to a document that wasn’t in the retrieved set, the model fabricated it, full stop. That is the “cited the wrong doc” failure the Stanford study kept finding, and it’s a hard expect, not a fuzzy score.

  // Citations rendered in the UI carry the source id in a data attribute.
  const citedIds = await chat.citationIds(); // reads [data-citation-id]

  expect(citedIds.length,
    'a grounded answer must cite at least one retrieved source').toBeGreaterThan(0);

  for (const id of citedIds) {
    expect(retrievedIds,
      `answer cited "${id}" which was never retrieved — fabricated citation`)
      .toContain(id);
  }

Two failures fall out of this for free: an answer that cites nothing (ungrounded — the model went off-script), and an answer that cites a source outside the retrieved set (fabricated attribution). Both are common, both ship silently past a toContain on the answer text, and both are caught here with zero model calls and zero flakiness. Put this assertion in your blocking suite. It is as deterministic as any other Playwright check because it compares two ID sets — this is the same discipline as asserting the contract instead of the prose, applied to citations.

How do you score faithfulness — the fuzzy part — reproducibly?

Use an LLM-as-judge with a claim-by-claim rubric and a threshold, and run it out of the hot path. Attribution proves the citations point at retrieved docs; it does not prove the sentences are supported by those docs. For that you need semantic judgment, and semantic judgment can only be scored, never equality-checked. Decompose the answer into claims, ask a judge model which claims the retrieved context supports, and assert the ratio clears a bar.

test('answer claims are supported by retrieved context @nightly', async ({ page }) => {
  const { answer, context } = await captureAnswerAndContext(page,
    'What is the refund window for enterprise plans?');

  const { score, unsupported } = await gradeFaithfulness({ answer, context });

  expect.soft(score,
    `unsupported claims:\n- ${unsupported.join('\n- ')}`)
    .toBeGreaterThanOrEqual(0.9);
});

Two rules keep this from becoming the flakiest test you own. First, the judge has to be reproducible — pinned model version, temperature: 0, a fixed rubric, and structured JSON output — the same hardening I walked through for making an LLM judge deterministic. Second, this is a graded gate, not a binary one. Faithfulness benchmarks now score 160+ models on exactly this axis precisely because it’s a spectrum; a single failed run at 0.87 is signal to investigate, not a reason to block a deploy. Reserve the hard expect for attribution and keep faithfulness as a soft assertion or a report.

What I’d do differently

The first time I built one of these, I put the faithfulness judge in the PR-blocking suite. Within a week it had blocked three legitimate deploys on scores between 0.85 and 0.92 — answers that were fine, judged slightly harshly on one paraphrased claim. The team learned to re-run until green, which is the exact habit that makes flaky tests worthless. I’d draw the line hard next time: anything a model scores is advisory; anything two ID sets decide is blocking. The attribution check would have caught every real fabrication we saw that quarter, and it never once cried wolf.

I’d also surface retrieval metadata from day one instead of reverse-engineering it later. Every hour I spent parsing citations out of rendered HTML was an hour I could have skipped if the /chat response had carried retrievedIds behind a test flag from the start.

Your one concrete next step

Pick your highest-traffic RAG answer and write the attribution test — nothing else. Capture the retrieval response, read the cited IDs off the DOM, assert the cited set is a subset of the retrieved set, and assert it’s non-empty. It’s fifteen lines, it runs in your existing suite with no judge and no new infrastructure, and it will catch the fabricated-citation failure that your text assertions have been waving through. Add faithfulness scoring later, on a schedule. Ship the deterministic half today.

If you want more field-tested QA automation breakdowns like this — no fluff, just what actually holds up at enterprise scale — subscribe below. And if you’re formalizing how your team tests AI features, the rest of the AI-powered testing pillar goes deep on judges, drift, and mocking strategy.