Designing an Answer UI Where Citations Aren't an Afterthought

Footnote-style links at the end of a paragraph don't get checked. Making provenance visible at the sentence level changes what users trust.

Profile picture of Serply
Serply
An answer with inline sentence-level source attributions

Most AI answer interfaces put citations at the end, as small numbered links. Nobody clicks them. The visual design says “here is the answer, and here are some links we found,” which is exactly the wrong relationship — and it means a fabricated citation looks identical to a real one until someone checks.

Making provenance structural rather than decorative takes work at three layers: how you ask the model for the answer, how you validate it, and how you render it.

Ask for structure, not prose with footnotes

Free text with [1] markers is what you get by default, and it’s hard to validate — the markers can point anywhere, and the mapping between claim and source is implicit. Ask for the structure directly:

import os
import json
import requests
from urllib.parse import quote_plus
from anthropic import Anthropic

API_KEY = os.environ["SERPLY_API_KEY"]
client = Anthropic()

ANSWER_TOOL = {
    "name": "answer",
    "description": "Provide a grounded answer with per-claim source attribution.",
    "input_schema": {
        "type": "object",
        "properties": {
            "summary": {
                "type": "string",
                "description": "One or two sentences answering the question directly.",
            },
            "claims": {
                "type": "array",
                "description": "The answer broken into individual factual claims.",
                "items": {
                    "type": "object",
                    "properties": {
                        "text": {"type": "string",
                                 "description": "One factual statement."},
                        "source_urls": {
                            "type": "array", "items": {"type": "string"},
                            "description": "URLs from the provided sources that "
                                           "directly support this statement.",
                        },
                        "quote": {
                            "type": "string",
                            "description": "The exact sentence from a source that "
                                           "supports this claim. Copy verbatim.",
                        },
                        "certainty": {
                            "type": "string",
                            "enum": ["stated", "implied", "inferred"],
                            "description": "stated: a source says this directly. "
                                           "implied: strongly suggested. "
                                           "inferred: your reasoning across sources.",
                        },
                    },
                    "required": ["text", "source_urls", "certainty"],
                },
            },
            "unanswered": {
                "type": "array", "items": {"type": "string"},
                "description": "Parts of the question the sources do not address.",
            },
        },
        "required": ["summary", "claims", "unanswered"],
    },
}

The certainty enum is the field that changes the output most. Without it, models present a cross-source inference with the same confidence as a directly quoted fact. With it, they mark the difference — and you can render it differently, which is the whole point.

The unanswered array gives the model a place to put gaps. Without one, gaps get filled.

Retrieval

def search(query: str, num: int = 8) -> list[dict]:
    resp = requests.get(
        f"https://api.serply.io/v1/search/q={quote_plus(query)}&num={num}",
        headers={"X-Api-Key": API_KEY, "X-Proxy-Location": "US"},
        timeout=30,
    )
    resp.raise_for_status()
    return resp.json().get("results", [])


def read(url: str) -> str | None:
    try:
        resp = requests.post(
            "https://api.serply.io/v1/request",
            headers={"X-Api-Key": API_KEY, "Content-Type": "application/json"},
            json={"url": url, "response_type": "markdown"},
            timeout=60,
        )
        resp.raise_for_status()
        return resp.text[:20_000]
    except requests.RequestException:
        return None

Markdown mode returns text as the body, so .text rather than .json().

Generating

from concurrent.futures import ThreadPoolExecutor

SYSTEM = """You answer questions strictly from the provided sources.

Rules:
- Every claim must cite at least one source URL from the provided set.
- Never cite a URL that is not in the sources. If you cannot support a claim,
  do not make it.
- Copy the supporting quote verbatim. Do not paraphrase it into the quote field.
- Mark certainty honestly. Most claims should be "stated". Use "inferred" when
  you are combining facts across sources — that is legitimate, but say so.
- Put anything the sources do not cover into `unanswered`. Do not fill gaps."""


def answer(question: str) -> dict:
    results = search(question)
    urls = [r["link"] for r in results[:5] if r.get("link")]

    with ThreadPoolExecutor(max_workers=5) as pool:
        texts = list(pool.map(read, urls))

    sources = [
        {"url": u, "title": r.get("title"), "text": t}
        for u, r, t in zip(urls, results, texts) if t
    ]
    if not sources:
        return {"summary": "No readable sources were found for this question.",
                "claims": [], "unanswered": [question], "sources": []}

    context = "\n\n".join(f"=== {s['url']} ===\n{s['text']}" for s in sources)

    msg = client.messages.create(
        model="claude-sonnet-4-5", max_tokens=3000, system=SYSTEM,
        tools=[ANSWER_TOOL], tool_choice={"type": "tool", "name": "answer"},
        messages=[{"role": "user",
                   "content": f"QUESTION: {question}\n\nSOURCES:\n{context}"}],
    )

    payload = next(b.input for b in msg.content if b.type == "tool_use")
    return validate(payload, sources)

Validate server-side, always

Never render what the model returned without checking it:

def validate(payload: dict, sources: list[dict]) -> dict:
    known = {s["url"] for s in sources}
    by_url = {s["url"]: s["text"] for s in sources}

    clean_claims = []
    for claim in payload.get("claims", []):
        cited = [u for u in claim.get("source_urls", [])]
        real = [u for u in cited if u in known]
        fabricated = [u for u in cited if u not in known]

        quote = (claim.get("quote") or "").strip()
        quote_found = any(
            quote and quote[:80].lower() in (by_url.get(u, "").lower())
            for u in real
        )

        clean_claims.append({
            **claim,
            "source_urls": real,
            "fabricated_urls": fabricated,
            "quote_verified": quote_found,
            "status": (
                "unsupported" if not real
                else "quote_mismatch" if quote and not quote_found
                else "ok"
            ),
        })

    return {
        "summary": payload.get("summary", ""),
        "claims": clean_claims,
        "unanswered": payload.get("unanswered", []),
        "sources": [{"url": s["url"], "title": s["title"]} for s in sources],
        "integrity": {
            "unsupported_claims": sum(1 for c in clean_claims
                                      if c["status"] == "unsupported"),
            "quote_mismatches": sum(1 for c in clean_claims
                                    if c["status"] == "quote_mismatch"),
            "fabricated_urls": sum(len(c["fabricated_urls"]) for c in clean_claims),
        },
    }

Two checks, both cheap, both catching things users would otherwise catch for you.

The URL check is a set membership test. A cited URL that never appeared in retrieval is fabricated, full stop — there’s no ambiguity to reason about.

The quote check is a substring match on the first 80 characters. It’s approximate on purpose; models normalise whitespace and occasionally trim a leading article. What it reliably catches is a quote that isn’t in the source at all, which is the case that matters.

Rendering the difference

The validation output is only useful if the UI uses it:

function renderClaim(claim) {
  const badge = {
    stated:   { label: 'Stated in source', cls: 'cert-stated' },
    implied:  { label: 'Implied',          cls: 'cert-implied' },
    inferred: { label: 'Inferred across sources', cls: 'cert-inferred' },
  }[claim.certainty] ?? { label: 'Unknown', cls: 'cert-unknown' };

  if (claim.status === 'unsupported') {
    return `<p class="claim unsupported">
      ${escapeHtml(claim.text)}
      <span class="warn">No source in the retrieved set supports this.</span>
    </p>`;
  }

  const links = claim.source_urls
    .map(u => `<a href="${u}" target="_blank" rel="noopener">${hostname(u)}</a>`)
    .join(', ');

  return `<p class="claim ${badge.cls}">
    ${escapeHtml(claim.text)}
    <span class="attribution">
      <span class="cert">${badge.label}</span> — ${links}
      ${claim.quote_verified
        ? `<blockquote class="verified">${escapeHtml(claim.quote)}</blockquote>`
        : ''}
    </span>
  </p>`;
}

Attribution sits with the sentence, not at the bottom of the page. Showing the hostname rather than a number means the reader knows whether a claim came from a standards body or a forum post without clicking anything — which is most of what checking a citation is for.

Verified quotes render inline. Unverified ones are simply omitted rather than shown with a warning, because a quote you can’t confirm adds nothing.

The unanswered section is not a failure state

function renderUnanswered(items) {
  if (!items.length) return '';
  return `<section class="gaps">
    <h3>Not covered by these sources</h3>
    <ul>${items.map(i => `<li>${escapeHtml(i)}</li>`).join('')}</ul>
  </section>`;
}

Design this section to look deliberate — same visual weight as the answer, not a greyed-out apology. Users read a prominent “we couldn’t establish X” as rigour. They read a hidden one, discovered later, as a system that overstated what it knew.

What to track

Log the integrity block on every answer. fabricated_urls above zero is the alarm — it means the model is inventing citations, and the number should be zero in a healthy pipeline. A rising quote_mismatch rate usually means your source text is being truncated before the quoted passage, not that the model is lying.