Cross-Checking Two Search Engines to Catch Agent Errors

• Both endpoints
• Measuring agreement
• What the numbers mean
• Feeding it to the agent
• Cost

Single-source retrieval has a quiet failure mode. If one search engine ranks a bad page first — an SEO farm, an outdated mirror, a confidently wrong forum answer — your agent reads it and repeats it. Nothing in the pipeline flags that anything went wrong, because from the agent’s point of view retrieval succeeded.

Querying two engines doesn’t fix that on its own. What it gives you is a disagreement signal, and disagreement is useful.

Both endpoints

Serply exposes Google and Bing on parallel paths, with the same path-embedded query convention:

import os
import requests
from urllib.parse import quote_plus
from concurrent.futures import ThreadPoolExecutor

API_KEY = os.environ["SERPLY_API_KEY"]
BASE = "https://api.serply.io/v1"
HEADERS = {"X-Api-Key": API_KEY, "X-Proxy-Location": "US"}


def google(query: str, num: int = 10) -> list[dict]:
    resp = requests.get(
        f"{BASE}/search/q={quote_plus(query)}&num={num}", headers=HEADERS, timeout=30
    )
    resp.raise_for_status()
    return resp.json().get("results", [])


def bing(query: str, num: int = 10) -> list[dict]:
    resp = requests.get(
        f"{BASE}/b/search/q={quote_plus(query)}&num={num}", headers=HEADERS, timeout=30
    )
    resp.raise_for_status()
    return resp.json().get("results", [])


def both(query: str, num: int = 10) -> tuple[list[dict], list[dict]]:
    with ThreadPoolExecutor(max_workers=2) as pool:
        g = pool.submit(google, query, num)
        b = pool.submit(bing, query, num)
        return g.result(), b.result()

Run them concurrently. Two sequential searches doubles your latency for a check that should be free in wall-clock terms.

Measuring agreement

Compare domains, not URLs. The same article often lives at slightly different URLs across engines — tracking parameters, AMP variants, trailing slashes:

from urllib.parse import urlparse


def domain(url: str) -> str:
    try:
        host = urlparse(url).netloc.lower()
        return host[4:] if host.startswith("www.") else host
    except Exception:
        return ""


def agreement(g: list[dict], b: list[dict], top_n: int = 10) -> dict:
    gd = [domain(r.get("link", "")) for r in g[:top_n]]
    bd = [domain(r.get("link", "")) for r in b[:top_n]]
    gs, bs = {d for d in gd if d}, {d for d in bd if d}

    overlap = gs & bs
    union = gs | bs
    return {
        "jaccard": len(overlap) / len(union) if union else 0.0,
        "shared": sorted(overlap),
        "google_only": sorted(gs - bs),
        "bing_only": sorted(bs - gs),
        "top1_match": bool(gd and bd and gd[0] == bd[0]),
    }

Comparing domains also sidesteps a Serply-specific detail worth knowing: results from certain buckets carry Google’s own tracking parameters on link rather than the clean destination. Domain extraction handles that gracefully; exact URL matching does not.

What the numbers mean

Run this across a set of queries and the pattern is consistent. Well-established factual questions produce high overlap — both engines surface the same canonical sources. Contested, very recent, or thin-content queries produce low overlap, because neither engine has a confident answer and they’re each guessing differently.

That makes Jaccard overlap a usable confidence proxy:

def confidence_band(score: float) -> str:
    if score >= 0.5:
        return "high"      # engines agree on the source set
    if score >= 0.25:
        return "medium"
    return "low"           # little consensus; treat claims cautiously

These thresholds are a starting point, not physics. Calibrate them against your own query distribution — a niche technical domain will run lower across the board than general consumer questions.

Feeding it to the agent

The useful part is passing the signal through, not hiding it:

def cross_checked_search(query: str, num: int = 10) -> str:
    g, b = both(query, num)
    agr = agreement(g, b)
    band = confidence_band(agr["jaccard"])

    shared_first = sorted(
        g, key=lambda r: (domain(r.get("link", "")) not in set(agr["shared"]),
                          r.get("position") or 99)
    )

    lines = [
        f"SOURCE AGREEMENT: {band} (overlap {agr['jaccard']:.0%}, "
        f"{len(agr['shared'])} shared domains)",
    ]
    if band == "low":
        lines.append(
            "Google and Bing largely disagree on sources for this query. "
            "Verify claims across at least two independent domains before "
            "stating them as fact."
        )
    lines.append("")

    for r in shared_first[:num]:
        mark = "✓" if domain(r.get("link", "")) in set(agr["shared"]) else " "
        lines.append(f"{mark} [{r.get('position')}] {r.get('title')}\n"
                     f"  {r.get('link')}\n  {r.get('description', '')}")
    return "\n".join(lines)

Two things happen here. Results that both engines surfaced float to the top, so the agent reads consensus sources first. And on low agreement, the tool output itself instructs the model to be careful — which, as with rate limits, lands far more reliably than the equivalent line in a system prompt.

Cost

This doubles your search spend. That’s a real trade, and it isn’t worth it on every call. A reasonable policy is to cross-check only where being wrong is expensive:

HIGH_STAKES = re.compile(
    r"\b(dosage|legal|regulation|safety|medical|tax|deadline|recall|"
    r"security|vulnerability|compliance)\b", re.I
)


def search(query: str, num: int = 10) -> str:
    if HIGH_STAKES.search(query):
        return cross_checked_search(query, num)
    return format_results(google(query, num))

For most agents the honest answer is that single-engine retrieval is fine for the bulk of traffic, and the cross-check earns its cost on the narrow slice where a confident wrong answer causes real harm.