What Your Agent Should Do When the Search API Says 429

Retrying a rate limit is the wrong instinct. Here's how to build search tools that degrade gracefully instead of hammering a closed door.

Profile picture of Serply
Serply
A rate limit counter approaching zero with a backoff curve

Agents amplify rate limit problems in a specific way: when a tool fails, the model tries again. It doesn’t know that “429” means “wait,” it just sees a failed call and reasons that maybe a slightly different query will work. Three agents in a group chat doing this simultaneously will exhaust a generous quota in under a minute.

The fix is mostly not backoff. It’s making the failure legible to the model.

Read the headers before you hit the wall

Serply returns two headers you can steer on:

  • x-ratelimit-requests-limit — the ceiling for the window
  • x-ratelimit-requests-remaining — what’s left

They’re on normal 200 responses, which means you can slow down before getting rejected:

import os
import time
import threading
import requests
from urllib.parse import quote_plus

API_KEY = os.environ["SERPLY_API_KEY"]


class RateAwareClient:
    def __init__(self, floor: int = 20):
        self.floor = floor
        self.remaining: int | None = None
        self.limit: int | None = None
        self._lock = threading.Lock()

    def _note(self, resp: requests.Response) -> None:
        with self._lock:
            rem = resp.headers.get("x-ratelimit-requests-remaining")
            lim = resp.headers.get("x-ratelimit-requests-limit")
            if rem is not None:
                self.remaining = int(rem)
            if lim is not None:
                self.limit = int(lim)

    @property
    def near_limit(self) -> bool:
        return self.remaining is not None and self.remaining < self.floor

    def search(self, query: str, num: int = 10) -> dict:
        resp = requests.get(
            f"https://api.serply.io/v1/search/q={quote_plus(query)}&num={num}",
            headers={"X-Api-Key": API_KEY},
            timeout=30,
        )
        self._note(resp)
        resp.raise_for_status()
        return resp.json()

Reserving a floor — refusing to spend the last twenty requests on speculative calls — leaves headroom for the request that actually matters. Without it, the first thing to fail is whatever the user asked for most recently.

Return the error, don’t raise it

This is the single highest-leverage change. Compare two tool implementations:

# Bad: the model sees a stack trace and tries again
def web_search(query: str) -> str:
    resp = requests.get(url, headers=headers, timeout=30)
    resp.raise_for_status()
    return format(resp.json())


# Good: the model sees an instruction
def web_search(query: str) -> str:
    resp = requests.get(url, headers=headers, timeout=30)
    if resp.status_code == 429:
        return (
            "SEARCH UNAVAILABLE: rate limit reached. Do not call this tool again "
            "in this conversation. Answer using information you have already "
            "gathered, and tell the user that live search was unavailable."
        )
    resp.raise_for_status()
    return format(resp.json())

The second version reliably produces a graceful answer. The first produces four more attempts and then a confused apology. Models follow instructions in tool output far more consistently than instructions in a system prompt about hypothetical future errors — the tool result is right there in context at the moment the decision gets made.

Backoff for the calls that should retry

Rate limits shouldn’t be retried immediately, but timeouts and 5xx responses should:

import random


def with_retry(fn, attempts: int = 3, base: float = 0.5):
    for i in range(attempts):
        try:
            return fn()
        except requests.HTTPError as e:
            code = e.response.status_code if e.response is not None else 0
            if code == 429 or (400 <= code < 500 and code != 408):
                raise  # never retry rate limits or client errors
            if i == attempts - 1:
                raise
        except (requests.Timeout, requests.ConnectionError):
            if i == attempts - 1:
                raise
        time.sleep(base * (2 ** i) + random.uniform(0, 0.3))

The jitter isn’t decorative. Parallel tool calls that all fail at the same moment will otherwise all retry at the same moment, which is how a transient blip becomes a thundering herd.

Note the explicit exclusion of 422 and 404 from retries. A malformed query won’t become well-formed on the second attempt, and Serply returns 422 for semantically invalid requests — retrying just spends quota to get the same answer.

Per-run budgets

Headers protect the account. A budget protects the individual run:

class Budget:
    def __init__(self, limit: int):
        self.limit, self.used = limit, 0

    def check(self) -> str | None:
        if self.used >= self.limit:
            return (
                f"SEARCH BUDGET EXHAUSTED ({self.limit} calls). Stop searching "
                "and answer with what you have."
            )
        self.used += 1
        return None

Wire it into the tool:

def web_search(query: str, num: int = 10, budget: Budget = None) -> str:
    if budget and (msg := budget.check()):
        return msg
    ...

Twenty-five calls is a reasonable ceiling for a research question. The number matters less than having one: an unbounded agent on a question with no good answer will search until something else stops it.

Deduplicate before you spend

A surprising share of rate limit pressure is the same query twice. In a multi-agent setup it’s routine — the researcher searches, the critic verifies with the identical string.

from functools import lru_cache


@lru_cache(maxsize=512)
def _search_cached(query: str, num: int) -> str:
    return _search_uncached(query, num)

An in-process LRU is enough for a single run. For a service, put a short TTL cache in front — five minutes for general queries, under a minute for anything about prices or news, where staleness defeats the purpose.

Concurrency ceiling

Finally, cap parallel calls. ThreadPoolExecutor(max_workers=20) on a fan-out of forty queries is the fastest way to find your rate limit:

from concurrent.futures import ThreadPoolExecutor

with ThreadPoolExecutor(max_workers=5) as pool:
    results = list(pool.map(client.search, queries))

Five is a sane default. If you need more throughput, raise it deliberately while watching x-ratelimit-requests-remaining, rather than discovering the ceiling in production.