A Video Research Agent That Doesn't Pretend to Watch

• What you actually get
• Designing the tool around the limitation
• Getting real content
• Complementing with web results
• A cheap output check

There’s a specific way video agents go wrong. The tool returns ten video titles, and the model — which has seen millions of video descriptions in training — writes a confident summary of what’s in those videos. It hasn’t watched anything. It’s pattern-matching on titles.

Being clear-eyed about what the video endpoint gives you is most of the fix.

What you actually get

import os
import requests
from urllib.parse import quote_plus

API_KEY = os.environ["SERPLY_API_KEY"]


def video_search(query: str, num: int = 10) -> list[dict]:
    resp = requests.get(
        f"https://api.serply.io/v1/video/q={quote_plus(query)}&num={num}",
        headers={"X-Api-Key": API_KEY, "X-Proxy-Location": "US"},
        timeout=30,
    )
    resp.raise_for_status()
    return resp.json().get("results", [])


for v in video_search("rust async runtime comparison"):
    print(v.get("realPosition"), v.get("title"))
    print("   ", v.get("link"))

Results come back under results, each with title, link, description, and realPosition. The response also carries the same SERP-feature arrays as web search — ads, related_searches, related_questions, and so on — populated only when the page had them.

The critical detail: description is usually empty. Google’s video results rarely include a snippet, so in practice you have a title and a URL. No transcript, no duration, no view count, no channel metadata. Any agent that reports those numbers is inventing them.

Also note that link often carries Google’s own tracking parameters (sa=, ved=, usg=) rather than a bare video URL.

Designing the tool around the limitation

The tool description should state what’s missing, because the model will otherwise fill the gap:

VIDEO_TOOL = {
    "name": "video_search",
    "description": (
        "Search for videos. Returns ONLY titles, URLs, and rank position — no "
        "transcripts, view counts, durations, upload dates, or channel names. "
        "Do not state or estimate any of those. To learn what a video actually "
        "covers, call read_page on its URL."
    ),
    "input_schema": {
        "type": "object",
        "properties": {
            "query": {"type": "string", "description": "Video search query, keyword style."},
            "num": {"type": "integer", "description": "Number of results, 1-50.", "default": 10},
        },
        "required": ["query"],
    },
}

Naming the specific fields that don’t exist works better than a general caution. “No metadata” is abstract; “no view counts, durations, upload dates” is concrete enough that the model won’t produce them.

Getting real content

The video page itself has description text, and often a transcript panel:

def read_page(url: str) -> str | None:
    try:
        resp = requests.post(
            "https://api.serply.io/v1/request",
            headers={"X-Api-Key": API_KEY, "Content-Type": "application/json"},
            json={"url": url, "response_type": "markdown"},
            timeout=60,
        )
        resp.raise_for_status()
        return resp.text
    except requests.RequestException:
        return None

How much you get back varies a lot by platform and page. Sometimes it’s the full description and chapter list; sometimes it’s mostly navigation. Treat a thin result as “unknown,” not “the video has no content”:

def content_quality(text: str | None) -> str:
    if not text:
        return "unavailable"
    words = len(text.split())
    if words < 120:
        return "thin"
    return "usable"

Complementing with web results

For most research questions the honest architecture is video-as-supplement. Web pages are where the substance is; videos are a pointer to a person who explained it:

from concurrent.futures import ThreadPoolExecutor


def web_search(query: str, num: int = 10) -> list[dict]:
    resp = requests.get(
        f"https://api.serply.io/v1/search/q={quote_plus(query)}&num={num}",
        headers={"X-Api-Key": API_KEY},
        timeout=30,
    )
    resp.raise_for_status()
    return resp.json().get("results", [])


def research(topic: str) -> dict:
    with ThreadPoolExecutor(max_workers=2) as pool:
        w = pool.submit(web_search, topic, 10)
        v = pool.submit(video_search, topic, 10)
        web, videos = w.result(), v.result()

    with ThreadPoolExecutor(max_workers=4) as pool:
        pages = list(pool.map(lambda r: read_page(r["link"]), web[:4]))

    return {
        "articles": [
            {"title": r.get("title"), "url": r.get("link"), "text": t}
            for r, t in zip(web[:4], pages) if t
        ],
        "videos": [
            {"title": v.get("title"), "url": v.get("link"),
             "rank": v.get("realPosition")}
            for v in videos[:8]
        ],
    }

Then keep the two source types separate in the prompt:

SYSTEM = """You write research briefs.

You have two kinds of sources:
1. ARTICLES — full text. Ground all factual claims in these and cite them.
2. VIDEOS — titles and URLs only. You have NOT seen their contents.

For videos, you may list them as further viewing and describe what the TITLE
suggests, explicitly framed as such ("titled as covering X"). Never state what
a video says, shows, concludes, or demonstrates. Never estimate length,
popularity, or recency."""

Separating sources by epistemic status inside the prompt is more reliable than one blanket warning. The model is being asked to keep two different confidence levels straight, and labelling them in the input is how you make that easy.

A cheap output check

import re

BANNED = re.compile(
    r"\b(the video (shows|explains|demonstrates|covers|argues)|"
    r"in this video|as the presenter|the speaker (says|notes)|"
    r"\d[\d,.]*[km]? views|\d+ minutes? long)\b", re.I
)


def check(answer: str, watched: bool = False) -> list[str]:
    return [m.group(0) for m in BANNED.finditer(answer)] if not watched else []

Crude, and it will occasionally flag something legitimate. But it catches the exact failure this whole design exists to prevent, and it costs nothing to run on every response.