Blog
Learn how to scrape the web and create tools with public data.
The Knowledge Your Agent Is Locked Out Of
The built-in web search inside coding agents is blocked from Reddit at the crawler level — not rate-limited, blocked. That's the one source where people say what actually happened. A structured endpoint reopens the door.
A Product Feedback Agent That Mines Reddit for Complaints
Your users file tickets. Everyone else complains on Reddit and never tells you. Here's an agent that reads the second group and reports what's actually breaking.
Serply vs. Built-In Agent Web Search: One Research Question, Measured
We ran the same research question through Claude Code's built-in WebSearch and WebFetch, then through Serply. They tied on the academic paper. One hard access boundary decided everything else.
What Reddit Data Actually Costs an AI Agent in 2026
Reddit's per-call rate is cheap. Getting to it — approval, tiers, and rate limits — is where most agent projects stall. An honest look at both paths.
Can Your Agent Trust What It Reads on Reddit?
Upvotes measure agreement, not accuracy. Four checks — recency, account history, dissent, and cross-source — that turn a thread into evidence instead of a vibe.
Fitting a 3,000-Comment Reddit Thread Into a Context Window
A big thread is a deep tree, not a list. Flattening it naively wastes most of your budget on replies nobody upvoted. Here's the pruning that keeps the signal.
What Reddit Knows That the Rest of the Web Doesn't
Every other source tells your agent what a product claims. Reddit tells it what happened to the people who bought one. That gap is why agents keep reaching for it.
Why Browser Agents Are So Expensive (And When to Skip Them)
A browser agent pays to re-read the same page on every step. Where the money actually goes, why open source doesn't fix it, and which of your tasks never needed a browser.
A Support Agent That Reads the Docs Instead of Remembering Them
Indexing your documentation into a vector store means re-indexing it forever. Site-scoped search reads the current version every time.
Designing an Answer UI Where Citations Aren't an Afterthought
Footnote-style links at the end of a paragraph don't get checked. Making provenance visible at the sentence level changes what users trust.