Blog

Learn how to scrape the web and create tools with public data.

A blocked built-in search on the left, a structured Reddit JSON response on the right

The Knowledge Your Agent Is Locked Out Of

The built-in web search inside coding agents is blocked from Reddit at the crawler level — not rate-limited, blocked. That's the one source where people say what actually happened. A structured endpoint reopens the door.
Reddit threads being sorted into recurring product complaint themes

A Product Feedback Agent That Mines Reddit for Complaints

Your users file tickets. Everyone else complains on Reddit and never tells you. Here's an agent that reads the second group and reports what's actually breaking.
A two-column comparison: vendor listicles and a blocked forum on the left, raw comment data on the right

Serply vs. Built-In Agent Web Search: One Research Question, Measured

We ran the same research question through Claude Code's built-in WebSearch and WebFetch, then through Serply. They tied on the academic paper. One hard access boundary decided everything else.
Two routes to Reddit data, one gated by an approval step

What Reddit Data Actually Costs an AI Agent in 2026

Reddit's per-call rate is cheap. Getting to it — approval, tiers, and rate limits — is where most agent projects stall. An honest look at both paths.
A highly upvoted comment being checked against its author's posting history

Can Your Agent Trust What It Reads on Reddit?

Upvotes measure agreement, not accuracy. Four checks — recency, account history, dissent, and cross-source — that turn a thread into evidence instead of a vibe.
A deep comment tree being pruned down to its highest-scoring branches

Fitting a 3,000-Comment Reddit Thread Into a Context Window

A big thread is a deep tree, not a list. Flattening it naively wastes most of your budget on replies nobody upvoted. Here's the pruning that keeps the signal.
A marketing page and a Reddit thread side by side, saying different things

What Reddit Knows That the Rest of the Web Doesn't

Every other source tells your agent what a product claims. Reddit tells it what happened to the people who bought one. That gap is why agents keep reaching for it.
A browser window re-rendering the same page while a token counter climbs

Why Browser Agents Are So Expensive (And When to Skip Them)

A browser agent pays to re-read the same page on every step. Where the money actually goes, why open source doesn't fix it, and which of your tasks never needed a browser.
A support agent retrieving answers from live documentation pages

A Support Agent That Reads the Docs Instead of Remembering Them

Indexing your documentation into a vector store means re-indexing it forever. Site-scoped search reads the current version every time.
An answer with inline sentence-level source attributions

Designing an Answer UI Where Citations Aren't an Afterthought

Footnote-style links at the end of a paragraph don't get checked. Making provenance visible at the sentence level changes what users trust.