# Bounded Same-Site Web Crawler

> Bounded Same-Site Web Crawler is a paid API for AI agents from x240.jvnclk88.workers.dev, paid per call via x402, $0.03/call, status unknown (last checked 2026-10-29).

Follows internal links from a start URL for up to 10 pages and returns each page's title and text snippet

## Facts

- Endpoint: POST https://x240.jvnclk88.workers.dev/crawl?utm_source=zero.xyz
- Price: $0.03/call
- Payment: x402
- Status: unknown
- Last checked: 2026-10-29
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/bounded-same-site-web-crawler-17cb52e5
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_FP4Rj6ynShvX3dQDgLG7j

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability bounded-same-site-web-crawler-17cb52e5 -d '<json body>'
```

Example prompt: Can you crawl the website at https://docs.example.com and pull back the titles and text snippets from up to 8 pages, following internal links from that starting page?

## When to prefer this

Choose this endpoint when you need to gather content from multiple pages of a single site in one call, rather than fetching URLs one at a time. It is ideal for mapping documentation sites, small business websites, or any starting URL where you want to discover and read linked pages automatically. Prefer the single-URL scraping endpoints when you already know exactly which page(s) you need, or use the batch scrape endpoint if you have a pre-defined list of up to 5 URLs rather than wanting automatic link-following.

## Known failure modes

- Start URL is unreachable or returns a non-200 status — crawl fails with an error
- URL points to a non-HTML resource (PDF, image) — may not yield crawlable links
- Site blocks crawlers via robots.txt or rate limiting — fewer pages returned than requested
- maxPages exceeds the maximum of 10 — request rejected with validation error
- Site has very few internal links — returns fewer pages than maxPages even if not erroring
- Very large or slow pages may cause timeouts before reaching the page limit

## How this service works

Bounded same-site crawl - follows links from a start URL for up to 10 pages (default 5) and returns each page's title and text snippet

## Output

A list of crawled pages (up to the specified maxPages limit), each with its page title and a text snippet extracted from that page's content, covering only pages within the same site as the start URL.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "properties": {
  "url": {
   "type": "string",
   "description": "Public http or https URL to fetch"
  },
  "maxPages": {
   "type": "integer",
   "maximum": 10,
   "minimum": 1,
   "description": "Maximum pages to fetch (default 5, max 10)"
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/bounded-same-site-web-crawler-17cb52e5/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from x240.jvnclk88.workers.dev](https://www.zero.xyz/host/x240.jvnclk88.workers.dev/llms.txt)