# x402serps Website Crawler

> x402serps Website Crawler is a paid API for AI agents from x402serps.com, paid per call via x402, $0.02/call, status unknown (last checked 2026-11-12).

Breadth-first crawls a website starting from a given URL, returning clean markdown content for up to 50 pages, priced per page of the requested limit

## Facts

- Endpoint: POST https://x402serps.com/api/v1/crawl?utm_source=zero.xyz
- Price: $0.02/call
- Payment: x402
- Status: unknown
- Last checked: 2026-11-12
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/x402serps-website-crawler-c5186de6
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_4G7X0gbel-gVH4RljKqjo

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability x402serps-website-crawler-c5186de6 -d '<json body>'
```

Example prompt: Crawl the docs site at https://docs.example.com starting from the homepage, grab up to 20 pages, limit each page to 5000 characters, and only follow paths that contain /guides — give me everything as clean markdown.

## When to prefer this

Choose this endpoint when you need to systematically extract content from multiple pages of a website in a single call, especially for documentation sites, marketing sites, or any domain where breadth-first traversal from a known starting URL is needed. It is preferable over single-page fetchers when you need 2–50 pages in one shot. The path filter (include) makes it ideal for scoped crawls like only grabbing /blog or /docs paths. Use the sibling fetch endpoint instead if you have a specific list of URLs rather than needing discovery.

## Known failure modes

- Invalid or unreachable start URL returns an error field on the page object
- Pages that exceed maxChars are truncated to the specified limit
- Limit exceeding 50 may be rejected or capped
- Non-public or auth-gated pages will fail with an error per page
- Network timeouts on slow pages may result in partial results or error entries
- Payment failure (insufficient USDC) will block the request entirely

## How this service works

x402serps: crawl a website — breadth-first from a start URL on the same host, returning clean markdown for up to 50 pages; priced per page of the requested limit

## Output

A JSON object containing the start URL, total page count crawled, and an array of page objects each with its URL, title, markdown content (up to maxChars characters), and any error message if a page failed to load. Up to 50 pages are returned depending on the requested limit.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "properties": {
  "url": {
   "type": "string",
   "description": "Start URL"
  },
  "limit": {
   "type": "number",
   "description": "Max pages 1-50, default 10 (you are charged for the limit)"
  },
  "include": {
   "type": "string",
   "description": "Only follow paths containing this substring"
  },
  "maxChars": {
   "type": "number",
   "description": "Max chars per page, default 8000"
  }
 }
}
```

## Response schema (JSON Schema)

```json
{
 "type": "json",
 "example": {
  "count": 5,
  "pages": [
   {
    "url": "https://www.x402.org/",
    "title": "x402",
    "markdown": "# x402 ..."
   }
  ],
  "start": "https://www.x402.org/"
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/x402serps-website-crawler-c5186de6/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from x402serps.com](https://www.zero.xyz/host/x402serps.com/llms.txt)
