# Sitemap Crawler - x402.orthogonal.com

> Sitemap Crawler - x402.orthogonal.com is a paid API for AI agents from x402.orthogonal.com, paid per call via x402, $0.03/call, status unknown (last checked 2026-09-16).

Crawls a domain's sitemap (including recursive sitemap index files) and returns all discovered page URLs, deduplicated and filtered to pages only.

## Facts

- Endpoint: GET https://x402.orthogonal.com/context-dev/web/scrape/sitemap
- Price: $0.03/call
- Payment: x402
- Status: unknown
- Last checked: 2026-09-16
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/sitemap-crawler-x402-orthogonal-com-5777305e
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_4Cc9Xs5sf9U3NAYQ_OfqG

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability sitemap-crawler-x402-orthogonal-com-5777305e
```

Example prompt: Can you crawl the sitemap for stripe.com and give me all the page URLs you discover, up to 5,000 links?

## When to prefer this

Use this endpoint when you need to enumerate all pages of a website via its sitemap, especially for SEO audits, content mapping, link checking, or building a crawl queue. It is preferable over generic URL scrapers when you want structured, deduplicated page discovery at scale with recursive sitemap index support. Best for domains that maintain a sitemap.xml.

## Known failure modes

- Domain has no sitemap.xml — returns empty or error
- Invalid or malformed domain name — validation error
- Domain is unreachable or returns non-200 — fetch failure
- Sitemap is password-protected or blocked by robots.txt
- maxLinks exceeds 100,000 or is less than 1 — validation error
- Very large sitemap index may hit timeout or concurrency limits

## How this service works

Crawls the sitemap of the given domain and returns all discovered page URLs. Supports sitemap index files (recursive), parallel fetching with concurrency control, deduplication, and filters out non-page resources (images, PDFs, etc.).

## Output

A list of all discovered page URLs found across the domain's sitemap and any nested sitemap index files. Results are deduplicated and filtered to exclude non-page resources such as images, PDFs, and other media files. The response respects the maxLinks cap and is fetched with parallel concurrency control.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "required": [
  "input"
 ],
 "properties": {
  "input": {
   "type": "object",
   "required": [
    "type",
    "method"
   ],
   "properties": {
    "type": {
     "type": "string",
     "const": "http"
    },
    "method": {
     "enum": [
      "GET"
     ],
     "type": "string"
    },
    "queryParams": {
     "type": "object",
     "properties": {
      "domain": {
       "type": "string",
       "description": "Domain name to crawl sitemaps for (e.g., 'example.com'). The domain will be automatically normalized and validated."
      },
      "maxLinks": {
       "type": "integer",
       "description": "Maximum number of links to return from the sitemap crawl. Defaults to 10,000. Minimum is 1, maximum is 100,000."
      }
     }
    }
   },
   "additionalProperties": false
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/sitemap-crawler-x402-orthogonal-com-5777305e/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from x402.orthogonal.com](https://www.zero.xyz/host/x402.orthogonal.com/llms.txt)