# x402.orth.sh Sitemap Crawler

> x402.orth.sh Sitemap Crawler is a paid API for AI agents from x402.orth.sh, paid per call via x402, $0.03/call, status unknown (last checked 2026-09-13).

Crawls the sitemap of a given domain and returns all discovered page URLs, supporting recursive sitemap index files, parallel fetching, deduplication, and non-page resource filtering.

## Facts

- Endpoint: GET https://x402.orth.sh/context-dev/web/scrape/sitemap
- Price: $0.03/call
- Payment: x402
- Status: unknown
- Last checked: 2026-09-13
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/x402-orth-sh-sitemap-crawler-4236637a
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_ykq7lUvghmGoNfNfRAW0k

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability x402-orth-sh-sitemap-crawler-4236637a
```

Example prompt: Can you crawl the sitemap for nytimes.com and give me all the page URLs you find — up to 5,000 links?

## When to prefer this

Use this endpoint when you need a comprehensive list of all publicly indexed pages on a domain, especially when the site has a structured sitemap. Prefer this over general web scraping when you want structured URL discovery rather than content extraction. Ideal for SEO audits, content indexing, site monitoring, or feeding URLs into downstream scraping workflows.

## Known failure modes

- Domain has no sitemap.xml or robots.txt pointing to a sitemap — returns empty result or error
- Invalid or unresolvable domain name — validation error returned
- maxLinks out of range (below 1 or above 100,000) — schema validation failure
- Sitemap server is unreachable or returns non-200 status — fetch error
- Rate limiting or timeout on large sitemap index files — partial results or timeout error

## How this service works

Crawls the sitemap of the given domain and returns all discovered page URLs. Supports sitemap index files (recursive), parallel fetching with concurrency control, deduplication, and filters out non-page resources (images, PDFs, etc.).

## Output

A deduplicated list of page URLs discovered by crawling the domain's sitemap (and any linked sitemap index files), filtered to exclude non-page resources like images and PDFs. Returns up to the specified maxLinks count (default 10,000).

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "required": [
  "input"
 ],
 "properties": {
  "input": {
   "type": "object",
   "required": [
    "type",
    "method"
   ],
   "properties": {
    "type": {
     "type": "string",
     "const": "http"
    },
    "method": {
     "enum": [
      "GET"
     ],
     "type": "string"
    },
    "queryParams": {
     "type": "object",
     "properties": {
      "domain": {
       "type": "string",
       "description": "Domain name to crawl sitemaps for (e.g., 'example.com'). The domain will be automatically normalized and validated."
      },
      "maxLinks": {
       "type": "integer",
       "description": "Maximum number of links to return from the sitemap crawl. Defaults to 10,000. Minimum is 1, maximum is 100,000."
      }
     }
    }
   },
   "additionalProperties": false
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/x402-orth-sh-sitemap-crawler-4236637a/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from x402.orth.sh](https://www.zero.xyz/host/x402.orth.sh/llms.txt)