# ScrapFly Crawl Sitemap

> ScrapFly Crawl Sitemap is a paid API for AI agents from test-scrapfly.orbonomy.xyz, paid per call via x402, $0.02/call, status unknown (last checked 2026-09-15).

Crawls and extracts the sitemap of a given website URL, returning structured sitemap data

## Facts

- Endpoint: POST https://test-scrapfly.orbonomy.xyz/api/crawl/sitemap
- Price: $0.02/call
- Payment: x402
- Status: unknown
- Last checked: 2026-09-15
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/scrapfly-crawl-sitemap-65d16770
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_ooSU1zMi7_DlSsqaysp0a

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability scrapfly-crawl-sitemap-65d16770 -d '<json body>'
```

Example prompt: Can you crawl the sitemap of https://example.com and give me all the URLs listed there so I can see what pages the site has?

## When to prefer this

Use this endpoint when you need to programmatically discover all pages on a website by crawling its sitemap, especially when you need to enumerate site structure for SEO analysis, content indexing, or link discovery. Prefer this over manual sitemap fetching when you need anti-bot bypass and structured output.

## Known failure modes

- URL is unreachable or returns a non-200 status — success: false returned
- Website has no sitemap.xml or robots.txt sitemap reference — empty or null data returned
- URL is malformed or missing — request validation error
- Rate limiting or bot protection on target site blocks the crawl
- Sitemap is too large or deeply nested and times out

## How this service works

69 endpoints for data, content generation, and web scraping. Pay per request via x402 on Base chain.

## Output

Returns a success boolean and a data object containing the parsed sitemap contents, including discovered URLs and any nested sitemap structure found at the provided website URL.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "required": [
  "url"
 ],
 "properties": {
  "url": {
   "type": "string",
   "description": "url"
  }
 }
}
```

## Response schema (JSON Schema)

```json
{
 "type": "object",
 "required": [
  "success"
 ],
 "properties": {
  "data": {
   "type": "object"
  },
  "success": {
   "type": "boolean"
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/scrapfly-crawl-sitemap-65d16770/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from test-scrapfly.orbonomy.xyz](https://www.zero.xyz/host/test-scrapfly.orbonomy.xyz/llms.txt)