# Agent Read Proxy - Sitemap URL Extractor

> Agent Read Proxy - Sitemap URL Extractor is a paid API for AI agents from agent-read-proxy.agent-read-proxy.workers.dev, paid per call via x402, $0.002/call, status unknown (last checked 2026-11-16).

Fetches and parses a website's sitemap to return a structured list of all discoverable page URLs with optional last-modified dates.

## Facts

- Endpoint: GET https://agent-read-proxy.agent-read-proxy.workers.dev/v1/sitemap?utm_source=zero.xyz
- Price: $0.002/call
- Payment: x402
- Status: unknown
- Last checked: 2026-11-16
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/agent-read-proxy-sitemap-url-extractor-c711e89e
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_A56ENRdeABM98aR1iXGo5

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability agent-read-proxy-sitemap-url-extractor-c711e89e
```

Example prompt: Can you fetch the sitemap for cloudflare.com and give me a list of all their pages — up to 500 URLs?

## When to prefer this

Use this endpoint when you need to discover all pages on a website systematically without crawling link-by-link. It is ideal when you want a quick, structured index of a site's content, especially for content audits, SEO analysis, documentation indexing, or preparing a list of pages to subsequently scrape. It is preferred over manual crawling when the site publishes a standard sitemap, and over general web search when you need comprehensive URL enumeration rather than search-relevance ranking.

## Known failure modes

- Site has no sitemap.xml — returns empty or error response
- Sitemap is blocked by robots.txt or requires authentication
- Malformed or non-standard sitemap format causes parsing failure
- limit parameter exceeds maximum of 5000 — validation error
- URL parameter missing or invalid URI format — schema validation error
- Sitemap is very large and processing times out on Cloudflare Workers

## How this service works

Web scraping API for AI agents: turn any public web page into clean text or Markdown with title, author, date and language metadata. $0.002 per page, paid in USDC on Base via x402. No API key or signup.

## Output

Returns a JSON object containing the site's root URL, an array of discovered page URLs (each with optional lastmod date), the total count of URLs found, and a list of sitemap files that were read to produce the result.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "required": [
  "input"
 ],
 "properties": {
  "input": {
   "type": "object",
   "required": [
    "type",
    "method"
   ],
   "properties": {
    "type": {
     "type": "string",
     "const": "http"
    },
    "method": {
     "enum": [
      "GET"
     ],
     "type": "string"
    },
    "queryParams": {
     "type": "object",
     "required": [
      "url"
     ],
     "properties": {
      "url": {
       "type": "string",
       "format": "uri",
       "description": "Any page on the site, or the sitemap URL itself"
      },
      "limit": {
       "type": "integer",
       "default": 1000,
       "maximum": 5000,
       "minimum": 1,
       "description": "Most URLs to return"
      }
     }
    }
   },
   "additionalProperties": false
  },
  "output": {
   "type": "object",
   "required": [
    "type"
   ],
   "properties": {
    "type": {
     "type": "string"
    },
    "example": {
     "type": "object"
    }
   }
  }
 }
}
```

## Response schema (JSON Schema)

```json
{
 "type": "json",
 "example": {
  "site": "https://www.cloudflare.com",
  "urls": [
   {
    "url": "https://www.cloudflare.com/learning/",
    "lastmod": "2026-10-01"
   }
  ],
  "count": 1,
  "sitemaps_read": [
   "https://www.cloudflare.com/sitemap.xml"
  ]
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/agent-read-proxy-sitemap-url-extractor-c711e89e/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from agent-read-proxy.agent-read-proxy.workers.dev](https://www.zero.xyz/host/agent-read-proxy.agent-read-proxy.workers.dev/llms.txt)
