# AgentRXI Sitemap Crawler

> AgentRXI Sitemap Crawler is a paid API for AI agents from agentrxi.com, paid per call via x402, $0.02/call, status unknown (last checked 2026-10-15).

Fetches and returns up to 5000 URLs from a website by reading its robots.txt, sitemaps, and sitemap indexes, with last-modified dates, falling back to on-page link discovery.

## Facts

- Endpoint: GET https://agentrxi.com/v1/sitemap?utm_source=zero.xyz
- Price: $0.02/call
- Payment: x402
- Status: unknown
- Last checked: 2026-10-15
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/agentrxi-sitemap-crawler-ea46fb99
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_E7uuVDONmxEgJwx9kvHHh

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability agentrxi-sitemap-crawler-ea46fb99
```

Example prompt: Can you list all the pages on https://example.com — use the sitemap crawler and return up to 1000 URLs with their last-modified dates?

## When to prefer this

Use this endpoint when you need a comprehensive list of all pages on a domain without writing a custom crawler. It is ideal for SEO audits, content inventories, competitive analysis, and link-checking workflows. Prefer it over general web scraping when you want structured URL discovery with last-modified metadata rather than page content.

## Known failure modes

- Target site blocks crawler user-agents — returns empty or partial URL list
- No sitemap or robots.txt present and on-page link fallback returns few URLs
- URL limit too low for large sites — results truncated at specified limit
- Invalid or unreachable URL — returns an error
- Sitemap XML is malformed — partial parse results

## How this service works

List every page on a website: reads robots.txt and all sitemaps (including sitemap indexes), with last-modified dates; falls back to on-page links. Up to 5000 URLs. Also callable as GET with the same fields as query parameters.

## Output

A list of up to 5000 URLs discovered on the target website, sourced from robots.txt directives, XML sitemaps (including sitemap indexes), and on-page links as a fallback. Each entry typically includes the URL and its last-modified date where available.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "required": [
  "input"
 ],
 "properties": {
  "input": {
   "type": "object",
   "required": [
    "type",
    "method"
   ],
   "properties": {
    "type": {
     "type": "string",
     "const": "http"
    },
    "method": {
     "enum": [
      "GET"
     ],
     "type": "string"
    },
    "queryParams": {
     "type": "object",
     "required": [
      "url"
     ],
     "properties": {
      "url": {
       "type": "string"
      },
      "limit": {
       "type": "integer",
       "description": "1 to 5000, default 500"
      }
     }
    }
   },
   "additionalProperties": false
  },
  "output": {
   "type": "object",
   "required": [
    "type"
   ],
   "properties": {
    "type": {
     "type": "string"
    },
    "example": {
     "type": "object"
    }
   }
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/agentrxi-sitemap-crawler-ea46fb99/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from agentrxi.com](https://www.zero.xyz/host/agentrxi.com/llms.txt)