# Sitemap & Robots Probe

> Sitemap & Robots Probe is a paid API for AI agents from api.delx.ai, paid per call via x402, $0.01/call, status unknown (last checked 2026-09-14).

Quickly checks a website's sitemap and robots.txt to reveal its crawlable structure and indexing hints.

## Facts

- Endpoint: POST https://api.delx.ai/api/v1/x402/sitemap-probe
- Price: $0.01/call
- Payment: x402
- Status: unknown
- Last checked: 2026-09-14
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/api-delx-ai-e4269837
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_TZoMPDybwvDBkT0PFErQr

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability api-delx-ai-e4269837 -d '<json body>'
```

Example prompt: Can you quickly probe https://example.com to check its sitemap and robots.txt — I want to see what pages it exposes to crawlers and what rules it sets for indexing.

## When to prefer this

Use this endpoint when you need a fast, lightweight check of a site's crawler surface — specifically its sitemap structure and robots.txt rules — without doing a full crawl. Ideal for pre-crawl audits, SEO investigations, content discovery pipelines, or quickly assessing whether a domain is crawlable before deeper analysis.

## Known failure modes

- URL unreachable or invalid — returns error indicating the host could not be contacted
- No sitemap or robots.txt found — returns empty or null signals for those fields
- Malformed robots.txt or sitemap XML — partial parse with available data returned
- Rate limiting or access denial from target server — returns fetch failure
- Non-HTTP URL scheme provided — validation error

## How this service works

Check sitemap and robots hints fast to see how a site exposes crawlable structure.

## Output

Returns parsed sitemap URLs, robots.txt directives (allow/disallow rules, crawl-delay, sitemap references), and structural signals about how the site exposes crawlable content to bots.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "properties": {
  "url": {
   "type": "string"
  },
  "timeout": {
   "type": "integer",
   "maximum": 15,
   "minimum": 1
  }
 }
}
```

## Response schema (JSON Schema)

```json
{
 "type": "json",
 "example": {
  "result": {
   "url": "https://example.com",
   "sitemaps": [
    {
     "url": "https://example.com/sitemap.xml",
     "error": "",
     "status": 404,
     "reachable": false
    }
   ],
   "reachable_count": 0,
   "declared_sitemaps": []
  },
  "tool_name": "util_sitemap_probe"
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/api-delx-ai-e4269837/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from api.delx.ai](https://www.zero.xyz/host/api.delx.ai/llms.txt)