# Crawlspur Sitemap robots.txt Permission Check

> Crawlspur Sitemap robots.txt Permission Check is a paid API for AI agents from crawlspur.halowerk.com, paid per call via x402, $0.001/call, status unknown (last checked 2026-09-14).

Checks whether a given sitemap URL is permitted (allowed) by the site's robots.txt file.

## Facts

- Endpoint: POST https://crawlspur.halowerk.com/v1/pruef/crawlspur/robots-erlaubt/sitemap
- Price: $0.001/call
- Payment: x402
- Status: unknown
- Last checked: 2026-09-14
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/crawlspur-sitemap-robots-txt-permission-check-931768ef
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_9CQBwVNq2iNM6ITuT7D36

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability crawlspur-sitemap-robots-txt-permission-check-931768ef -d '<json body>'
```

Example prompt: Can you check whether my sitemap at https://example.com/sitemap.xml is actually allowed by the robots.txt file on that domain?

## When to prefer this

Use this endpoint when you need to programmatically verify whether a specific sitemap URL is crawlable according to the site's robots.txt, especially during SEO audits, pre-launch checks, or diagnosing why a sitemap is not being indexed. Prefer this over general robots.txt parsers when you want a direct pass/fail answer scoped specifically to the sitemap object.

## Known failure modes

- Target URL unreachable or returns non-2xx HTTP status
- robots.txt file not found or inaccessible at the domain
- Malformed or invalid sitemap URL provided
- URL exceeds maximum length of 2048 characters
- DNS resolution failure for the target domain
- Timeout when fetching robots.txt or sitemap

## How this service works

Prüft am Objekt sitemap den Befund Freigabe durch robots.txt.

## Output

Returns the finding (Befund) indicating whether the specified sitemap URL is permitted or disallowed by the site's robots.txt file, including the relevant permission status derived from parsing the robots.txt directives.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "properties": {
  "ziel": {
   "oneOf": [
    {
     "type": "string",
     "maxLength": 2048,
     "minLength": 1,
     "description": "Öffentliche HTTP(S)-Adresse; bei DNS-Befunden Domain oder öffentliche IP."
    },
    {
     "type": "object",
     "required": [
      "adresse"
     ],
     "properties": {
      "adresse": {
       "type": "string",
       "maxLength": 2048,
       "minLength": 1
      },
      "selektor": {
       "type": "string",
       "maxLength": 240,
       "minLength": 1,
       "description": "CSS-Selektor: genau ein Objekt, sonst erster Treffer des Objektfilters."
      },
      "vergleich": {
       "type": "string",
       "maxLength": 2048,
       "description": "Öffentliche Vergleichsadresse für Link-, Sitemap- oder Sprachbefunde."
      },
      "user_agent": {
       "type": "string",
       "pattern": "^[A-Za-z0-9_-]{1,80}$"
      },
      "dkim_selektor": {
       "type": "string",
       "pattern": "^[A-Za-z0-9_-]{1,63}$"
      }
     },
     "additionalProperties": false
    }
   ]
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/crawlspur-sitemap-robots-txt-permission-check-931768ef/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from crawlspur.halowerk.com](https://www.zero.xyz/host/crawlspur.halowerk.com/llms.txt)