# Crawlspur URL-Regel Crawl-Delay Check

> Crawlspur URL-Regel Crawl-Delay Check is a paid API for AI agents from crawlspur.halowerk.com, paid per call via x402, $0.001/call, status unknown (last checked 2026-09-16).

Checks the declared crawl delay setting for a URL rule object by fetching and analyzing the relevant robots.txt directive.

## Facts

- Endpoint: POST https://crawlspur.halowerk.com/v1/pruef/crawlspur/crawl-delay/url-regel
- Price: $0.001/call
- Payment: x402
- Status: unknown
- Last checked: 2026-09-16
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/crawlspur-url-regel-crawl-delay-check-f332b82c
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_HxquSRf12iOWc4e4wffW_

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability crawlspur-url-regel-crawl-delay-check-f332b82c -d '<json body>'
```

Example prompt: Check the declared crawl delay for the URL rule at https://example.com/products/* — I want to know what crawl delay is set in its robots.txt for that URL pattern.

## When to prefer this

Use this endpoint when you need to specifically check the crawl-delay directive for a URL rule object in a robots.txt file, particularly during SEO audits, crawler configuration, or troubleshooting indexing delays. Choose this over generic robots.txt parsers when you need a targeted per-URL-rule crawl delay check as part of a systematic crawl configuration audit.

## Known failure modes

- Target URL is unreachable or returns non-200 HTTP status
- robots.txt file is missing or inaccessible
- No crawl delay directive found for the specified URL rule
- Invalid URL format provided
- CSS selector matches no element or multiple elements
- Request timeout on slow or unresponsive domains

## How this service works

Prüft am Objekt url-regel den Befund deklarierte Crawl-Verzögerung.

## Output

Returns the declared crawl delay value found in the robots.txt for the specified URL rule, indicating how many seconds the crawl delay is set to (if any), along with details about the matching rule or directive analyzed.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "properties": {
  "ziel": {
   "oneOf": [
    {
     "type": "string",
     "maxLength": 2048,
     "minLength": 1,
     "description": "Öffentliche HTTP(S)-Adresse; bei DNS-Befunden Domain oder öffentliche IP."
    },
    {
     "type": "object",
     "required": [
      "adresse"
     ],
     "properties": {
      "adresse": {
       "type": "string",
       "maxLength": 2048,
       "minLength": 1
      },
      "selektor": {
       "type": "string",
       "maxLength": 240,
       "minLength": 1,
       "description": "CSS-Selektor: genau ein Objekt, sonst erster Treffer des Objektfilters."
      },
      "vergleich": {
       "type": "string",
       "maxLength": 2048,
       "description": "Öffentliche Vergleichsadresse für Link-, Sitemap- oder Sprachbefunde."
      },
      "user_agent": {
       "type": "string",
       "pattern": "^[A-Za-z0-9_-]{1,80}$"
      },
      "dkim_selektor": {
       "type": "string",
       "pattern": "^[A-Za-z0-9_-]{1,63}$"
      }
     },
     "additionalProperties": false
    }
   ]
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/crawlspur-url-regel-crawl-delay-check-f332b82c/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from crawlspur.halowerk.com](https://www.zero.xyz/host/crawlspur.halowerk.com/llms.txt)