# robots.txt Linter and Allow/Disallow Analyzer

> robots.txt Linter and Allow/Disallow Analyzer is a paid API for AI agents from api.ozdreamtools.de, paid per call via x402, $0.05/call, status unknown (last checked 2026-10-10).

Validates a robots.txt file against RFC 9309, reporting all checks (not just failures), and computes per-URL allow/disallow decisions with winning rule details for a given user agent.

## Facts

- Endpoint: POST https://api.ozdreamtools.de/api/robots/lint?utm_source=zero.xyz
- Price: $0.05/call
- Payment: x402
- Status: unknown
- Last checked: 2026-10-10
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/robots-txt-linter-and-allow-disallow-analyzer-fb2f57e5
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_fY6z7UCP0br1nhiiCndzJ

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability robots-txt-linter-and-allow-disallow-analyzer-fb2f57e5 -d '<json body>'
```

Example prompt: Can you lint this robots.txt content and tell me whether Googlebot is allowed to access /blog/private and /admin — I want to know which rule wins and why: 'User-agent: *\nDisallow: /admin\nAllow: /blog'

## When to prefer this

Choose this endpoint when you need exhaustive RFC 9309 compliance reporting on a robots.txt — not just error detection but confirmation of every passing check too — and when you need to trace exactly why a specific URL is allowed or disallowed for a named crawler, including the winning rule and tiebreak. It makes no outbound requests and processes only caller-supplied content, making it suitable for CI/CD pipelines, pre-deploy audits, or debugging without exposing live infrastructure.

## Known failure modes

- Malformed or missing robots.txt content returns a parsing error
- Invalid or missing user agent causes decision computation to fail
- Empty URL list means no per-URL decisions are returned
- Extremely large robots.txt content may hit size validation limits
- RFC ambiguity edge cases are noted but may produce unexpected interpretations

## How this service works

Lints and validates a caller-supplied robots.txt against RFC 9309 and reports every check, not only the failures: group structure, product token charset, path patterns, special characters, other records, size limit. Given a user agent and URLs it recomputes the allow/disallow decision per URL, naming the winning rule, its octet length and the tiebreak that applied. Where the RFC is silent or contradicts itself, the reading used is stated. Makes no outbound request.

## Output

A detailed lint report covering every RFC 9309 check (group structure, product token charset, path patterns, special characters, other records, size limit) with pass/fail status for each; plus per-URL allow/disallow decisions for the specified user agent, naming the winning rule, its octet length, and the tiebreak logic applied, along with notes on any RFC ambiguities and the interpretation used.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "properties": {
  "urls": {
   "type": "array",
   "items": {
    "type": "string"
   },
   "description": "URLs or paths whose access decision should be recomputed, at most 50"
  },
  "robots_txt": {
   "type": "string",
   "description": "Contents of the robots.txt to check"
  },
  "user_agent": {
   "type": "string",
   "description": "Product token to evaluate for, e.g. 'GPTBot'. A full User-Agent header is resolved to the token it contains. Required when 'urls' is given."
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/robots-txt-linter-and-allow-disallow-analyzer-fb2f57e5/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from api.ozdreamtools.de](https://www.zero.xyz/host/api.ozdreamtools.de/llms.txt)