# llms.txt & AI-Crawler Audit API

> llms.txt & AI-Crawler Audit API is a paid API for AI agents from x402-extract-service.onrender.com, paid per call via x402, $0.02/call, status unknown (last checked 2026-09-17).

Audits a domain's llms.txt file for presence, validity, dead links, and AI crawler access rules in robots.txt

## Facts

- Endpoint: POST https://x402-extract-service.onrender.com/llmsaudit
- Price: $0.02/call
- Payment: x402
- Status: unknown
- Last checked: 2026-09-17
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/llms-txt-ai-crawler-audit-api-523d8214
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_LLy-9lgGwZrqx---JrJUp

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability llms-txt-ai-crawler-audit-api-523d8214 -d '<json body>'
```

Example prompt: Can you audit example.com to check whether it has a valid llms.txt file, which AI crawlers like GPTBot or ClaudeBot are allowed or blocked, and whether any links in the file are broken?

## When to prefer this

Choose this endpoint when you specifically need to audit a domain's llms.txt file for AI crawler compatibility, link health, and robots.txt AI-bot permissions. It is purpose-built for the llms.txt standard and AI crawler access compliance, unlike generic broken-link checkers or robots.txt parsers that don't understand the llms.txt format or AI-specific crawler rules.

## Known failure modes

- Domain does not have a llms.txt file — returns present:false with a FAIL summary
- Dead links found in llms.txt — listed in deadLinks array with HTTP status codes
- AI crawler blocked — reflected in crawlers array with access:blocked
- Domain unreachable or DNS failure — likely returns an error or timeout
- Invalid domain format provided — may return a validation error
- Payment failure via x402 — call does not proceed

## How this service works

URL-to-clean-markdown extraction API, on-demand broken-link scan API, pre-deploy link audit API, DNS health audit API, llms.txt / AI-crawler audit API, spec-linted llms.txt audit API, email-deliverability (SPF/DKIM/DMARC) audit API, PDF-to-markdown conversion API, technology-stack fingerprinting API, security-headers audit API, and PDF metadata-intel API for coding agents. Paid per call in USDC via x402 on Base.

## Output

Returns a JSON object with: whether llms.txt is present and its byte size, a list of AI crawlers (e.g. GPTBot, ClaudeBot) with allowed/blocked status, any dead links found in llms.txt with HTTP status codes, a human-readable summary string, total links checked, audit duration in milliseconds, and whether the file passed validation overall.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "properties": {
  "domain": {
   "type": "string",
   "description": "Bare domain to audit (no URL), e.g. \"example.com\"."
  }
 }
}
```

## Response schema (JSON Schema)

```json
{
 "type": "json",
 "example": {
  "ok": true,
  "valid": false,
  "domain": "example.com",
  "llmsTxt": {
   "url": "https://example.com/llms.txt",
   "bytes": 48210,
   "present": true
  },
  "summary": "FAIL: llms.txt present (48210 bytes), 120 link(s) checked, 1 dead; 1 AI crawler(s) blocked in robots.txt (ClaudeBot).",
  "crawlers": [
   {
    "bot": "GPTBot",
    "access": "allowed"
   },
   {
    "bot": "ClaudeBot",
    "access": "blocked"
   }
  ],
  "checkedAt": "2026-09-11T00:00:00.000Z",
  "deadLinks": [
   {
    "url": "https://example.com/old-page",
    "error": "HTTP 404",
    "status": 404
   }
  ],
  "durationMs": 8120,
  "linksChecked": 120,
  "pricePaidAtomic": "20000"
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/llms-txt-ai-crawler-audit-api-523d8214/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from x402-extract-service.onrender.com](https://www.zero.xyz/host/x402-extract-service.onrender.com/llms.txt)