# AI Crawler Access Checker

> AI Crawler Access Checker is a paid API for AI agents from api.timzinin.com, paid per call via x402, $0.01/call, status unknown (last checked 2026-09-15).

Checks whether AI bots and crawlers (GPTBot, ClaudeBot, etc.) are allowed or blocked by a website's robots.txt and llms.txt, returning per-bot verdicts and AI access scores

## Facts

- Endpoint: POST https://api.timzinin.com/api/ai-crawler-access-checker
- Price: $0.01/call
- Payment: x402
- Status: unknown
- Last checked: 2026-09-15
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/ai-crawler-access-checker-2c5b150c
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_fQBcP6Xgb9mSIa9S6Lq_u

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability ai-crawler-access-checker-2c5b150c -d '<json body>'
```

Example prompt: Can you check whether AI crawlers like GPTBot, ClaudeBot, and PerplexityBot are allowed to access https://openai.com, and give me their verdicts plus the overall AI access score?

## When to prefer this

Use this endpoint when you need to determine whether AI crawlers and LLM training bots can access a specific website, including per-bot granularity, path-level restrictions, and presence of llms.txt. Prefer this over manually parsing robots.txt when you need structured, scored output with vendor attribution (OpenAI, Anthropic, Perplexity) and a normalized AI access score rather than raw text parsing.

## Known failure modes

- Website is unreachable or returns a non-200 status for robots.txt — error field will be populated
- Domain does not exist — results in a network error or null response
- llms.txt returns 403 or 404 — reflected in llmsFullTxt.status and llmsTxt.status fields
- Malformed URL input — may cause a 400-level error or empty results
- Rate limiting by the target website during robots.txt fetch — may cause incomplete results

## How this service works

Pay-per-call data & tool APIs

## Output

A JSON object containing per-bot verdicts (allowed/partial/blocked) for major AI crawlers (GPTBot, ClaudeBot, OAI-SearchBot, PerplexityBot, etc.), matched robots.txt rules, restricted paths, robots.txt and llms.txt existence and HTTP status, a list of fully blocked bots, sitemaps found, an aiAccessScore (0–100), an aiSearchScore (0–100), a human-readable summary, and run metadata including elapsed time and item count.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "properties": {
  "websites": {
   "type": "array",
   "maxItems": 100
  },
  "maxConcurrency": {
   "type": "integer",
   "maximum": 50,
   "minimum": 1
  }
 }
}
```

## Response schema (JSON Schema)

```json
{
 "type": "json",
 "example": {
  "meta": {
   "run_ms": 1200,
   "settled": true,
   "item_count": 1
  },
  "results": [
   {
    "bots": [
     {
      "bot": "GPTBot",
      "vendor": "OpenAI",
      "purpose": "training",
      "verdict": "partial",
      "matchedRule": "wildcard",
      "restrictedPaths": [
       "/microsoft-for-startups/"
      ]
     },
     {
      "bot": "OAI-SearchBot",
      "vendor": "OpenAI",
      "purpose": "search",
      "verdict": "partial",
      "matchedRule": "wildcard",
      "restrictedPaths": [
       "/microsoft-for-startups/"
      ]
     },
     {
      "bot": "ClaudeBot",
      "vendor": "Anthropic",
      "purpose": "training",
      "verdict": "partial",
      "matchedRule": "wildcard",
      "restrictedPaths": [
       "/microsoft-for-startups/"
      ]
     },
     {
      "bot": "PerplexityBot",
      "vendor": "Perplexity",
      "purpose": "search",
      "verdict": "partial",
      "matchedRule": "wildcard",
      "restrictedPaths": [
       "/microsoft-for-startups/"
      ]
     }
    ],
    "error": null,
    "found": true,
    "llmsTxt": {
     "exists": false,
     "status": 403
    },
    "summary": "All audited AI crawlers are allowed.",
    "website": "https://openai.com",
    "sitemaps": [
     "https://openai.com/sitemap.xml"
    ],
    "checkedAt": "2026-07-26T13:55:53.146Z",
    "robotsTxt": {
     "error": null,
     "exists": true,
     "status": 200
    },
    "blockedBots": [],
    "llmsFullTxt": {
     "exists": false,
     "status": 403
    },
    "aiAccessScore": 100,
    "aiSearchScore": 100
   }
  ]
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/ai-crawler-access-checker-2c5b150c/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from api.timzinin.com](https://www.zero.xyz/host/api.timzinin.com/llms.txt)