# AgentReady URL Preflight Check

> AgentReady URL Preflight Check is a paid API for AI agents from agentready-audit.seshiccse023.chatgpt.site, paid per call via x402, $0.005/call, status unknown (last checked 2026-09-14).

Checks whether a given URL is allowed to be fetched by a specific crawler agent by parsing robots.txt rules, then verifies the page's reachability and basic HTTP metadata.

## Facts

- Endpoint: POST https://agentready-audit.seshiccse023.chatgpt.site/api/preflight/url-check
- Price: $0.005/call
- Payment: x402
- Status: unknown
- Last checked: 2026-09-14
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/agentready-url-preflight-check-24e660a0
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_LzGxT1p-PfNTrXLuCsEvl

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability agentready-url-preflight-check-24e660a0 -d '<json body>'
```

Example prompt: Before my pipeline fetches https://example.com/articles/ai-news, can you preflight that URL for the GPTBot crawler — check if robots.txt allows it and whether the page is actually reachable?

## When to prefer this

Use this endpoint when an AI agent or crawler pipeline needs to check robots.txt compliance and page reachability for a specific URL before fetching it, especially when operating under ethical crawling constraints or building agentic pipelines that must respect robots exclusion standards. Prefer this over a raw HTTP fetch when you need a structured, machine-readable verdict on whether fetching is permitted and feasible, rather than just trying the fetch and handling errors reactively.

## Known failure modes

- URL is malformed or not a valid HTTP/HTTPS URI — returns validation error
- Target server is unreachable or times out — page object may show reachable:false
- robots.txt itself returns a non-200 status — robots.state set to 'unavailable' and access may default to allowed
- URL redirects to a blocked or error page — finalUrl differs from input URL
- Rate limits or payment issues if USDC payment is not properly handled via x402

## How this service works

Free website triage plus x402 audits, Base USDC receipt verification, robots-aware URL preflight, agent-readable extraction, metadata, and copy-ready AI-discovery fixes.

## Output

Returns a JSON object containing: the checked URL, the timestamp of the check, the user agent used, whether robots.txt permits access (allowedByRobots boolean), whether the page is fetchable (fetchable boolean), the specific matched robots rule (or null), a robots object with the robots.txt URL and its fetch status, a page object with HTTP status, final URL after redirects, reachability, content type, and content length, plus a notes array with human-readable advisory messages.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "properties": {
  "url": {
   "type": "string",
   "format": "uri",
   "description": "Public HTTP(S) URL to check before fetching"
  },
  "userAgent": {
   "type": "string",
   "maxLength": 120,
   "description": "Optional crawler product token; defaults to *"
  }
 }
}
```

## Response schema (JSON Schema)

```json
{
 "type": "json",
 "example": {
  "url": "https://example.com/private",
  "page": {
   "status": 200,
   "finalUrl": "https://example.com/private",
   "reachable": true,
   "contentType": "text/html",
   "contentLength": 1256
  },
  "notes": [
   "robots.txt permits this user agent and the page is reachable."
  ],
  "robots": {
   "url": "https://example.com/robots.txt",
   "state": "unavailable",
   "status": 404,
   "sitemaps": []
  },
  "checkedAt": "2026-08-09T00:00:00.000Z",
  "fetchable": true,
  "userAgent": "GPTBot",
  "matchedRule": null,
  "allowedByRobots": true
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/agentready-url-preflight-check-24e660a0/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from agentready-audit.seshiccse023.chatgpt.site](https://www.zero.xyz/host/agentready-audit.seshiccse023.chatgpt.site/llms.txt)