# Conway Replicatio Robots.txt Fetcher

> Conway Replicatio Robots.txt Fetcher is a paid API for AI agents from conway-replicatio-cloudflare.nikitamakiel1.workers.dev, paid per call via x402, $0.01/call, status unknown (last checked 2026-10-11).

Fetches and returns the robots.txt content from a given URL via a pay-per-call HTTP utility service

## Facts

- Endpoint: POST https://conway-replicatio-cloudflare.nikitamakiel1.workers.dev/api/x402/robots?utm_source=zero.xyz
- Price: $0.01/call
- Payment: x402
- Status: unknown
- Last checked: 2026-10-11
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/conway-replicatio-robots-txt-fetcher-c3435110
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_5PqqwQN6QtsJBXhjrUEqE

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability conway-replicatio-robots-txt-fetcher-c3435110 -d '<json body>'
```

Example prompt: Can you fetch the robots.txt file for https://example.com and tell me which paths are disallowed for bots?

## When to prefer this

Use this endpoint when you need a quick, pay-per-call fetch of a website's robots.txt without managing your own HTTP infrastructure. It's ideal for AI agents that need to check crawl permissions before web scraping, or for automated pipelines that audit SEO crawl rules on demand. It settles payments in USDC on Base via x402, making it suitable for autonomous agent workflows that operate without manual billing setup.

## Known failure modes

- Invalid or unreachable URL returns an error response with ok: false
- Malformed URI in the body causes schema validation failure
- Target site blocks the fetch request (403/404) resulting in an empty or error data payload
- Network timeout if the target server is slow to respond
- Missing required 'url' field in the body causes a 400-level error

## How this service works

Autonomous pay-per-call machine utilities for AI agents, automation and developers. Payments settle in USDC on Base through x402.

## Output

Returns a JSON object with an 'ok' boolean indicating success, a 'data' object containing the robots.txt content or parsed response from the target URL, and a 'service' field confirming the robots service was used.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "required": [
  "input"
 ],
 "properties": {
  "input": {
   "type": "object",
   "required": [
    "type",
    "method",
    "bodyType",
    "body"
   ],
   "properties": {
    "body": {
     "type": "object",
     "required": [
      "url"
     ],
     "properties": {
      "url": {
       "type": "string",
       "format": "uri"
      }
     },
     "additionalProperties": false
    },
    "type": {
     "type": "string",
     "const": "http"
    },
    "method": {
     "enum": [
      "POST"
     ],
     "type": "string"
    },
    "bodyType": {
     "enum": [
      "json",
      "form-data",
      "text"
     ],
     "type": "string"
    }
   },
   "additionalProperties": false
  },
  "output": {
   "type": "object",
   "required": [
    "type"
   ],
   "properties": {
    "type": {
     "type": "string"
    },
    "example": {
     "type": "object"
    }
   }
  }
 }
}
```

## Response schema (JSON Schema)

```json
{
 "type": "json",
 "example": {
  "ok": true,
  "data": {},
  "service": "robots"
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/conway-replicatio-robots-txt-fetcher-c3435110/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from conway-replicatio-cloudflare.nikitamakiel1.workers.dev](https://www.zero.xyz/host/conway-replicatio-cloudflare.nikitamakiel1.workers.dev/llms.txt)