# UOS Genesis Machine – robots_check

> UOS Genesis Machine – robots_check is a paid API for AI agents from genesis.palsus2023.workers.dev, paid per call via x402, $0.002/call, status unknown (last checked 2026-10-02).

Fetches and returns the robots.txt file and crawler directives for any given public website URL.

## Facts

- Endpoint: POST https://genesis.palsus2023.workers.dev/v1/robots_check?utm_source=zero.xyz
- Price: $0.002/call
- Payment: x402
- Status: unknown
- Last checked: 2026-10-02
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/uos-genesis-machine-robots-check-e0780d49
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_II_sPmWCpG1wSWAqf8n5L

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability uos-genesis-machine-robots-check-e0780d49 -d '<json body>'
```

Example prompt: Can you grab the robots.txt file for https://example.com and tell me what crawler rules it sets?

## When to prefer this

Use this endpoint when an agent needs to programmatically verify crawl permissions before scraping or indexing a website, especially in automated pipeline contexts where robots.txt compliance must be checked at scale. Prefer this over manual inspection or full-page scraping tools when you only need crawler directives rather than page content.

## Known failure modes

- 404 status if the website has no robots.txt file
- Network timeout if the target domain is unreachable or slow
- Non-200 status if the domain blocks the fetching agent
- Malformed or empty robots.txt returned for unusual server configurations
- Invalid URL input causes schema validation error

## How this service works

Autonomous paid machine utilities using x402 and Base USDC.

## Output

Returns a JSON object containing the full robots.txt content as a string, the URL of the robots.txt file that was fetched, and the HTTP status code of the response (e.g. 200 if found, 404 if missing).

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "required": [
  "input"
 ],
 "properties": {
  "input": {
   "type": "object",
   "required": [
    "type",
    "method",
    "bodyType",
    "body"
   ],
   "properties": {
    "body": {
     "type": "object",
     "required": [
      "url"
     ],
     "properties": {
      "url": {
       "type": "string",
       "format": "uri",
       "description": "Public website URL whose robots.txt crawler directives should be fetched and inspected."
      }
     }
    },
    "type": {
     "type": "string",
     "const": "http"
    },
    "method": {
     "enum": [
      "POST"
     ],
     "type": "string"
    },
    "bodyType": {
     "enum": [
      "json"
     ],
     "type": "string"
    }
   },
   "additionalProperties": false
  },
  "output": {
   "type": "object"
  }
 },
 "additionalProperties": true
}
```

## Response schema (JSON Schema)

```json
{
 "type": "json",
 "example": {
  "url": "https://example.com/robots.txt",
  "robots": "User-agent: *",
  "status": 200
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/uos-genesis-machine-robots-check-e0780d49/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from genesis.palsus2023.workers.dev](https://www.zero.xyz/host/genesis.palsus2023.workers.dev/llms.txt)