# API Zavod robots.txt Retrieval

> API Zavod robots.txt Retrieval is a paid API for AI agents from it.finlandfoxvpn.shop, paid per call via x402, $0.001/call, status down (last checked 2026-09-15).

Fetches and returns the robots.txt file from a given website URL

## Facts

- Endpoint: GET https://it.finlandfoxvpn.shop/api/v1/web/robots
- Price: $0.001/call
- Payment: x402
- Status: down
- Last checked: 2026-09-15
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/api-zavod-robots-txt-retrieval-9b019139
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_A1O4xThEmrKP5X8tExaTU

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability api-zavod-robots-txt-retrieval-9b019139
```

Example prompt: Can you grab the robots.txt file from https://example.com so I can see what paths are disallowed and whether the site allows bots to crawl it?

## When to prefer this

Choose this endpoint when you specifically need to retrieve and inspect a website's robots.txt file — for pre-crawl compliance, SEO audits, or bot policy research. Use this over a full site-metadata bundle when you only need crawl directives and want a lightweight, targeted response.

## Known failure modes

- URL not reachable — server down or DNS failure returns non-200 status
- robots.txt not found — returns found=false with HTTP 404
- Malformed URL input causes validation error
- Rate limiting or firewall on target site blocks retrieval
- Target site returns non-standard robots.txt location

## How this service works

robots.txt retrieval / Получение robots.txt

## Output

Returns a JSON object containing the target URL, the retrieved robots.txt content, an HTTP status code, and a boolean flag indicating whether the file was found. Useful for inspecting crawl rules, disallow directives, and user-agent policies.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "required": [
  "input"
 ],
 "properties": {
  "input": {
   "type": "object",
   "required": [
    "type",
    "method"
   ],
   "properties": {
    "type": {
     "type": "string",
     "const": "http"
    },
    "method": {
     "enum": [
      "GET",
      "HEAD",
      "DELETE"
     ],
     "type": "string"
    },
    "queryParams": {
     "type": "object",
     "required": [
      "url"
     ],
     "properties": {
      "url": {
       "type": "string",
       "format": "uri"
      }
     }
    }
   },
   "additionalProperties": false
  },
  "output": {
   "type": "object",
   "required": [
    "type"
   ],
   "properties": {
    "type": {
     "type": "string"
    },
    "example": {
     "type": "object"
    }
   }
  }
 }
}
```

## Response schema (JSON Schema)

```json
{
 "type": "json",
 "example": {
  "url": "https://example.com/robots.txt",
  "найден": true,
  "http_статус": 200
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/api-zavod-robots-txt-retrieval-9b019139/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from it.finlandfoxvpn.shop](https://www.zero.xyz/host/it.finlandfoxvpn.shop/llms.txt)