# Automated /llms.txt & Agent Documentation Context Compiler

> Automated /llms.txt & Agent Documentation Context Compiler is a paid API for AI agents from arxiv-digest.agentdiet.dev, paid per call via MPP, $0.015/call, status unknown (last checked 2026-10-21).

Crawls any documentation or SaaS website and generates a standards-compliant /llms.txt manifest plus a JSON-LD schema bundle for AI agent consumption

## Facts

- Endpoint: GET https://arxiv-digest.agentdiet.dev/v1/llms-txt-compiler?utm_source=zero.xyz
- Price: $0.015/call
- Payment: MPP
- Status: unknown
- Last checked: 2026-10-21
- Activations on Zero: 0
- Provider: arxiv-digest.agentdiet.dev
- Website: https://arxiv-digest.agentdiet.dev
- Canonical page: https://www.zero.xyz/c/arxiv-digest-agentdiet-dev-automated-llms-txt-agent-documentation-9d8122d3
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_fIKgMhTvC10QrWD3fJzhp

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability arxiv-digest-agentdiet-dev-automated-llms-txt-agent-documentation-9d8122d3
```

Example prompt: Crawl https://stripe.com/docs and compile a standards-compliant /llms.txt manifest and JSON-LD schema bundle so AI agents can discover and understand that documentation.

## When to prefer this

Choose this endpoint when you need to programmatically generate an /llms.txt manifest or JSON-LD agent context bundle from an existing documentation or SaaS website, especially when targeting standards compliance (IETF MPP, MCP) for AI agent discovery. Prefer this over manual authoring or generic scrapers when the output must be structured for agent consumption rather than raw HTML extraction.

## Known failure modes

- URL is unreachable or returns non-200 status — crawl fails with error
- Target site blocks crawlers via robots.txt or Cloudflare — partial or empty manifest returned
- URL is malformed or missing — validation error on required 'url' parameter
- Site has minimal textual content (heavy JS SPA) — incomplete manifest due to render limitations
- Payment not settled via x402 — HTTP 402 response blocking the call

## How this service works

Crawls any documentation or SaaS site and compiles a standards-compliant /llms.txt manifest and JSON-LD schema bundle. (Live x402 v2, IETF MPP & MCP endpoint at https://arxiv-digest.agentdiet.dev/v1/llms-txt-compiler)

## Output

Returns a standards-compliant /llms.txt manifest and JSON-LD schema bundle derived from crawling the target URL, suitable for direct use by AI agents, MCP servers, or agent discovery registries to understand the documentation or product at that URL.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "required": [
  "url"
 ],
 "properties": {
  "url": {
   "type": "string",
   "minLength": 1,
   "description": "Website URL to crawl and generate a ready-to-consume /llms.txt manifest for"
  }
 }
}
```

## Response schema (JSON Schema)

```json
{
 "type": "object"
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/arxiv-digest-agentdiet-dev-automated-llms-txt-agent-documentation-9d8122d3/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from arxiv-digest.agentdiet.dev](https://www.zero.xyz/host/arxiv-digest.agentdiet.dev/llms.txt)