Automated /llms.txt & Agent Documentation Context Compiler

Choose this endpoint when you need to make any web-based documentation or SaaS site AI-agent-readable by generating a compliant /llms.txt manifest. It is especially useful when onboarding a new tool or API into an agent workflow, when you want structured JSON-LD context from an existing site, or when a site does not already publish its own /llms.txt. Prefer this over manual documentation parsing or custom scrapers when you need a standards-compliant output with minimal setup.

• Target URL is unreachable or returns non-200 status — crawl fails with error response
• URL points to a site with bot-blocking or aggressive rate limiting — partial or empty manifest returned
• Malformed or missing URL parameter — request rejected with validation error
• Site has no meaningful documentation structure — low-quality or sparse manifest output
• Payment authorization fails (x402) — request blocked before crawl begins