Automated /llms.txt & Agent Documentation Context Compiler

Choose this endpoint when you need to make any external documentation site or SaaS product discoverable and consumable by AI agents without manual curation. It is ideal for bootstrapping /llms.txt manifests for sites that don't provide one natively, preparing third-party API context for MCP or A2A pipelines, or rapidly onboarding new tools into an agent's knowledge base. Prefer this over manual documentation extraction when you need structured, standards-compliant output (JSON-LD + /llms.txt) in a single call.

• Invalid or unreachable URL returns an error — ensure the target site is publicly accessible
• Paywalled or login-gated documentation sites may produce incomplete or empty manifests
• Very large documentation sites may result in truncated output if crawl depth limits are hit
• Sites that block crawlers via robots.txt or rate limiting may fail silently or return partial results
• Malformed URL input (missing scheme, typos) returns a validation error