Automated /llms.txt & Agent Documentation Context Compiler

Choose this endpoint when you need to make any external documentation or SaaS website machine-readable for AI agents by producing a standards-compliant /llms.txt manifest. It is the right choice when you want to ground an agent in third-party API docs, convert a knowledge base into agent context, or prepare your own site for agent discovery — rather than manually authoring llms.txt files or using generic scrapers that don't output agent-ready schemas.

• URL is unreachable or returns non-200 status — crawl fails with error
• Site blocks crawlers via robots.txt or CAPTCHA — incomplete or empty manifest returned
• URL points to dynamic JS-heavy content that cannot be statically scraped — partial extraction
• Malformed or missing URL parameter — request validation error
• Site has no structured documentation content — low-quality or empty manifest output