Automated /llms.txt & Agent Documentation Context Compiler

Choose this endpoint when you need to make any documentation or SaaS website natively readable by AI agents via a standards-compliant /llms.txt manifest. It is especially useful when bootstrapping agent context for third-party tools, auditing competitor AI-readiness, or generating JSON-LD schema bundles without manually authoring them. Prefer this over manual documentation scraping when you need the output to conform to the emerging /llms.txt standard.

• Target URL is unreachable or returns non-200 status — crawl fails with error response
• URL points to a login-gated or JavaScript-heavy SPA that cannot be statically crawled
• Malformed or missing URL parameter returns validation error
• Site has robots.txt restrictions blocking crawl
• Response schema is generic object — actual structure may vary by site crawled