Web URL Token Optimizer & Distiller

Choose this endpoint when your agent needs to read web pages or process HTML but must stay within a token budget or minimize LLM costs. It is especially valuable when processing documentation pages, news articles, or data-heavy pages where raw content would be too large. Prefer over generic scrapers when you need BM25-filtered excerpts relevant to a specific query, or when you want structured/skeleton extraction modes rather than raw text. Ideal for agentic pipelines where LLM input cost is a concern and you want measurable ROI proof of compression.

• Invalid or unreachable URL returns fetch error
• max_tokens below minimum of 20 rejected with validation error
• mode=budget without a query parameter may return unfiltered content
• Payment failure via x402 blocks the request
• Rate limits may apply per agent_id
• Very large HTML pages may exceed processing limits
• builtin:// scheme only valid for documented built-in targets like stripe-docs