Web Content Distiller with BM25 Budget Control

Choose this endpoint when you need to ingest web content into an LLM pipeline and want to minimize token consumption. It is especially valuable when dealing with bloated HTML pages, documentation sites, or when you need query-focused excerpts rather than full-page dumps. The BM25 budget mode is uniquely suited for retrieving only the most relevant passage from a long page within a strict token ceiling, which no generic scraper provides. Prefer it over raw HTTP fetch + manual parsing when you want compression ROI metrics alongside the content.

• URL is unreachable or returns a non-200 HTTP status — service may return an error status
• Paywalled content blocks fetching — only publicly accessible HTML is returned
• mode=budget without a query field may produce unexpected results
• max_tokens set too low (below 20) — rejected by schema validation
• Raw HTML input is malformed or empty — distillation may return minimal or error output
• Rate limiting or payment failure via x402 — request rejected before processing