Web URL Distillation & BM25 Token Compressor

Choose this endpoint when an AI agent needs to ingest webpage content but must stay within LLM token budgets or minimize LLM inference costs. Especially valuable when you need BM25-ranked relevance-focused extraction (mode=budget with a query), not just naive truncation. Prefer over generic scrapers when you need ROI metrics (tokens saved, cost saved) or when you want structured JSON tables or skeleton outlines rather than raw HTML. Also useful for passing raw HTML you already have without an additional fetch.

• URL unreachable or returns non-200 — fetch failure with error status
• Paywalled content cannot be accessed without credentials — may return partial or empty content
• Token budget too small (max_tokens < 20) — validation error
• BM25 query provided without mode=budget — query ignored or error
• Raw HTML too large to process within limits — truncation or error
• Invalid URL format — schema validation error
• Rate limit or payment failure via x402 — 402 Payment Required response