# Web Distillation with BM25 Budget Control

> Web Distillation with BM25 Budget Control is a paid API for AI agents from whale-momentum.agentdiet.dev, paid per call via MPP, $0.001/call, status unknown (last checked 2026-10-23).

Fetches and distills a URL or raw HTML into compressed Markdown, structured JSON, skeleton, or BM25-focused budget excerpts with token savings proof

## Facts

- Endpoint: POST https://whale-momentum.agentdiet.dev/v1/rag-chunk?utm_source=zero.xyz
- Price: $0.001/call
- Payment: MPP
- Status: unknown
- Last checked: 2026-10-23
- Activations on Zero: 0
- Provider: whale-momentum.agentdiet.dev
- Website: https://whale-momentum.agentdiet.dev
- Canonical page: https://www.zero.xyz/c/whale-momentum-agentdiet-dev-web-distillation-with-bm25-budget-control-19db5b00
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_qciETYSJhvfpoFsYHlW_a

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability whale-momentum-agentdiet-dev-web-distillation-with-bm25-budget-control-19db5b00 -d '<json body>'
```

Example prompt: Distill the page at https://stripe.com/docs/payments into clean Markdown using budget mode, focused on 'payment intents', and keep it under 500 tokens — I want to see how many tokens you saved.

## When to prefer this

Choose this endpoint when you need to feed webpage content into an LLM pipeline cost-efficiently, especially when token budget is a concern. It is ideal for RAG indexing, agent research tasks, or any workflow that needs clean structured or Markdown text from URLs or raw HTML. The BM25 budget mode is uniquely useful when you want only the most query-relevant content within a strict token ceiling, which no generic scraper provides. Prefer over raw scrapers when you need compression metrics and ROI proof.

## Known failure modes

- URL is unreachable or returns non-200 — fetch failure error
- HTML or URL content is empty or too short to distill
- max_tokens set below 20 triggers validation error
- BM25 query missing when mode=budget — may return unranked or full output
- Paywalled or JavaScript-heavy pages may return incomplete content
- Payment failure via x402 if USDC balance insufficient

## How this service works

Distill a URL or raw HTML payload with BM25 budget control (Live x402 v2, IETF MPP & MCP endpoint at https://whale-momentum.agentdiet.dev/v1/rag-chunk)

## Output

Returns a JSON object with a 'data' field containing distilled content (Markdown text, structured JSON tables/metadata, skeleton outline, or BM25-ranked budget excerpt depending on mode), plus an 'roi_proof' object showing raw_tokens, distilled_tokens, tokens_saved, compression_ratio_pct, roi_multiplier, and llm_cost_saved_usd. Also includes optional cache hit/miss status and billing details.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "required": [
  "url"
 ],
 "properties": {
  "url": {
   "type": "string",
   "minLength": 1,
   "description": "Target webpage URL to fetch and distill (or builtin://stripe-docs)"
  },
  "html": {
   "type": "string",
   "description": "Optional raw HTML string to distill directly without fetching"
  },
  "mode": {
   "enum": [
    "markdown",
    "structured",
    "skeleton",
    "budget"
   ],
   "type": "string",
   "default": "markdown",
   "description": "Distillation mode"
  },
  "query": {
   "type": "string",
   "description": "BM25 focus query when mode=budget"
  },
  "max_tokens": {
   "type": "integer",
   "minimum": 20,
   "description": "Strict token budget ceiling when mode=budget"
  }
 }
}
```

## Response schema (JSON Schema)

```json
{
 "type": "object",
 "required": [
  "status",
  "roi_proof",
  "data"
 ],
 "properties": {
  "data": {
   "type": "object",
   "description": "Distilled Markdown, structured JSON tables/metadata, skeleton outline, or BM25 budget excerpt."
  },
  "cache": {
   "type": "object",
   "properties": {
    "status": {
     "enum": [
      "HIT",
      "MISS"
     ],
     "type": "string"
    },
    "cache_key": {
     "type": "string"
    },
    "latency_ms": {
     "type": "number"
    }
   }
  },
  "status": {
   "type": "string",
   "description": "Execution status ('ok')"
  },
  "billing": {
   "type": "object",
   "properties": {
    "agent_id": {
     "type": "string"
    },
    "payment_method": {
     "type": "string"
    },
    "fee_charged_usd": {
     "type": "number"
    }
   }
  },
  "roi_proof": {
   "type": "object",
   "properties": {
    "raw_tokens": {
     "type": "integer"
    },
    "tokens_saved": {
     "type": "integer"
    },
    "roi_multiplier": {
     "type": "number"
    },
    "distilled_tokens": {
     "type": "integer"
    },
    "llm_cost_saved_usd": {
     "type": "number"
    },
    "compression_ratio_pct": {
     "type": "number"
    }
   }
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/whale-momentum-agentdiet-dev-web-distillation-with-bm25-budget-control-19db5b00/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from whale-momentum.agentdiet.dev](https://www.zero.xyz/host/whale-momentum.agentdiet.dev/llms.txt)