# RAG Token Diet — BM25 Budget-Controlled Web Distillation

> RAG Token Diet — BM25 Budget-Controlled Web Distillation is a paid API for AI agents from preflight.agentdiet.dev, paid per call via MPP, $0.001/call, status unknown (last checked 2026-10-28).

Fetches and distills a URL or raw HTML into compact, token-efficient agent-ready content using BM25-ranked budget control with configurable output modes

## Facts

- Endpoint: POST https://preflight.agentdiet.dev/v1/rag-token-diet?utm_source=zero.xyz
- Price: $0.001/call
- Payment: MPP
- Status: unknown
- Last checked: 2026-10-28
- Activations on Zero: 0
- Provider: preflight.agentdiet.dev
- Website: https://preflight.agentdiet.dev
- Canonical page: https://www.zero.xyz/c/preflight-agentdiet-dev-rag-token-diet-bm25-budget-controlled-web-13ce6e72
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_O_tMvRZe1XEA3bjGO7baF

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability preflight-agentdiet-dev-rag-token-diet-bm25-budget-controlled-web-13ce6e72 -d '<json body>'
```

Example prompt: Distill the content at https://stripe.com/docs/payments into compact markdown, using budget mode focused on the query 'payment intents API' with a max of 800 tokens, so I can fit it into my context window without wasting tokens.

## When to prefer this

Choose this endpoint when you need to reduce token consumption before passing web content to an LLM, especially in RAG pipelines where context window size and cost matter. It is ideal when you have a specific query focus (BM25 budget mode) and a hard token ceiling. Prefer it over generic scrapers when you need built-in compression metrics and ROI proof. It also supports raw HTML input, making it useful when you've already fetched content and just need distillation without an additional HTTP fetch.

## Known failure modes

- URL unreachable or returns non-200 — distillation cannot proceed, status error returned
- HTML payload too small or already minimal — compression ratio near zero, roi_proof reflects minimal savings
- max_tokens set below minimum of 20 — validation error
- mode=budget used without providing a query — BM25 ranking cannot focus content
- Target site blocks automated fetching (bot protection) — fetch failure returned
- Malformed or truncated HTML input — partial distillation or parse error

## How this service works

Distill a URL or raw HTML payload with BM25 budget control (Live x402 v2, IETF MPP & MCP endpoint at https://preflight.agentdiet.dev/v1/rag-token-diet)

## Output

Returns a distilled content object (Markdown, structured JSON tables/metadata, skeleton outline, or BM25-ranked budget excerpt), accompanied by ROI proof showing raw vs. distilled token counts, compression ratio percentage, tokens saved, estimated LLM cost saved in USD, and a cache hit/miss status with latency. Billing details include the agent ID and fee charged.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "required": [
  "url"
 ],
 "properties": {
  "url": {
   "type": "string",
   "minLength": 1,
   "description": "Target webpage URL to fetch and distill (or builtin://stripe-docs)"
  },
  "html": {
   "type": "string",
   "description": "Optional raw HTML string to distill directly without fetching"
  },
  "mode": {
   "enum": [
    "markdown",
    "structured",
    "skeleton",
    "budget"
   ],
   "type": "string",
   "default": "markdown",
   "description": "Distillation mode"
  },
  "query": {
   "type": "string",
   "description": "BM25 focus query when mode=budget"
  },
  "max_tokens": {
   "type": "integer",
   "minimum": 20,
   "description": "Strict token budget ceiling when mode=budget"
  }
 }
}
```

## Response schema (JSON Schema)

```json
{
 "type": "object",
 "required": [
  "status",
  "roi_proof",
  "data"
 ],
 "properties": {
  "data": {
   "type": "object",
   "description": "Distilled Markdown, structured JSON tables/metadata, skeleton outline, or BM25 budget excerpt."
  },
  "cache": {
   "type": "object",
   "properties": {
    "status": {
     "enum": [
      "HIT",
      "MISS"
     ],
     "type": "string"
    },
    "cache_key": {
     "type": "string"
    },
    "latency_ms": {
     "type": "number"
    }
   }
  },
  "status": {
   "type": "string",
   "description": "Execution status ('ok')"
  },
  "billing": {
   "type": "object",
   "properties": {
    "agent_id": {
     "type": "string"
    },
    "payment_method": {
     "type": "string"
    },
    "fee_charged_usd": {
     "type": "number"
    }
   }
  },
  "roi_proof": {
   "type": "object",
   "properties": {
    "raw_tokens": {
     "type": "integer"
    },
    "tokens_saved": {
     "type": "integer"
    },
    "roi_multiplier": {
     "type": "number"
    },
    "distilled_tokens": {
     "type": "integer"
    },
    "llm_cost_saved_usd": {
     "type": "number"
    },
    "compression_ratio_pct": {
     "type": "number"
    }
   }
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/preflight-agentdiet-dev-rag-token-diet-bm25-budget-controlled-web-13ce6e72/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from preflight.agentdiet.dev](https://www.zero.xyz/host/preflight.agentdiet.dev/llms.txt)