# Paywall-Clean URL & HTML Distiller with BM25 Budget Control

> Paywall-Clean URL & HTML Distiller with BM25 Budget Control is a paid API for AI agents from token-intel.agentdiet.dev, paid per call via MPP, $0.001/call, status unknown (last checked 2026-10-23).

Fetches and distills a URL or raw HTML into compressed Markdown, structured JSON, a skeleton outline, or a BM25-ranked budget excerpt, returning token-savings ROI proof.

## Facts

- Endpoint: POST https://token-intel.agentdiet.dev/v1/paywall-clean-extract?utm_source=zero.xyz
- Price: $0.001/call
- Payment: MPP
- Status: unknown
- Last checked: 2026-10-23
- Activations on Zero: 0
- Provider: token-intel.agentdiet.dev
- Website: https://token-intel.agentdiet.dev
- Canonical page: https://www.zero.xyz/c/token-intel-agentdiet-dev-paywall-clean-url-html-distiller-with-bm25-29788716
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_YYNjk9j65DfPzszFNBQHa

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability token-intel-agentdiet-dev-paywall-clean-url-html-distiller-with-bm25-29788716 -d '<json body>'
```

Example prompt: Fetch https://stripe.com/docs/payments/checkout and distill it into clean markdown using budget mode, focusing on 'payment intent creation', keeping it under 800 tokens — and show me how many tokens you saved.

## When to prefer this

Choose this endpoint when you need to feed webpage content to an LLM but want to minimize token consumption and cost — especially useful for long documentation pages, paywalled articles, or any HTML where only the topically relevant sections matter. The BM25 budget mode is uniquely suited for RAG pipelines where you need only the highest-relevance passages within a strict token ceiling. Prefer this over generic scraping endpoints when ROI proof (tokens saved, cost saved) is important for auditing or optimization workflows.

## Known failure modes

- URL is unreachable or returns non-200 → fetch error with status detail
- Paywall blocks content fetch → partial or empty distillation
- max_tokens set too low for meaningful content → truncated output or error
- Invalid mode enum value → 400 validation error
- HTML input is malformed or empty → empty or degraded distillation
- BM25 query not provided when mode=budget → fallback or error
- Rate limit or payment failure via x402 → 402 Payment Required

## How this service works

Distill a URL or raw HTML payload with BM25 budget control (Live x402 v2, IETF MPP & MCP endpoint at https://token-intel.agentdiet.dev/v1/paywall-clean-extract)

## Output

Returns a JSON object with a 'data' field containing the distilled content (Markdown string, structured JSON tables/metadata, skeleton outline, or BM25-ranked excerpt depending on mode), a 'roi_proof' object showing raw_tokens, distilled_tokens, tokens_saved, compression_ratio_pct, roi_multiplier, and llm_cost_saved_usd, plus optional cache hit/miss status and billing details.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "required": [
  "url"
 ],
 "properties": {
  "url": {
   "type": "string",
   "minLength": 1,
   "description": "Target webpage URL to fetch and distill (or builtin://stripe-docs)"
  },
  "html": {
   "type": "string",
   "description": "Optional raw HTML string to distill directly without fetching"
  },
  "mode": {
   "enum": [
    "markdown",
    "structured",
    "skeleton",
    "budget"
   ],
   "type": "string",
   "default": "markdown",
   "description": "Distillation mode"
  },
  "query": {
   "type": "string",
   "description": "BM25 focus query when mode=budget"
  },
  "max_tokens": {
   "type": "integer",
   "minimum": 20,
   "description": "Strict token budget ceiling when mode=budget"
  }
 }
}
```

## Response schema (JSON Schema)

```json
{
 "type": "object",
 "required": [
  "status",
  "roi_proof",
  "data"
 ],
 "properties": {
  "data": {
   "type": "object",
   "description": "Distilled Markdown, structured JSON tables/metadata, skeleton outline, or BM25 budget excerpt."
  },
  "cache": {
   "type": "object",
   "properties": {
    "status": {
     "enum": [
      "HIT",
      "MISS"
     ],
     "type": "string"
    },
    "cache_key": {
     "type": "string"
    },
    "latency_ms": {
     "type": "number"
    }
   }
  },
  "status": {
   "type": "string",
   "description": "Execution status ('ok')"
  },
  "billing": {
   "type": "object",
   "properties": {
    "agent_id": {
     "type": "string"
    },
    "payment_method": {
     "type": "string"
    },
    "fee_charged_usd": {
     "type": "number"
    }
   }
  },
  "roi_proof": {
   "type": "object",
   "properties": {
    "raw_tokens": {
     "type": "integer"
    },
    "tokens_saved": {
     "type": "integer"
    },
    "roi_multiplier": {
     "type": "number"
    },
    "distilled_tokens": {
     "type": "integer"
    },
    "llm_cost_saved_usd": {
     "type": "number"
    },
    "compression_ratio_pct": {
     "type": "number"
    }
   }
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/token-intel-agentdiet-dev-paywall-clean-url-html-distiller-with-bm25-29788716/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from token-intel.agentdiet.dev](https://www.zero.xyz/host/token-intel.agentdiet.dev/llms.txt)