# Paywall-Clean Web Distiller with BM25 Budget Control

> Paywall-Clean Web Distiller with BM25 Budget Control is a paid API for AI agents from mesh.agentdiet.dev, paid per call via MPP, $0.001/call, status unknown (last checked 2026-10-23).

Fetches a URL or accepts raw HTML and distills it into clean Markdown, structured JSON, a skeleton outline, or a BM25-ranked token-budget excerpt

## Facts

- Endpoint: POST https://mesh.agentdiet.dev/v1/paywall-clean-extract?utm_source=zero.xyz
- Price: $0.001/call
- Payment: MPP
- Status: unknown
- Last checked: 2026-10-23
- Activations on Zero: 0
- Provider: mesh.agentdiet.dev
- Website: https://mesh.agentdiet.dev
- Canonical page: https://www.zero.xyz/c/mesh-agentdiet-dev-paywall-clean-web-distiller-with-bm25-budget-control-1b32d900
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_JNAzet4MOA6fDHKg-CLTE

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability mesh-agentdiet-dev-paywall-clean-web-distiller-with-bm25-budget-control-1b32d900 -d '<json body>'
```

Example prompt: Can you fetch https://stripe.com/docs/payments and distill it into clean markdown, keeping the output under 2000 tokens using BM25 budget mode focused on the query 'payment intents'?

## When to prefer this

Choose this endpoint when you need to extract clean, token-efficient content from a URL or HTML blob for downstream LLM processing, especially when context window size or LLM cost is a concern. It is particularly valuable when you want BM25-ranked relevance filtering to surface only the most query-relevant passages within a strict token ceiling. Prefer it over generic web scrapers when you need structured JSON tables, skeleton outlines, or Markdown alongside token savings metrics. It supports both live URL fetching and offline raw HTML distillation, making it versatile for both real-time pipelines and pre-fetched content workflows.

## Known failure modes

- URL is unreachable or returns a non-200 status — endpoint may return an error status
- HTML payload is empty or malformed — distillation may fail or return minimal output
- max_tokens is set too low relative to content size — output may be aggressively truncated
- BM25 query is missing when mode=budget — may fall back to default behavior or return an error
- Payment not provided or insufficient — returns HTTP 402 requiring x402 USDC payment
- Rate limiting or upstream paywall blocks fetch — content may be unavailable

## How this service works

Distill a URL or raw HTML payload with BM25 budget control (Live x402 v2, IETF MPP & MCP endpoint at https://mesh.agentdiet.dev/v1/paywall-clean-extract)

## Output

Returns a JSON object containing: distilled content in the chosen format (Markdown text, structured JSON tables/metadata, skeleton outline, or BM25-ranked budget excerpt) under the 'data' key; an 'roi_proof' object showing raw vs distilled token counts, compression ratio, tokens saved, and estimated LLM cost saved in USD; optional cache status (HIT/MISS) and billing details including fee charged.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "required": [
  "url"
 ],
 "properties": {
  "url": {
   "type": "string",
   "minLength": 1,
   "description": "Target webpage URL to fetch and distill (or builtin://stripe-docs)"
  },
  "html": {
   "type": "string",
   "description": "Optional raw HTML string to distill directly without fetching"
  },
  "mode": {
   "enum": [
    "markdown",
    "structured",
    "skeleton",
    "budget"
   ],
   "type": "string",
   "default": "markdown",
   "description": "Distillation mode"
  },
  "query": {
   "type": "string",
   "description": "BM25 focus query when mode=budget"
  },
  "max_tokens": {
   "type": "integer",
   "minimum": 20,
   "description": "Strict token budget ceiling when mode=budget"
  }
 }
}
```

## Response schema (JSON Schema)

```json
{
 "type": "object",
 "required": [
  "status",
  "roi_proof",
  "data"
 ],
 "properties": {
  "data": {
   "type": "object",
   "description": "Distilled Markdown, structured JSON tables/metadata, skeleton outline, or BM25 budget excerpt."
  },
  "cache": {
   "type": "object",
   "properties": {
    "status": {
     "enum": [
      "HIT",
      "MISS"
     ],
     "type": "string"
    },
    "cache_key": {
     "type": "string"
    },
    "latency_ms": {
     "type": "number"
    }
   }
  },
  "status": {
   "type": "string",
   "description": "Execution status ('ok')"
  },
  "billing": {
   "type": "object",
   "properties": {
    "agent_id": {
     "type": "string"
    },
    "payment_method": {
     "type": "string"
    },
    "fee_charged_usd": {
     "type": "number"
    }
   }
  },
  "roi_proof": {
   "type": "object",
   "properties": {
    "raw_tokens": {
     "type": "integer"
    },
    "tokens_saved": {
     "type": "integer"
    },
    "roi_multiplier": {
     "type": "number"
    },
    "distilled_tokens": {
     "type": "integer"
    },
    "llm_cost_saved_usd": {
     "type": "number"
    },
    "compression_ratio_pct": {
     "type": "number"
    }
   }
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/mesh-agentdiet-dev-paywall-clean-web-distiller-with-bm25-budget-control-1b32d900/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from mesh.agentdiet.dev](https://www.zero.xyz/host/mesh.agentdiet.dev/llms.txt)