# AgentDiet Article Extractor – BM25 Budget-Controlled Web Distillation

> AgentDiet Article Extractor – BM25 Budget-Controlled Web Distillation is a paid API for AI agents from agentdiet.dev, paid per call via MPP, $0.001/call, status unknown (last checked 2026-10-23).

Fetches a URL or accepts raw HTML and distills it into compact Markdown, structured JSON, a skeleton outline, or a BM25-ranked token-budget excerpt — returning compression ROI metrics alongside the content.

## Facts

- Endpoint: POST https://agentdiet.dev/v1/article-extractor?utm_source=zero.xyz
- Price: $0.001/call
- Payment: MPP
- Status: unknown
- Last checked: 2026-10-23
- Activations on Zero: 0
- Provider: agentdiet.dev
- Website: https://agentdiet.dev
- Canonical page: https://www.zero.xyz/c/agentdiet-dev-agentdiet-article-extractor-bm25-budget-controlled-web-f5d3b1c2
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_4jIBuDWF4UHN6dxB3ophr

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability agentdiet-dev-agentdiet-article-extractor-bm25-budget-controlled-web-f5d3b1c2 -d '<json body>'
```

Example prompt: Distill the article at https://example.com/long-article into clean Markdown using budget mode, focused on 'machine learning infrastructure', and keep it under 300 tokens — I want to see how many tokens were saved too.

## When to prefer this

Choose this endpoint when you need to extract and compress web article content before feeding it to an LLM — especially when token cost is a concern. The BM25 budget mode is uniquely suited for RAG pipelines where you need only the topically relevant portion of a long page within a strict token ceiling. Prefer this over generic scraping services when you also want compression ROI metrics (tokens saved, cost saved in USD) returned alongside the content. Use it instead of full-page scrapers when your downstream LLM context window is limited.

## Known failure modes

- URL unreachable or returns non-200 — distillation fails with an error status
- Raw HTML is empty or malformed — may produce empty or low-quality output
- max_tokens set too low (below 20) — rejected by schema validation
- BM25 query omitted when mode=budget — may fall back to default behavior or return error
- Payment not authorized via x402 — request blocked with 402 Payment Required
- Idempotency-Key reuse across different payloads — may return cached result from prior call

## How this service works

Distill a URL or raw HTML payload with BM25 budget control (Live x402 v2, IETF MPP & MCP endpoint at https://agentdiet.dev/v1/article-extractor)

## Output

Returns a JSON object with: a 'data' field containing the distilled content (Markdown text, structured JSON, skeleton outline, or BM25-ranked excerpt depending on mode); an 'roi_proof' object showing raw_tokens, distilled_tokens, tokens_saved, compression_ratio_pct, roi_multiplier, and llm_cost_saved_usd; optional cache metadata (HIT/MISS, latency_ms); optional billing info; and a status string ('ok').

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "required": [
  "url"
 ],
 "properties": {
  "url": {
   "type": "string",
   "minLength": 1,
   "description": "Target webpage URL to fetch and distill (or builtin://stripe-docs)"
  },
  "html": {
   "type": "string",
   "description": "Optional raw HTML string to distill directly without fetching"
  },
  "mode": {
   "enum": [
    "markdown",
    "structured",
    "skeleton",
    "budget"
   ],
   "type": "string",
   "default": "markdown",
   "description": "Distillation mode"
  },
  "query": {
   "type": "string",
   "description": "BM25 focus query when mode=budget"
  },
  "max_tokens": {
   "type": "integer",
   "minimum": 20,
   "description": "Strict token budget ceiling when mode=budget"
  }
 }
}
```

## Response schema (JSON Schema)

```json
{
 "type": "object",
 "required": [
  "status",
  "roi_proof",
  "data"
 ],
 "properties": {
  "data": {
   "type": "object",
   "description": "Distilled Markdown, structured JSON tables/metadata, skeleton outline, or BM25 budget excerpt."
  },
  "cache": {
   "type": "object",
   "properties": {
    "status": {
     "enum": [
      "HIT",
      "MISS"
     ],
     "type": "string"
    },
    "cache_key": {
     "type": "string"
    },
    "latency_ms": {
     "type": "number"
    }
   }
  },
  "status": {
   "type": "string",
   "description": "Execution status ('ok')"
  },
  "billing": {
   "type": "object",
   "properties": {
    "agent_id": {
     "type": "string"
    },
    "payment_method": {
     "type": "string"
    },
    "fee_charged_usd": {
     "type": "number"
    }
   }
  },
  "roi_proof": {
   "type": "object",
   "properties": {
    "raw_tokens": {
     "type": "integer"
    },
    "tokens_saved": {
     "type": "integer"
    },
    "roi_multiplier": {
     "type": "number"
    },
    "distilled_tokens": {
     "type": "integer"
    },
    "llm_cost_saved_usd": {
     "type": "number"
    },
    "compression_ratio_pct": {
     "type": "number"
    }
   }
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/agentdiet-dev-agentdiet-article-extractor-bm25-budget-controlled-web-f5d3b1c2/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from agentdiet.dev](https://www.zero.xyz/host/agentdiet.dev/llms.txt)