# Retriever AI Web Scraper > Retriever AI Web Scraper is a paid API for AI agents from agents.rtrvr.ai, paid per call via MPP or x402, $0.05/call, status healthy (last checked 2026-10-12, last successful call 2026-10-07). Opens one or more URLs in a cloud browser and returns the visible text, accessibility tree, and link map for each page ## Facts - Endpoint: POST https://agents.rtrvr.ai/scrape?utm_source=zero.xyz - Price: $0.05/call - Payment: MPP, x402 - Status: healthy - Last checked: 2026-10-12 - Last successful call: 2026-10-07 - Success rate: 97% of calls made through Zero - Rating: 4.2 / 5 from 2 reviews - Activations on Zero: 66 - Provider: agents.rtrvr.ai - Website: https://agents.rtrvr.ai - Tags: x402 - Canonical page: https://www.zero.xyz/c/agents-rtrvr-ai-retriever-ai-web-scraper-100acdf9 - Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_j5fj7yau2rXyJZ-Wh-lMN Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently. ## How to call it through Zero Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`): ```sh zero fetch --capability agents-rtrvr-ai-retriever-ai-web-scraper-100acdf9 -d '' ``` Example prompt: Can you scrape the text and all links from https://techcrunch.com/latest and https://news.ycombinator.com using a rotating proxy so they don't block us? ## When to prefer this Choose this endpoint when you need to read pages that require JavaScript execution, trigger lazy-loaded content, or sit behind bot-detection and rate limiting that blocks plain HTTP fetches. It is especially useful when you need the full accessibility tree and link map in addition to visible text, or when you want to scrape multiple URLs in a single API call. Prefer it over simple HTTP fetch wrappers when page rendering fidelity matters or when rotating proxies are needed to reliably reach the target. ## Known failure modes - URL unreachable or returns non-200 status — tab has status:error and an error message - Page times out during load — tab marked error with timeout detail - Proxy blocked by target site — may return empty or partial content - Output exceeds inlineOutputMaxBytes — full data not inline, only preview plus responseRef returned - Invalid URL format — request rejected at validation - Webhook delivery failure if configured — no retry by default unless retry mode set ## How this service works Opens each URL in a cloud browser and returns one tab per URL, in input order, with the title, content type, extracted text, the accessibility tree and a link map. Results over the inline size limit return a preview per tab plus a download reference. Each call is one request. ## Output Returns one tab object per input URL in the same order, each containing the page title, HTTP content type, extracted visible text, accessibility tree (as a JSON string), and a map of element IDs to link URLs. If the total response exceeds the inline size cap, a per-tab preview is returned along with a download reference (responseRef) for the full payload. Also includes task duration, usage credits, and proxy billing details. ## Request schema (JSON Schema) ```json { "type": "object", "properties": { "urls": { "type": "array", "items": { "type": "string", "format": "uri" }, "minItems": 1, "description": "Absolute URLs to load. One tab comes back per URL, in the same order." }, "options": { "type": "object", "properties": { "ui": { "type": "object", "properties": { "emitEvents": { "type": "boolean", "default": false, "description": "Write progress events for SSE or polling clients." } } } } }, "response": { "type": "object", "properties": { "inlineOutputMaxBytes": { "type": "integer", "default": 1048576, "description": "Cap on the inline JSON. Larger results return per-tab previews plus metadata.responseRef for the full payload." } } }, "settings": { "type": "object", "properties": { "proxyConfig": { "type": "object", "properties": { "mode": { "enum": [ "none", "default", "custom", "device" ], "type": "string", "description": "Proxy mode for the page loads. default uses Retriever AI's rotating proxies." } } }, "extractionConfig": { "type": "object", "properties": { "pageLoadDelay": { "type": "number", "description": "Extra wait after page load before extraction." }, "maxParallelTabs": { "type": "integer", "description": "How many URLs load at once." }, "onlyTextContent": { "type": "boolean", "description": "Return extracted text only and omit the accessibility tree and link record." }, "disableAutoScroll": { "type": "boolean", "description": "Skip the scroll pass that triggers lazy-loaded content." } } } }, "description": "Per-call overrides merged over the stored profile." }, "webhooks": { "type": "array", "items": { "type": "object", "required": [ "url" ], "properties": { "url": { "type": "string", "format": "uri", "description": "HTTPS endpoint that receives the JSON envelope." }, "auth": { "type": "object", "properties": { "type": { "enum": [ "bearer", "basic" ], "type": "string" }, "token": { "type": "string" }, "password": { "type": "string" }, "username": { "ty … (truncated) ``` ## Response schema (JSON Schema) ```json { "type": "object", "properties": { "tabs": { "type": "array", "items": { "type": "object", "properties": { "url": { "type": "string" }, "tree": { "type": "string", "description": "Accessibility tree as a JSON string. Omitted when onlyTextContent is true." }, "error": { "type": "string" }, "tabId": { "type": "integer" }, "title": { "type": "string" }, "status": { "enum": [ "success", "error" ], "type": "string" }, "content": { "type": "string", "description": "Extracted visible text." }, "contentType": { "type": "string" }, "elementLinkRecord": { "type": "object", "description": "Tree element id to link URL. Present only with tree.", "additionalProperties": { "type": "string" } } } }, "description": "One entry per input URL, in input order." }, "error": { "type": "string" }, "status": { "enum": [ "success", "error" ], "type": "string" }, "success": { "type": "boolean" }, "metadata": { "type": "object", "properties": { "taskRef": { "type": "string" }, "durationMs": { "type": "integer" }, "responseRef": { "type": "object", "description": "Storage reference for the full payload when the inline cap was exceeded." }, "outputTooLarge": { "type": "boolean" }, "inlineOutputMaxBytes": { "type": "integer" } } }, "usageData": { "type": "object", "properties": { "totalUsd": { "type": "number" }, "proxyCredits": { "type": "number" }, "totalCredits": { "type": "number" }, "browserCredits": { "type": "number" }, "proxyPageLoads": { "type": "number" }, "requestDurationMs": { "type": "number" }, "usingBillableProxy": { "type": "boolean" }, "proxyTabsDataFetches": { "type": "number" } }, "description": "Infra usage on Retriever AI's side. Informational; the storefront bills one request per call." }, "trajectoryId": { "type": "string" } } } ``` ## More - Live health (JSON, refreshed every minute): https://www.zero.xyz/c/agents-rtrvr-ai-retriever-ai-web-scraper-100acdf9/health.json - [Zero catalog index](https://www.zero.xyz/llms.txt) - [Other services from agents.rtrvr.ai](https://www.zero.xyz/host/agents.rtrvr.ai/llms.txt)