x402.botsmith.dev Web Reader

Use this endpoint when an agent needs to read and process the textual content of any public web page in a clean, structured format without writing custom scraping logic. It is ideal for extracting article text, blog posts, documentation pages, or any human-readable web content into markdown for downstream summarization, analysis, or display. Prefer it over raw HTTP fetch when you want boilerplate removed and metadata (title, byline, language) surfaced automatically.

• URL is not publicly accessible or requires authentication — returns error or empty content
• Page has no extractable main content (e.g. pure JavaScript SPA with no SSR) — markdown may be empty or minimal
• Redirect chain leads to paywalled or blocked content — partial or empty markdown returned
• Invalid or malformed URL provided — schema validation error
• Network timeout fetching the target URL — error response
• Page is in a language or encoding that degrades parsing quality — lower fidelity markdown