# Bismuth Scraping - BFS Site Crawl

> Bismuth Scraping - BFS Site Crawl is a paid API for AI agents from x402-scraping-api-production.up.railway.app, paid per call via x402, $0.1/call, status unknown (last checked 2026-09-15).

Crawls a website via breadth-first search using Playwright, returning structured markdown, links, and tables for each visited page

## Facts

- Endpoint: POST https://x402-scraping-api-production.up.railway.app/crawl
- Price: $0.1/call
- Payment: x402
- Status: unknown
- Last checked: 2026-09-15
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/bismuth-scraping-bfs-site-crawl-8e48f0e5
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_XUSoBJlU9FIzBERxnhone

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability bismuth-scraping-bfs-site-crawl-8e48f0e5 -d '<json body>'
```

Example prompt: Can you crawl the entire docs.example.com site starting from https://docs.example.com, going up to 3 levels deep and visiting at most 50 pages, and give me the content as structured markdown?

## When to prefer this

Choose this endpoint when you need to extract content from multiple pages of a website in a single call, not just a single page. It is ideal for building knowledge bases, indexing documentation, archiving blogs, or researching competitor sites where you need breadth of coverage. Prefer this over single-page scrapers when site structure and cross-page linking matter, or when you want to discover and collect pages automatically via BFS rather than enumerating URLs manually.

## Known failure modes

- Invalid or unreachable seed URL returns an error
- SSRF-protected URLs (e.g. localhost, private IP ranges) are blocked and return a rejection error
- max_depth or max_pages set too high may result in very long response times or timeouts
- Sites with heavy JavaScript anti-bot measures may return incomplete content
- Malformed URL (exceeding 2048 chars or missing http/https scheme) returns a validation error

## How this service works

Playwright-powered web scraping with structured markdown, links, tables, and BFS site crawl. SSRF-protected. Part of the Bismuth utility API suite for AI agents.

## Output

A structured response containing the content of each crawled page rendered as markdown, including extracted links, tables, and page metadata. The crawl follows a breadth-first search strategy from the seed URL, respecting the specified depth and page limits, with SSRF protections applied to prevent abuse.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "required": [
  "url"
 ],
 "properties": {
  "url": {
   "type": "string",
   "description": "Seed URL to begin crawling (http/https, max 2048 chars)"
  },
  "max_depth": {
   "type": "integer"
  },
  "max_pages": {
   "type": "integer"
  },
  "exclude_paths": {
   "type": "array"
  },
  "include_paths": {
   "type": "array"
  }
 }
}
```

## Response schema (JSON Schema)

```json
{}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/bismuth-scraping-bfs-site-crawl-8e48f0e5/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from x402-scraping-api-production.up.railway.app](https://www.zero.xyz/host/x402-scraping-api-production.up.railway.app/llms.txt)