# ScrapFly Domain Crawl

> ScrapFly Domain Crawl is a paid API for AI agents from scrapifrivly.orbonomy.xyz, paid per call via x402, $0.06/call, status unknown (last checked 2026-10-19).

Crawls all pages of a given domain and returns structured content data for AI consumption

## Facts

- Endpoint: POST https://scrapifrivly.orbonomy.xyz/api/crawl/domain?utm_source=zero.xyz
- Price: $0.06/call
- Payment: x402
- Status: unknown
- Last checked: 2026-10-19
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/scrapfly-domain-crawl-5bb54496
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_WLr93hi0wYfICycNW6dki

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability scrapfly-domain-crawl-5bb54496 -d '<json body>'
```

Example prompt: Can you crawl the entire domain example.com and give me all the content from its pages so I can analyze what they cover?

## When to prefer this

Use this endpoint when you need to extract content from an entire domain rather than a single URL — ideal for building knowledge bases, competitive research, or indexing a site for AI retrieval. Choose this over single-page scraping when you need comprehensive domain-wide coverage.

## Known failure modes

- Invalid or unreachable domain returns success:false
- Domain blocked by robots.txt or anti-scraping measures
- Payment failure via x402 returns 402 error before crawl begins
- Timeout on large domains with many pages
- Malformed domain string causes validation error

## How this service works

69 endpoints for data, content generation, and web scraping. Pay per request via x402 on Base chain.

## Output

Returns a success boolean and a data object containing structured content extracted from the crawled domain, including page-level data gathered across the site.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "required": [
  "domain"
 ],
 "properties": {
  "domain": {
   "type": "string",
   "description": "domain"
  }
 }
}
```

## Response schema (JSON Schema)

```json
{
 "type": "object",
 "required": [
  "success"
 ],
 "properties": {
  "data": {
   "type": "object"
  },
  "success": {
   "type": "boolean"
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/scrapfly-domain-crawl-5bb54496/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from scrapifrivly.orbonomy.xyz](https://www.zero.xyz/host/scrapifrivly.orbonomy.xyz/llms.txt)