# Tavily Crawl

> Tavily Crawl is a paid API for AI agents from Tavily at tavily.mpp.paywithlocus.com, paid per call via MPP, $0.21+ (scales with page limit) (dynamic), status unknown (last checked 2026-10-12).

Crawls a website starting from a given URL, traversing multiple pages up to configurable depth and breadth limits, and extracting content from each page

## Facts

- Endpoint: POST https://tavily.mpp.paywithlocus.com/tavily/crawl?utm_source=zero.xyz
- Price: $0.21+ (scales with page limit) (dynamic)
- Payment: MPP
- Status: unknown
- Last checked: 2026-10-12
- Activations on Zero: 0
- Provider: Tavily
- Docs: https://docs.tavily.com
- Website: https://tavily.com
- Tags: ai-search, web-search, extraction, crawling, search, web
- Canonical page: https://www.zero.xyz/c/tavily-crawl-002ca16e
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_msAn0Blq3n8zNFPqrv8om

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability tavily-crawl-002ca16e -d '<json body>'
```

Example prompt: Crawl https://docs.example.com starting from the homepage, go up to 3 levels deep, and extract all the content from up to 20 pages using advanced extraction depth.

## When to prefer this

Use this endpoint when you need to extract content from multiple interconnected pages of a website, rather than a single page. Ideal for indexing documentation sites, gathering competitive intelligence across a whole domain, or building knowledge bases from web content. Prefer this over the Extract endpoint when you need multi-page traversal, and over the Search endpoint when you already know the target site and don't need a query-based search.

## Known failure modes

- URL is unreachable or returns non-200 status — crawl returns empty or partial results
- Crawl limit reached before full site traversal — only partial content returned
- Max depth too shallow to reach target pages — relevant content may be missed
- Website blocks crawlers via robots.txt or rate limiting — reduced or no content returned
- Invalid URL format — returns validation error

## How this service works

Crawl a website and extract content from discovered pages. Combines mapping and extraction in a single call.

## Output

A collection of crawled pages with their URLs and extracted text content, gathered by traversing the site from the starting URL according to the specified depth, breadth, and instruction constraints.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "required": [
  "url"
 ],
 "properties": {
  "url": {
   "type": "string",
   "description": "Starting URL to crawl"
  },
  "limit": {
   "type": "number",
   "description": "Maximum number of pages to crawl (default 10)"
  },
  "max_depth": {
   "type": "number",
   "description": "Maximum crawl depth"
  },
  "max_breadth": {
   "type": "number",
   "description": "Maximum breadth per level"
  },
  "instructions": {
   "type": "string",
   "description": "Natural language instructions to guide crawling"
  },
  "extract_depth": {
   "type": "string",
   "description": "Extraction depth: 'basic' (default) or 'advanced'"
  }
 },
 "additionalProperties": true
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/tavily-crawl-002ca16e/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from tavily.mpp.paywithlocus.com](https://www.zero.xyz/host/tavily.mpp.paywithlocus.com/llms.txt)