# Context.dev Web Crawl

> Context.dev Web Crawl is a paid API for AI agents from Context.dev at mpp.orthogonal.com, paid per call via MPP, $0.03/call, status unknown (last checked 2026-10-09).

Crawls a website starting from a given URL, extracts page content as Markdown, and returns results for all crawled pages

## Facts

- Endpoint: POST https://mpp.orthogonal.com/context-dev/web/crawl?utm_source=zero.xyz
- Price: $0.03/call
- Payment: MPP
- Status: unknown
- Last checked: 2026-10-09
- Activations on Zero: 0
- Provider: Context.dev
- Docs: https://www.context.dev
- Website: https://www.context.dev
- Tags: scraping, brand, context, extraction, web, data
- Canonical page: https://www.zero.xyz/c/context-dev-web-crawl-49046443
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_bl8klANGOk34exzP4_ORw

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability context-dev-web-crawl-49046443 -d '<json body>'
```

Example prompt: Can you crawl the entire docs site at https://docs.example.com starting from the homepage and give me the content of every page as markdown?

## When to prefer this

Use this endpoint when you need to extract content from multiple pages of a website in a single call, rather than scraping individual pages one at a time. Ideal for indexing documentation sites, extracting all blog posts, or gathering content across an entire domain. Prefer this over the single-page Markdown endpoint when the scope spans more than one URL or when you need to discover and extract all linked pages from a starting point.

## Known failure modes

- URL is unreachable or returns non-200 status — crawl fails or returns empty results
- Target site blocks crawlers via robots.txt or rate limiting — partial or empty results
- Crawl depth/page limit exceeded — only partial site content returned
- Invalid or malformed starting URL — error response indicating bad input
- Site uses heavy JavaScript rendering — content may be incomplete or missing
- Crawl times out on very large sites — partial results returned

## How this service works

Performs a crawl starting from a given URL, extracts page content as Markdown, and returns results for all crawled pages. Only follows links within the same domain as the starting URL. Costs 1 credit per successful page crawled.

## Output

An array of crawled page results, each containing the page URL and its full content converted to Markdown format, covering all pages discovered during the crawl starting from the seed URL.

## Example request

```json
{
 "url": "https://example.com"
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/context-dev-web-crawl-49046443/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from mpp.orthogonal.com](https://www.zero.xyz/host/mpp.orthogonal.com/llms.txt)