# Context.dev Sitemap Crawler

> Context.dev Sitemap Crawler is a paid API for AI agents from Context.dev at mpp.orthogonal.com, paid per call via MPP, $0.03/call, status unknown (last checked 2026-10-11).

Crawls a domain's sitemap (including recursive sitemap index files) and returns all discovered page URLs

## Facts

- Endpoint: GET https://mpp.orthogonal.com/context-dev/web/scrape/sitemap?utm_source=zero.xyz
- Price: $0.03/call
- Payment: MPP
- Status: unknown
- Last checked: 2026-10-11
- Activations on Zero: 0
- Provider: Context.dev
- Docs: https://www.context.dev
- Website: https://www.context.dev
- Tags: scraping, brand, context, extraction, web, data
- Canonical page: https://www.zero.xyz/c/context-dev-sitemap-crawler-c9871ccf
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_RuBTfJEZ-JG5EmcpLN-by

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability context-dev-sitemap-crawler-c9871ccf
```

Example prompt: Can you crawl the sitemap for example.com and give me a full list of every page URL it has, including any linked sub-sitemaps?

## When to prefer this

Use this endpoint when you need a complete, structured list of all publicly indexed pages on a domain without having to crawl the site page-by-page. It's ideal for SEO analysis, content audits, competitor research, or pre-flight checks before deeper scraping. Prefer this over general web crawlers when the target site maintains a well-structured sitemap.

## Known failure modes

- Domain has no sitemap.xml — returns empty result or 404 error
- Sitemap is malformed XML — parsing error returned
- Domain is unreachable or rate-limiting the crawler — timeout or connection error
- Sitemap is extremely large — may hit response size or time limits
- Sitemap uses non-standard format not supported by parser

## How this service works

Crawls the sitemap of the given domain and returns all discovered page URLs. Supports sitemap index files (recursive), parallel fetching with concurrency control, deduplication, and filters out non-page resources (images, PDFs, etc.).

## Output

A list of all page URLs discovered from the domain's sitemap.xml and any linked sitemap index files, returned recursively so no sub-sitemap is missed.

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/context-dev-sitemap-crawler-c9871ccf/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from mpp.orthogonal.com](https://www.zero.xyz/host/mpp.orthogonal.com/llms.txt)