# Bytemine Crawl Map – Website URL Discovery

> Bytemine Crawl Map – Website URL Discovery is a paid API for AI agents from x402.orthogonal.com, paid per call via x402, $0.03/call, status unknown (last checked 2026-09-14).

Discovers and returns all URLs on a website by combining sitemap parsing and link extraction, scored and sorted by relevance without scraping page content.

## Facts

- Endpoint: POST https://x402.orthogonal.com/bytemine/crawl/map
- Price: $0.03/call
- Payment: x402
- Status: unknown
- Last checked: 2026-09-14
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/bytemine-crawl-map-website-url-discovery-a75055d0
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_PFK_QCDZ_2l7DO-VzFd1K

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability bytemine-crawl-map-website-url-discovery-a75055d0 -d '<json body>'
```

Example prompt: Can you map all the URLs on https://stripe.com — I want up to 200 URLs, filtered to ones containing 'docs'?

## When to prefer this

Choose this endpoint when you need to inventory or explore the URL structure of a website without fetching full page content — ideal for SEO audits, content discovery, competitor research, or pre-crawl planning. It is faster and cheaper than full-content scraping endpoints and supports optional keyword filtering to narrow results. Prefer this over generic scraping endpoints when you only need URLs, not HTML or text bodies.

## Known failure modes

- Target site has no sitemap and few internal links, resulting in a sparse URL list
- URL is unreachable or returns a non-200 status, causing an empty or error response
- Keyword filter is too restrictive and returns zero matching URLs
- Site uses JavaScript-rendered navigation that prevents static link extraction
- Rate limiting or robots.txt blocking by the target domain

## How this service works

Discover all URLs on a website via sitemap + link extraction. Returns scored/sorted URLs without scraping content. 1 credit.

## Output

Returns a scored and sorted list of discovered URLs on the target website, derived from both sitemap parsing and link extraction, without fetching or returning page content. URLs may be filtered by keyword and capped at a specified limit.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "properties": {
  "url": {
   "type": "string",
   "description": "Website URL to map"
  },
  "limit": {
   "type": "integer",
   "description": "Max URLs to return (default 100)"
  },
  "search": {
   "type": "string",
   "description": "Keyword filter for URLs"
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/bytemine-crawl-map-website-url-discovery-a75055d0/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from x402.orthogonal.com](https://www.zero.xyz/host/x402.orthogonal.com/llms.txt)