# AGISHub Crawl Map

> AGISHub Crawl Map is a paid API for AI agents from api.agishub.com, paid per call via x402, $0.05/call, status unknown (last checked 2026-09-14).

Discovers all URLs reachable from a domain within a configurable link depth, returning a flat list of absolute URLs while respecting robots.txt crawl delays.

## Facts

- Endpoint: POST https://api.agishub.com/paid/crawl-map
- Price: $0.05/call
- Payment: x402
- Status: unknown
- Last checked: 2026-09-14
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/agishub-crawl-map-dd4d5222
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_hglv6ATY7VaBxLHPZQPM6

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability agishub-crawl-map-dd4d5222 -d '<json body>'
```

Example prompt: Can you map all the URLs reachable from https://example.com — up to 150 URLs — and only include ones that match the pattern '/blog'?

## When to prefer this

Use this endpoint when you need to discover the full URL structure of a website before targeted scraping or content extraction. It is ideal for pre-crawl reconnaissance, sitemap generation, and finding pages matching a URL pattern. Prefer this over manual link-following or a full page-fetch endpoint when you need breadth-first link discovery across an entire domain with optional subdomain inclusion.

## Known failure modes

- Invalid or unreachable root URL returns an error
- Limit of 0 or above 200 is rejected by schema validation
- Regex filter syntax errors may return no results or an error
- Sites blocking crawlers via robots.txt may yield fewer URLs than the limit
- Network timeouts on slow or large sites may truncate results

## How this service works

Discover all URLs reachable from a domain within a link depth limit. Returns a flat list of absolute URLs, respects robots.txt crawl delays. Use it to map a site's structure before crawling.

## Output

A flat JSON list of absolute URLs reachable from the given root domain, filtered by the optional regex pattern, capped at the specified limit (default 100, max 200). The crawler respects robots.txt crawl delays during discovery.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "required": [
  "input",
  "output"
 ],
 "properties": {
  "input": {
   "type": "object",
   "required": [
    "type",
    "method",
    "bodyType",
    "body"
   ],
   "properties": {
    "body": {
     "type": "object",
     "required": [
      "url"
     ],
     "properties": {
      "url": {
       "type": "string",
       "format": "uri",
       "description": "Root domain URL to map (e.g., https://example.com)."
      },
      "limit": {
       "type": "integer",
       "maximum": 200,
       "minimum": 0,
       "description": "Maximum URLs to return (default 100, max 200 per call). Respects robots.txt crawl-delay.",
       "exclusiveMinimum": true
      },
      "search": {
       "type": "string",
       "description": "Optional regex or plain string to filter results (case-insensitive)."
      },
      "include_subdomains": {
       "type": "boolean",
       "description": "Include URLs from subdomains (default false, same domain only)."
      }
     },
     "additionalProperties": false
    },
    "type": {
     "enum": [
      "http"
     ],
     "type": "string"
    },
    "method": {
     "enum": [
      "POST"
     ],
     "type": "string"
    },
    "bodyType": {
     "enum": [
      "json"
     ],
     "type": "string"
    }
   },
   "additionalProperties": false
  },
  "output": {
   "type": "object",
   "required": [
    "type",
    "example"
   ],
   "properties": {
    "type": {
     "enum": [
      "json"
     ],
     "type": "string"
    },
    "example": {
     "type": "object"
    }
   },
   "additionalProperties": false
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/agishub-crawl-map-dd4d5222/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from api.agishub.com](https://www.zero.xyz/host/api.agishub.com/llms.txt)