# Johto Data API - Web Crawler

> Johto Data API - Web Crawler is a paid API for AI agents from johto-x402-api.johtodata.workers.dev, paid per call via x402, $0.05/call, status unknown (last checked 2026-10-30).

Crawls a website starting from a given URL and returns the content of up to 20 pages as Markdown, with metadata per page

## Facts

- Endpoint: GET https://johto-x402-api.johtodata.workers.dev/v1/crawl?utm_source=zero.xyz
- Price: $0.05/call
- Payment: x402
- Status: unknown
- Last checked: 2026-10-30
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/johto-data-api-web-crawler-fed8079a
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_C8weR0-l_aUmNdWrouazV

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability johto-data-api-web-crawler-fed8079a
```

Example prompt: Crawl the site at https://docs.example.com starting from the homepage, grab up to 10 pages, and give me each page's content as Markdown so I can use it for RAG.

## When to prefer this

Choose this endpoint when you need to crawl multiple pages of a website (up to 20) and receive structured Markdown output per page, especially for LLM ingestion, RAG pipelines, or competitive intelligence. It is pay-per-call with no charge on failure, making it cost-safe for exploratory crawls. Prefer over single-page scrapers when you need to follow links within a site automatically.

## Known failure modes

- URL is unreachable or returns non-200 status — no charge applied
- URL pattern does not start with http/https — rejected by schema validation
- pages parameter outside 1-20 range — rejected
- max_chars outside 500-20000 range — rejected
- Site blocks crawlers (robots.txt, bot detection) — may return empty or partial results
- Payment failure via x402 — request not fulfilled

## How this service works

Valuable data. Quality leads. Paid per call in USDC on Base via x402. Not charged if the fetch fails or nothing matches. Prices per call (USDC): /v1/jp-invoice-check $0.002; /v1/sec-13f $0.003; /v1/sec-8k $0.003; /v1/sanctions-screen $0.0025; /v1/tenders $0.011; /v1/uk-tenders $0.003; /v1/fr-tenders $0.006; /v1/au-contracts $0.0035; /v1/wallet-risk $0.0025; /v1/kyb $0.0029; /v1/read $0.0019; /v1/crawl $0.0045; /v1/jobs $0.01; /v1/domain-profile $0.003.

## Output

A JSON array of page objects, each containing the page URL, crawl depth, title, HTTP status code, load timestamp, full Markdown content (possibly truncated), a content hash, a description field, and an estimated token count. Also includes a top-level 'rows' count indicating how many pages were returned.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "required": [
  "input"
 ],
 "properties": {
  "input": {
   "type": "object",
   "required": [
    "type",
    "method"
   ],
   "properties": {
    "type": {
     "type": "string",
     "const": "http"
    },
    "method": {
     "enum": [
      "GET"
     ],
     "type": "string"
    },
    "queryParams": {
     "type": "object",
     "required": [
      "url"
     ],
     "properties": {
      "url": {
       "type": "string",
       "pattern": "^https?://\\S+$",
       "description": "Start URL, http(s). Only pages under it are crawled."
      },
      "pages": {
       "type": "string",
       "pattern": "^([1-9]|1[0-9]|20)$",
       "description": "Max pages, 1-20 (default 20)"
      },
      "max_chars": {
       "type": "string",
       "pattern": "^[0-9]{3,5}$",
       "description": "Max Markdown characters per page, 500-20000 (default 20000)"
      }
     }
    }
   },
   "additionalProperties": false
  },
  "output": {
   "type": "object",
   "required": [
    "type"
   ],
   "properties": {
    "type": {
     "type": "string"
    },
    "example": {
     "type": "object",
     "required": [
      "data",
      "rows"
     ],
     "properties": {
      "data": {
       "type": "array",
       "items": {
        "type": "object",
        "properties": {
         "url": {
          "type": "string"
         },
         "depth": {
          "type": "integer"
         },
         "title": {
          "type": "string"
         },
         "status": {
          "type": "integer"
         },
         "loadedAt": {
          "type": "string"
         },
         "markdown": {
          "type": "string"
         },
         "truncated": {
          "type": "boolean"
         },
         "contentHash": {
          "type": "string"
         },
         "description": {},
         "tokensEstimate": {
          "type": "integer"
         }
        }
       }
      },
      "rows": {
       "type": "integer"
      }
     }
    }
   }
  }
 }
}
```

## Response schema (JSON Schema)

```json
{
 "type": "json",
 "example": {
  "data": [
   {
    "url": "https://crawlee.dev/python/docs/examples/beautifulsoup-crawler.md",
    "depth": 1,
    "title": "BeautifulSoup crawler",
    "status": 200,
    "loadedAt": "2026-10-22T08:54:44Z",
    "markdown": "* [](https://crawlee.dev/python/python)\n* [Examples](https://crawlee.dev/python/ ...",
    "truncated": true,
    "contentHash": "fc2f086388e350bc",
    "description": null,
    "tokensEstimate": 1504
   }
  ],
  "rows": 1
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/johto-data-api-web-crawler-fed8079a/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from johto-x402-api.johtodata.workers.dev](https://www.zero.xyz/host/johto-x402-api.johtodata.workers.dev/llms.txt)