# Citiview Website Crawler

> Citiview Website Crawler is a paid API for AI agents from agents.citiviewgroup.com, paid per call via x402, $0.025/call, status unknown (last checked 2026-10-15).

Crawls a website starting from a given URL, follows same-site links up to 100 pages, and returns each page's URL, title, and cleaned Markdown text.

## Facts

- Endpoint: GET https://agents.citiviewgroup.com/v1/crawl?utm_source=zero.xyz
- Price: $0.025/call
- Payment: x402
- Status: unknown
- Last checked: 2026-10-15
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/citiview-website-crawler-f5616b3c
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_ULrDdyNY-A9SGmHHsSJHC

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability citiview-website-crawler-f5616b3c
```

Example prompt: Can you crawl the documentation site at https://docs.example.com/api/ — only follow links under /api/, up to 50 pages — and give me all the page titles and their content as markdown?

## When to prefer this

Choose this endpoint when you need to extract textual content from multiple pages of a website in one call, especially for documentation sites, blogs, or content archives. It is ideal when you want clean Markdown output rather than raw HTML, and when following internal links automatically saves you from orchestrating individual page fetches. Prefer it over single-page scrapers when the target content spans many URLs under a shared domain or path prefix.

## Known failure modes

- Start URL is unreachable or returns non-200 — crawl fails or returns empty results
- robots.txt disallows crawling — pages are skipped or crawl is blocked
- Path prefix does not match any linked pages — only the start page is returned
- Max pages exceeded silently — results capped at 100
- Store link expires after 7 days — downstream consumers must fetch before expiry
- JavaScript-heavy SPAs may not render fully — content may be incomplete

## How this service works

Crawl a website into clean Markdown: follows same-site links from a start page (optionally under a path prefix), up to 100 pages, respecting robots.txt. Returns each page's URL, title and text. $0.005 + $0.001 per page. Add store=true (+$0.001) to get a 7-day share link instead of the data inline.

## Output

A list of up to 100 crawled pages, each with its URL, title, and full page content converted to clean Markdown. If store=true is requested, returns a 7-day share link pointing to the data instead of returning it inline.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "required": [
  "input"
 ],
 "properties": {
  "input": {
   "type": "object",
   "required": [
    "type",
    "method"
   ],
   "properties": {
    "type": {
     "type": "string",
     "const": "http"
    },
    "method": {
     "enum": [
      "GET"
     ],
     "type": "string"
    },
    "queryParams": {
     "type": "object",
     "required": [
      "url"
     ],
     "properties": {
      "url": {
       "type": "string",
       "description": "Start page"
      },
      "store": {
       "type": "boolean",
       "description": "Return a 7-day share link (+$0.001)"
      },
      "max_pages": {
       "type": "integer",
       "description": "1-100, default 20"
      },
      "path_prefix": {
       "type": "string",
       "description": "Only follow links under this path, e.g. /docs/"
      }
     }
    }
   },
   "additionalProperties": false
  },
  "output": {
   "type": "object",
   "required": [
    "type"
   ],
   "properties": {
    "type": {
     "type": "string"
    },
    "example": {
     "type": "object"
    }
   }
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/citiview-website-crawler-f5616b3c/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from agents.citiviewgroup.com](https://www.zero.xyz/host/agents.citiviewgroup.com/llms.txt)