# TD Agent Utility – Site Crawl Verification

> TD Agent Utility – Site Crawl Verification is a paid API for AI agents from tdagentutility.co.uk, paid per call via x402, $0.02/call, status unknown (last checked 2026-10-16).

Crawls a target website and returns HTTP response data, crawl statistics, and pass/fail verification status for deployment validation.

## Facts

- Endpoint: POST https://tdagentutility.co.uk/v1/site-crawl?utm_source=zero.xyz
- Price: $0.02/call
- Payment: x402
- Status: unknown
- Last checked: 2026-10-16
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/td-agent-utility-site-crawl-verification-8a5fbe37
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_zGlg5mEQ-kPW2OMvcl84a

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability td-agent-utility-site-crawl-verification-8a5fbe37 -d '<json body>'
```

Example prompt: Can you crawl https://myapp.example.com and give me an independent verification of whether the site is live and returning pages successfully — I just deployed a new version and want to confirm everything is up.

## When to prefer this

Choose this endpoint when you need an independent, third-party HTTP crawl-based verification of a website's liveness and deployment status — particularly after a new release, DNS change, or infrastructure update. It is ideal when a simple ping or HEAD request is not sufficient and you need evidence of actual page content being served. Prefer this over generic uptime monitors when you want deterministic pass/fail evidence with crawl statistics.

## Known failure modes

- Target URL is unreachable or returns 5xx errors — result_status will be FAIL with warnings
- DNS resolution failure for the target domain — crawl fails with zero successful requests
- Target URL is malformed or missing — request rejected
- Site blocks crawlers via robots.txt or firewall — pages array may be empty with warnings
- Timeout on slow-loading pages — partial results with failed count > 0

## How this service works

Independent website and deployment verification for AI agents: HTTP, DNS, TLS and Chromium screenshot evidence. Three deterministic tools, x402 V2 Base USDC.

## Output

A JSON object with a 'tool' field set to 'site_crawl', an array of crawled pages, a summary object containing counts of attempted, successful, and failed requests, bytes read, and frontier pages remaining, a list of warnings, the checked_at timestamp, the requested URL, and a result_status of PASS or FAIL.

## Response schema (JSON Schema)

```json
{
 "type": "json",
 "example": {
  "tool": "site_crawl",
  "pages": [],
  "summary": {
   "failed": 0,
   "requests": 0,
   "attempted": 0,
   "bytes_read": 0,
   "successful": 0,
   "frontier_remaining": 0,
   "requested_max_pages": 0
  },
  "warnings": [],
  "checked_at": "2026-10-15T00:00:00Z",
  "requested_url": "",
  "result_status": "PASS"
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/td-agent-utility-site-crawl-verification-8a5fbe37/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from tdagentutility.co.uk](https://www.zero.xyz/host/tdagentutility.co.uk/llms.txt)