# Oassis Web Crawler (Job-based)

> Oassis Web Crawler (Job-based) is a paid API for AI agents from api.oassis.dev, paid per call via x402, $0.002/call, status unknown (last checked 2026-10-11).

Follows a site's links and crawls every reachable page as an asynchronous background job, returning a job ID for status polling.

## Facts

- Endpoint: GET https://api.oassis.dev/web/v1/crawl?utm_source=zero.xyz
- Price: $0.002/call
- Payment: x402
- Status: unknown
- Last checked: 2026-10-11
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/oassis-web-crawler-job-based-555e29cb
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_-QKv24gjiOrnQFy7IrmhN

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability oassis-web-crawler-job-based-555e29cb
```

Example prompt: Please crawl the entire site at https://docs.example.com — follow all the links and collect every page — and give me the job ID so I can check on it when it's done.

## When to prefer this

Choose this endpoint when you need to recursively crawl an entire website by following links — not just scrape a single URL. It is ideal for bulk content ingestion, site indexing for RAG pipelines, or knowledge base extraction where completeness across many pages matters. Prefer this over single-page scrape endpoints when the number of target URLs is unknown upfront and you want the service to discover and collect pages automatically. The async job model makes it suitable for large sites where results won't be ready instantly.

## Known failure modes

- Invalid or unreachable URL results in a failed job status on polling
- No URL provided in the body causes a 400 bad request
- Rate limiting or payment failure (x402) returns 402 Payment Required
- Crawl limit exceeded stops collection at the configured page count
- Private or bot-blocked sites may return empty or partial results

## How this service works

Follows a site's links and reads every page, as a job.

## Output

Returns a JSON object containing a job ID (e.g. 'e6af…'), the crawl limit (number of pages), and an initial status of 'queued'. The agent can then poll a separate status endpoint using the job ID to retrieve crawled page content as the job progresses.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "required": [
  "input",
  "output"
 ],
 "properties": {
  "input": {
   "type": "object",
   "required": [
    "type",
    "method",
    "bodyType",
    "body"
   ],
   "properties": {
    "body": {
     "type": "object"
    },
    "type": {
     "enum": [
      "http"
     ],
     "type": "string"
    },
    "method": {
     "enum": [
      "GET"
     ],
     "type": "string"
    },
    "bodyType": {
     "enum": [
      "json"
     ],
     "type": "string"
    }
   },
   "additionalProperties": false
  },
  "output": {
   "type": "object",
   "required": [
    "type",
    "example"
   ],
   "properties": {
    "type": {
     "enum": [
      "json"
     ],
     "type": "string"
    },
    "example": {
     "type": "object"
    }
   },
   "additionalProperties": false
  }
 }
}
```

## Response schema (JSON Schema)

```json
{
 "type": "json",
 "example": {
  "jobId": "e6af…",
  "limit": 5,
  "status": "queued"
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/oassis-web-crawler-job-based-555e29cb/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from api.oassis.dev](https://www.zero.xyz/host/api.oassis.dev/llms.txt)