# Pixelart Semantic Object Detection (Vision Model)

> Pixelart Semantic Object Detection (Vision Model) is a paid API for AI agents from pixelart.withzero.xyz, paid per call via MPP, $0.02/call, status unknown (last checked 2026-09-15).

Detects and names distinct objects in a pixel-art sprite frame using a multimodal LLM that correctly handles occlusion and painterly shading, optionally materializing each object as a separate canvas layer.

## Facts

- Endpoint: POST https://pixelart.withzero.xyz/api/v1/canvas/{id}/segment/semantic
- Price: $0.02/call
- Payment: MPP
- Status: unknown
- Last checked: 2026-09-15
- Activations on Zero: 1
- Provider: pixelart.withzero.xyz
- Website: https://pixelart.withzero.xyz
- Canonical page: https://www.zero.xyz/c/pixelart-withzero-xyz-pixelart-semantic-object-detection-vision-model-2cedbd74
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_w0wpGdeCokJSjM4891clX

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability pixelart-withzero-xyz-pixelart-semantic-object-detection-vision-model-2cedbd74 -d '<json body>'
```

Example prompt: Segment frame 0 of canvas abc-123 into up to 6 named parts — hint that it's a lich king boss holding a staff — and materialize each detected object as its own layer.

## When to prefer this

Use this endpoint instead of the free /segment when the sprite has occluded parts (e.g. a staff gripped by a hand), painterly or rim-lit shading that confuses algorithmic segmentation, or when you need human-readable named labels per object for animation rigging. It costs $0.02 USDC per call but produces semantically meaningful groupings that pure algorithmic approaches cannot.

## Known failure modes

- Canvas ID not found — 404 with canvasId error
- Frame index out of range — validation error on frame field
- maxObjects outside 2–12 range — schema validation error
- Vision model fails to identify distinct regions — may return fewer objects than maxObjects
- hint too long (>500 chars) — validation error
- Payment not included or insufficient — payment required error

## How this service works

Group deterministic regions into named animation parts such as head, body, cape, limbs, and held items. Dry-run by default; materialization is revision-guarded and replaces only the selected frame by default. Full agent guide: https://pixelart.withzero.xyz/llms.txt

## Output

Returns an array of detected objects sorted largest-first, each with a model-assigned name (e.g. 'staff', 'cape', 'left-claw'), pixel count, bounding box (x, y, w, h), and centroid coordinates. When materialize=true, each object also includes a layerId UUID referencing the newly created z-ordered canvas layer containing that object's pixels.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "required": [
  "frame",
  "maxObjects",
  "materialize"
 ],
 "properties": {
  "hint": {
   "type": "string",
   "maxLength": 500,
   "description": "Optional context for the vision model, e.g. \"a lich king boss holding a staff\" — improves part naming and grouping."
  },
  "frame": {
   "type": "integer",
   "default": 0,
   "maximum": 9007199254740991,
   "minimum": 0,
   "description": "Frame to segment. Default 0 (the primary canvas)."
  },
  "maxObjects": {
   "type": "integer",
   "default": 6,
   "maximum": 12,
   "minimum": 2,
   "description": "Upper bound on named objects the vision model may return. Default 6."
  },
  "materialize": {
   "type": "boolean",
   "default": false,
   "description": "When true, create one layer per detected object (largest = zIndex 0) with its cel on the target frame."
  }
 },
 "additionalProperties": false
}
```

## Response schema (JSON Schema)

```json
{
 "type": "object",
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "required": [
  "canvasId",
  "frame",
  "objects",
  "materialized"
 ],
 "properties": {
  "frame": {
   "type": "integer",
   "maximum": 9007199254740991,
   "minimum": -9007199254740991
  },
  "objects": {
   "type": "array",
   "items": {
    "type": "object",
    "required": [
     "name",
     "pixelCount",
     "bbox",
     "centroid"
    ],
    "properties": {
     "bbox": {
      "type": "object",
      "required": [
       "x",
       "y",
       "w",
       "h"
      ],
      "properties": {
       "h": {
        "type": "integer",
        "maximum": 9007199254740991,
        "minimum": -9007199254740991
       },
       "w": {
        "type": "integer",
        "maximum": 9007199254740991,
        "minimum": -9007199254740991
       },
       "x": {
        "type": "integer",
        "maximum": 9007199254740991,
        "minimum": -9007199254740991
       },
       "y": {
        "type": "integer",
        "maximum": 9007199254740991,
        "minimum": -9007199254740991
       }
      },
      "additionalProperties": false
     },
     "name": {
      "type": "string",
      "description": "Object name chosen by the vision model (e.g. \"staff\", \"cape\", \"head\")."
     },
     "layerId": {
      "type": "string",
      "format": "uuid",
      "pattern": "^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|00000000-0000-0000-0000-000000000000|ffffffff-ffff-ffff-ffff-ffffffffffff)$",
      "description": "Present when materialize=true."
     },
     "centroid": {
      "type": "object",
      "required": [
       "x",
       "y"
      ],
      "properties": {
       "x": {
        "type": "integer",
        "maximum": 9007199254740991,
        "minimum": -9007199254740991
       },
       "y": {
        "type": "integer",
        "maximum": 9007199254740991,
        "minimum": -9007199254740991
       }
      },
      "additionalProperties": false
     },
     "pixelCount": {
      "type": "integer",
      "maximum": 9007199254740991,
      "minimum": -9007199254740991
     }
    },
    "additionalProperties": false
   },
   "description": "Detected objects, largest first. Occluded parts (e.g. a staff split by a gripping hand) are grouped into one object."
  },
  "canvasId": {
   "type": "string",
   "format": "uuid",
   "pattern": "^([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-8][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}|0
… (truncated)
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/pixelart-withzero-xyz-pixelart-semantic-object-detection-vision-model-2cedbd74/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from pixelart.withzero.xyz](https://www.zero.xyz/host/pixelart.withzero.xyz/llms.txt)