# AI Calibration Error Calculator

> AI Calibration Error Calculator is a paid API for AI agents from aialign.halowerk.com, paid per call via x402, $0.003/call, status unknown (last checked 2026-09-14).

Computes Brier loss, expected calibration error (ECE), and maximum calibration error (MCE) from a set of confidence scores and binary correctness labels, partitioned into equal-width bins.

## Facts

- Endpoint: POST https://aialign.halowerk.com/v1/confidence-calib
- Price: $0.003/call
- Payment: x402
- Status: unknown
- Last checked: 2026-09-14
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/ai-calibration-error-calculator-beef1bac
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_hv_18GfwVwOJKu3bwl-Ph

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability ai-calibration-error-calculator-beef1bac -d '<json body>'
```

Example prompt: Check how well-calibrated these 500 model predictions are — I'll give you each prediction's confidence score (between 0 and 1) and whether it was correct (true/false). Split them into 10 equal-width bins and give me the Brier loss, expected calibration error, and maximum calibration error.

## When to prefer this

Choose this endpoint when you need fast, stateless calibration metrics (Brier, ECE, MCE) for a batch of binary predictions with scalar confidences. It is ideal for post-hoc model evaluation pipelines, CI/CD regression checks on model calibration, or alignment audits where you have ground-truth correctness labels. It does not perform label validation, class-imbalance correction, or multi-class calibration — for those needs, use a more specialized tool.

## Known failure modes

- Empty predictions array returns a validation error
- Confidence values outside [0,1] are rejected
- bin_count below 2 or above 100 is rejected
- Prediction IDs exceeding 128 characters fail schema validation
- Payload exceeding 100,000 predictions is rejected with a size error
- Missing required fields (prediction_id, confidence, correct) cause a 400 error

## How this service works

Compares scalar confidence with binary correctness, calculates mean squared Brier loss, partitions confidence into caller-selected equal-width bins and reports expected and maximum calibration error. It does not validate labels, correct class imbalance or prove calibration beyond the supplied sample.

## Output

Returns the mean squared Brier loss across all predictions, the expected calibration error (ECE) as the weighted average gap between confidence and accuracy per bin, the maximum calibration error (MCE) as the worst-case bin gap, and per-bin statistics including confidence range, mean confidence, observed accuracy, and sample count.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "properties": {
  "bin_count": {
   "type": "integer",
   "maximum": 100,
   "minimum": 2
  },
  "predictions": {
   "type": "array",
   "items": {
    "type": "object",
    "required": [
     "prediction_id",
     "confidence",
     "correct"
    ],
    "properties": {
     "correct": {
      "type": "boolean"
     },
     "confidence": {
      "type": "number",
      "maximum": 1,
      "minimum": 0
     },
     "prediction_id": {
      "type": "string",
      "maxLength": 128,
      "minLength": 1
     }
    },
    "additionalProperties": false
   },
   "maxItems": 100000,
   "minItems": 1
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/ai-calibration-error-calculator-beef1bac/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from aialign.halowerk.com](https://www.zero.xyz/host/aialign.halowerk.com/llms.txt)