# Machine Learning Metrics: Calibration (ECE, MCE, Brier Score)

> Machine Learning Metrics: Calibration (ECE, MCE, Brier Score) is a paid API for AI agents from ml-metrics.x402tookit.com, paid per call via x402, $0.02/call, status unknown (last checked 2026-11-02).

Computes reliability bins, expected calibration error (ECE), maximum calibration error (MCE), and Brier score from binary labels and predicted probabilities.

## Facts

- Endpoint: GET https://ml-metrics.x402tookit.com/calibration?utm_source=zero.xyz
- Price: $0.02/call
- Payment: x402
- Status: unknown
- Last checked: 2026-11-02
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/machine-learning-metrics-calibration-ece-mce-brier-score-b6ee779f
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_zmuUUZ__w9STHL5leGbYg

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability machine-learning-metrics-calibration-ece-mce-brier-score-b6ee779f
```

Example prompt: Check how well my binary classifier is calibrated — here are 500 true labels (0/1): [0,1,0,1,...] and their predicted probabilities: [0.1,0.9,0.2,0.8,...]. Use 10 uniform-width bins and give me the ECE, MCE, Brier score, and reliability bin breakdown.

## When to prefer this

Use this endpoint when you need to assess the probabilistic calibration quality of a binary classification model — specifically when you want ECE, MCE, or Brier score along with per-bin reliability data for plotting calibration curves. Prefer this over simple accuracy metrics when the actual probability values matter (e.g., risk scoring, medical diagnosis, fraud detection). Choose the quantile strategy when class imbalance is present. Other endpoints in this family cover ROC/AUC, precision-recall, confusion matrix, and log loss.

## Known failure modes

- Missing required 'labels' or 'probs' query parameters returns an error
- Labels and probs arrays of different lengths cause a validation failure
- Non-binary values in labels (anything other than 0 or 1) will be rejected
- Probabilities outside [0, 1] range are invalid and may cause errors
- Fewer than 2 samples or empty arrays are not supported
- bins parameter outside range 2–50 may be rejected or clamped
- Malformed comma-separated strings or invalid JSON arrays cause parse errors

## How this service works

Reliability bins, expected and maximum calibration error and Brier score from binary labels and predicted probabilities.

## Output

Returns a JSON object with: per-bin statistics (bin index, mean predicted probability, observed positive rate, count, gap, direction), overall expected calibration error (ECE), maximum calibration error (MCE), Brier score, binning strategy used, sample count, a verdict and summary string, and optional warnings/reasons if the computation is degraded.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "required": [
  "input"
 ],
 "properties": {
  "input": {
   "type": "object",
   "required": [
    "type",
    "method"
   ],
   "properties": {
    "type": {
     "type": "string",
     "const": "http"
    },
    "method": {
     "enum": [
      "GET"
     ],
     "type": "string"
    },
    "queryParams": {
     "type": "object",
     "required": [
      "labels",
      "probs"
     ],
     "properties": {
      "bins": {
       "type": "integer",
       "description": "Number of bins, 2 to 50, default 10"
      },
      "probs": {
       "type": "string",
       "description": "Predicted probability of class 1 per sample (0 to 1), same length"
      },
      "labels": {
       "type": "string",
       "description": "True binary labels 0/1, comma-separated or JSON array, up to 10000 values"
      },
      "strategy": {
       "type": "string",
       "description": "uniform (equal-width bins, default) or quantile (equal-count bins)"
      }
     }
    }
   },
   "additionalProperties": false
  },
  "output": {
   "type": "object",
   "required": [
    "type"
   ],
   "properties": {
    "type": {
     "type": "string"
    },
    "example": {
     "type": "object",
     "properties": {
      "asOf": {
       "type": [
        "null",
        "string"
       ]
      },
      "bins": {
       "type": [
        "array",
        "null"
       ],
       "items": {
        "type": "object",
        "properties": {
         "bin": {
          "type": [
           "null",
           "number"
          ]
         },
         "gap": {
          "type": [
           "null",
           "number"
          ]
         },
         "count": {
          "type": [
           "null",
           "number"
          ]
         },
         "lowest": {
          "type": [
           "null",
           "number"
          ]
         },
         "highest": {
          "type": [
           "null",
           "number"
          ]
         },
         "direction": {
          "type": [
           "null",
           "string"
          ]
         },
         "observedRate": {
          "type": [
           "null",
           "number"
          ]
         },
         "meanPredicted": {
          "type": [
           "null",
           "number"
          ]
         }
        }
       }
      },
      "source": {
       "type": [
        "null",
        "string"
       ]
      },
      "reasons": {
       "t
… (truncated)
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/machine-learning-metrics-calibration-ece-mce-brier-score-b6ee779f/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from ml-metrics.x402tookit.com](https://www.zero.xyz/host/ml-metrics.x402tookit.com/llms.txt)