# BLEU and ROUGE Metrics Calculator

> BLEU and ROUGE Metrics Calculator is a paid API for AI agents from ml-metrics.x402tookit.com, paid per call via x402, $0.02/call, status unknown (last checked 2026-11-02).

Computes sentence BLEU (1–4 gram with brevity penalty and optional smoothing) and ROUGE-1, ROUGE-2, ROUGE-L F1/precision/recall scores for a candidate text against one or more reference texts.

## Facts

- Endpoint: GET https://ml-metrics.x402tookit.com/bleu-rouge?utm_source=zero.xyz
- Price: $0.02/call
- Payment: x402
- Status: unknown
- Last checked: 2026-11-02
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/bleu-and-rouge-metrics-calculator-0beb6647
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_2c4FdGZkrgJWx4UctplUZ

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability bleu-and-rouge-metrics-calculator-0beb6647
```

Example prompt: Score my model's output against these two reference sentences using BLEU up to 4-gram with epsilon smoothing and also give me ROUGE-1, ROUGE-2, and ROUGE-L — candidate: 'The cat sat on the mat', references: 'The cat is on the mat' and 'A cat was sitting on the mat'.

## When to prefer this

Choose this endpoint when you need standard NLP evaluation metrics (BLEU and ROUGE) in a single call without setting up NLTK, sacrebleu, or rouge-score locally. Ideal for agents evaluating summarization, translation, or generation quality in real time, especially when infrastructure-free computation is needed. Prefer it over rolling your own when you need brevity-penalty-correct BLEU with configurable n-gram order and smoothing alongside ROUGE-L simultaneously.

## Known failure modes

- Missing required 'candidate' or 'references' query parameter — returns 400 validation error
- Candidate or reference text exceeds 5000-character limit — returns 400 or truncation error
- More than 20 reference texts provided — exceeds limit, returns 400
- Invalid 'smoothing' or 'tokenizer' enum value — returns 400 with description
- Empty candidate or reference string — may return null BLEU or ROUGE values with a rougeNote
- Payment not provided or insufficient — returns 402 Payment Required

## How this service works

Sentence BLEU (1-4 gram, brevity penalty, optional smoothing) and ROUGE-1, ROUGE-2, ROUGE-L F1 for text against references.

## Output

Returns a JSON object containing the BLEU score and its n-gram order, ROUGE-1, ROUGE-2, and ROUGE-L each with precision, recall, and F1, along with metadata such as smoothing method used, tokenizer, an optional summary/verdict, and any degraded-mode or disclaimer notes.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "required": [
  "input"
 ],
 "properties": {
  "input": {
   "type": "object",
   "required": [
    "type",
    "method"
   ],
   "properties": {
    "type": {
     "type": "string",
     "const": "http"
    },
    "method": {
     "enum": [
      "GET"
     ],
     "type": "string"
    },
    "queryParams": {
     "type": "object",
     "required": [
      "candidate",
      "references"
     ],
     "properties": {
      "maxn": {
       "type": "integer",
       "description": "Highest BLEU n-gram order, 1 to 4, default 4"
      },
      "candidate": {
       "type": "string",
       "description": "Generated text, up to 5000 characters"
      },
      "smoothing": {
       "type": "string",
       "description": "none (default), add1 (add one to n>1 counts), or epsilon (0.1 for zero matches)"
      },
      "tokenizer": {
       "type": "string",
       "description": "whitespace (default, text is split on spaces as given) or words (lowercase, punctuation removed)"
      },
      "references": {
       "type": "string",
       "description": "One or more reference texts separated by a line break or |||, up to 20 references, 5000 characters each"
      }
     }
    }
   },
   "additionalProperties": false
  },
  "output": {
   "type": "object",
   "required": [
    "type"
   ],
   "properties": {
    "type": {
     "type": "string"
    },
    "example": {
     "type": "object",
     "properties": {
      "asOf": {
       "type": [
        "null",
        "string"
       ]
      },
      "bleu": {
       "type": [
        "null",
        "number"
       ]
      },
      "rouge1": {
       "type": [
        "null",
        "object"
       ],
       "properties": {
        "f1": {
         "type": [
          "null",
          "number"
         ]
        },
        "recall": {
         "type": [
          "null",
          "number"
         ]
        },
        "precision": {
         "type": [
          "null",
          "number"
         ]
        }
       }
      },
      "rouge2": {
       "type": [
        "null",
        "object"
       ],
       "properties": {
        "f1": {
         "type": [
          "null",
          "number"
         ]
        },
        "recall": {
         "type": [
          "null",
          "number"
         ]
        },
        "precision": {
         "type": [
          "null",
          "number"
         ]
        }
       }
      },
   
… (truncated)
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/bleu-and-rouge-metrics-calculator-0beb6647/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from ml-metrics.x402tookit.com](https://www.zero.xyz/host/ml-metrics.x402tookit.com/llms.txt)