# RAG Quality Evaluator

> RAG Quality Evaluator is a paid API for AI agents from rag-quality-evaluator.johntaylor1online.workers.dev, paid per call via x402, $0.5/call, status unknown (last checked 2026-10-18).

Evaluates the quality of Retrieval-Augmented Generation (RAG) pipeline outputs by analyzing input-output pairs

## Facts

- Endpoint: POST https://rag-quality-evaluator.johntaylor1online.workers.dev/evaluate?utm_source=zero.xyz
- Price: $0.5/call
- Payment: x402
- Status: unknown
- Last checked: 2026-10-18
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/rag-quality-evaluator-10ef74e8
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_GoJaGkpHj_LzCVhmftMdn

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability rag-quality-evaluator-10ef74e8 -d '<json body>'
```

Example prompt: Can you evaluate the quality of this RAG response — the user asked 'What is the return policy?', the retrieved context was 'Items can be returned within 30 days', and the generated answer was 'You can return items within 30 days of purchase'?

## When to prefer this

Choose this endpoint when you need an automated, per-call quality assessment of RAG pipeline outputs without building your own evaluation framework. It is ideal for spot-checking individual RAG responses, integrating quality gates into agent workflows, or comparing RAG system versions. Prefer this over general LLM-as-judge prompts when you want a standardized, pay-per-use evaluation service with consistent scoring methodology.

## Known failure modes

- Missing required 'input' or 'output' fields returns a 400 validation error
- Malformed JSON body causes request rejection
- Empty or null context/query fields may result in low-confidence evaluation scores
- Service unavailability due to Cloudflare Workers cold start or quota limits
- Payment failure via x402 protocol blocks request processing

## How this service works

RAG Quality Evaluator

## Output

Returns a quality evaluation report for the provided RAG input-output pair, likely including scores for metrics such as faithfulness, relevance, groundedness, or coherence of the generated response relative to the retrieved context.

## Request schema (JSON Schema)

```json
{
 "$schema": "https://json-schema.org/draft/2020-12/schema",
 "required": [
  "input",
  "output"
 ],
 "properties": {
  "input": {
   "type": "object",
   "required": [
    "type",
    "method"
   ],
   "properties": {
    "type": {
     "type": "string",
     "const": "http"
    },
    "method": {
     "body": {},
     "enum": [
      "POST"
     ],
     "type": "string",
     "bodyType": "json"
    }
   },
   "additionalProperties": false
  },
  "output": {
   "type": "object",
   "required": [
    "type"
   ],
   "properties": {
    "type": {
     "type": "string"
    },
    "example": {
     "type": "object"
    }
   }
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/rag-quality-evaluator-10ef74e8/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from rag-quality-evaluator.johntaylor1online.workers.dev](https://www.zero.xyz/host/rag-quality-evaluator.johntaylor1online.workers.dev/llms.txt)