Latitude 10 Agent Eval Kit

{
  "type": "json",
  "example": {
    "payment": {
      "network": "eip155:8453",
      "transaction": "0x…"
    },
    "product": {
      "name": "Latitude 10 Agent Eval Kit for Node.js",
      "slug": "agent-eval-kit",
      "content": "https://latitude10.tech/api/kit/download?t=…",
      "version": "1.0.3",
      "delivery": "kit",
      "description": "For Node.js: tests for what your agent does, not just what it says: a Vitest library and a CLI for TypeScript agents. Each run records a trajectory (model calls, tool calls with arguments and results, payments, tokens, cost, latency, outcome; the Agent Trace Kit format) and 25 matchers check it: which tools were called, in what order and with what arguments, cost, token, latency and step budgets, retries, payments, leaks, approvals and refusals. Mocked, failing, replayed and prompt-injected tools (a dated corpus of 24 original payloads) and a scripted model, so most suites run in CI with no API key. An LLM judge (Anthropic by default, or OpenAI or any compatible endpoint) with rubrics, votes and a cache; skipped loudly without a key. Trials with pass@k and pass^k, YAML suites, baselines that fail CI on regressions, JUnit and a job summary, and agent-eval import to turn a Trace Kit session into a regression test. Recording adapters for the Anthropic SDK, Claude Agent SDK, OpenAI SDK, OpenAI Agents SDK, Vercel AI SDK and MCP. Includes a setup wizard (node setup.mjs) running 85 example cases in seven folders (tool selection, hallucination, permissions, retries, failures, prompt injection, regression), agent-eval init for an existing project, and a GitHub Actions workflow.",
      "contentExpiresAt": "2026-10-02T12:00:00.000Z"
    }
  }
}