# Netzhandwerker A/B Test Evaluator

> Netzhandwerker A/B Test Evaluator is a paid API for AI agents from tools.halowerk.com, paid per call via x402, $0.002/call, status unknown (last checked 2026-10-12).

Analyzes a two-group conversion experiment using both frequentist (p-value, confidence interval, required sample size) and Bayesian (Beta-Binomial posteriors, probability variant wins, expected loss) methods in closed form.

## Facts

- Endpoint: POST https://tools.halowerk.com/v1/abtest/evaluate?utm_source=zero.xyz
- Price: $0.002/call
- Payment: x402
- Status: unknown
- Last checked: 2026-10-12
- Activations on Zero: 0
- Tags: x402
- Canonical page: https://www.zero.xyz/c/netzhandwerker-a-b-test-evaluator-0a8c97b8
- Structured record (JSON): https://api.zero.xyz/v1/capabilities/cap_YsQTnz8uurd42_zTPl4dI

Status and success rate cover calls made through Zero and Zero's own probes. Third-party monitors may report differently.

## How to call it through Zero

Zero handles the 402 payment challenge and records the run. With the Zero CLI installed (`npm i -g @zeroxyz/cli`):

```sh
zero fetch --capability netzhandwerker-a-b-test-evaluator-0a8c97b8 -d '<json body>'
```

Example prompt: Analyze my A/B test: the control had 4200 visitors and 312 conversions, the variant had 4180 visitors and 351 conversions. Use a 95% confidence level, 80% power, and a minimum detectable effect of 10% relative lift — give me both the frequentist p-value and the Bayesian probability the variant is better.

## When to prefer this

Choose this endpoint when you need a single call that returns both frequentist and Bayesian statistics for a binary conversion experiment without sampling variance — the closed-form Bayesian computation means results are deterministic and reproducible. Especially valuable when you have peeked at results multiple times and need the Pocock-style adjusted threshold via looks_taken. Prefer this over general statistics libraries when you need all key metrics (p-value, CI, sample size, posterior probability, expected loss, credible intervals) in one structured response for an agent workflow.

## Known failure modes

- conversions exceed visitors for either group — returns validation error
- confidence_level or power outside allowed range (0.5–0.9999 / 0.5–0.999) — returns 422
- looks_taken above 100 — returns validation error
- minimum_detectable_effect_rel outside 0.001–5 range — returns error
- payment not provided or insufficient — returns 402 Payment Required
- zero visitors in control or variant — returns 422 minimum constraint error

## How this service works

Reads one conversion experiment two ways at once. Frequentist: two-proportion test, p value, confidence interval on the absolute lift, and the sample size the requested minimum detectable effect needs at the requested power. Bayesian: Beta-Binomial posteriors, probability the variant is better, expected loss either way, credible intervals - closed form, not sampled, so the numbers repeat exactly. looks_taken carries the adjusted threshold and the rate you actually ran at.

## Output

Returns frequentist results (p-value, absolute lift confidence interval, required sample size for the specified MDE and power) and Bayesian results (probability variant beats control, expected loss for choosing either arm, credible intervals for both rates) computed in closed form. Also returns the adjusted significance threshold and actual false positive rate when multiple interim looks have been taken.

## Request schema (JSON Schema)

```json
{
 "type": "object",
 "properties": {
  "power": {
   "type": "number",
   "default": 0.8,
   "maximum": 0.999,
   "minimum": 0.5,
   "description": "Probability of finding that effect if it is there."
  },
  "control": {
   "type": "object",
   "required": [
    "visitors",
    "conversions"
   ],
   "properties": {
    "visitors": {
     "type": "integer",
     "minimum": 1
    },
    "conversions": {
     "type": "integer",
     "minimum": 0
    }
   },
   "description": "Baseline group. conversions may not exceed visitors."
  },
  "variant": {
   "type": "object",
   "required": [
    "visitors",
    "conversions"
   ],
   "properties": {
    "visitors": {
     "type": "integer",
     "minimum": 1
    },
    "conversions": {
     "type": "integer",
     "minimum": 0
    }
   },
   "description": "Group under test. conversions may not exceed visitors."
  },
  "prior_beta": {
   "type": "number",
   "default": 1,
   "maximum": 1000,
   "minimum": 0.0001,
   "description": "Beta prior, failures."
  },
  "alternative": {
   "enum": [
    "two_sided",
    "greater"
   ],
   "type": "string",
   "default": "two_sided",
   "description": "two_sided asks whether the groups differ, greater whether the variant is above the control."
  },
  "looks_taken": {
   "type": "integer",
   "default": 1,
   "maximum": 100,
   "minimum": 1,
   "description": "How often the running test has been evaluated. Above 1 the threshold is adjusted and the real false positive rate is reported."
  },
  "prior_alpha": {
   "type": "number",
   "default": 1,
   "maximum": 1000,
   "minimum": 0.0001,
   "description": "Beta prior, successes. 1 with prior_beta 1 is uniform."
  },
  "confidence_level": {
   "type": "number",
   "default": 0.95,
   "maximum": 0.9999,
   "minimum": 0.5
  },
  "minimum_detectable_effect_rel": {
   "type": "number",
   "default": 0.1,
   "maximum": 5,
   "minimum": 0.001,
   "description": "Relative effect the test should be able to find, e.g. 0.1 for ten percent above the control rate."
  }
 }
}
```

## More

- Live health (JSON, refreshed every minute): https://www.zero.xyz/c/netzhandwerker-a-b-test-evaluator-0a8c97b8/health.json
- [Zero catalog index](https://www.zero.xyz/llms.txt)
- [Other services from tools.halowerk.com](https://www.zero.xyz/host/tools.halowerk.com/llms.txt)