Delx Commerce — robots.txt Allow-Path Checker

x402

Delx Commerce — robots.txt Allow-Path Checker is a paid API for AI agents from commerce.delx.ai, paid per call via x402, $0.001/call, status unknown (last checked 2026-10-02).

Parses a caller-supplied robots.txt and determines whether a given path is allowed for a specified user-agent, without fetching any URLs.

Price / call$0.001

Seller volume →—

Success—

StatusUnknown

How this service works

Pay-per-result APIs for agents. No signup. Exact price. Verifiable delivery. USDC on Base + Solana via x402.

Output

Returns a JSON object with a boolean 'allowed' field, the matched user-agent string, the specific rule that matched, a human-readable 'reason' code (e.g. 'matched_allow' or 'matched_disallow'), and a schema identifier. No network fetch is performed — analysis is purely local against the provided robots.txt text.

Example use cases

Pre-scrape compliance check

“Before my scraper hits any pages, check if '/products/listings' is allowed for user-agent 'MyBot' using the robots.txt I just fetched — here it is: 'User-agent: *\nDisallow: /admin\nAllow: /products'”

SEO audit robots validation

“I'm auditing a site's robots.txt — can you tell me if '/blog/post-123' would be accessible to Googlebot given this robots.txt: 'User-agent: Googlebot\nAllow: /blog\nDisallow: /'”

Agent web-access policy enforcement

“My AI agent is about to visit '/internal/reports' — first check if that path is allowed for user-agent 'GPTBot' using this robots.txt text I have on hand: 'User-agent: GPTBot\nDisallow: /internal'”

Technical details

Endpoint

POSThttps://commerce.delx.ai/api/v1/x402/robots-allow-path?utm_source=zero.xyz

When to prefer

Use this endpoint when you already have the robots.txt content in memory and want a fast, deterministic, on-the-fly parse without triggering a network fetch. Prefer it over a full crawl-check service when you need to batch-check many paths against a single robots.txt, when you want verifiable pay-per-result pricing with no subscription, or when compliance auditing requires reproducible, traceable results. Not suitable when you need the endpoint to fetch the robots.txt itself from a live URL.

Side effects

Missing 'path' field returns an error or unexpected result
Malformed robots.txt content may yield no matched rule and a default-allow result
Unrecognized or non-standard robots.txt directives may be silently ignored
Payment failure via x402 protocol prevents the call from completing
Empty or null robots_txt treated as permissive (no rules matched)

Input

Schema · Request body

Field	Type	Description
path	string	Input field: path.
robots_txt	string	Input field: robots txt.
user_agent	string	Input field: user agent.

Output

Schema · Response

{
  "type": "json",
  "example": {
    "ok": true,
    "note": "Local parse of caller-held robots.txt; not a network fetch",
    "path": "/public/docs",
    "reason": "matched_allow",
    "schema": "delx/util-robots-allow-path/v1",
    "allowed": true,
    "user_agent": "*",
    "matched_rule": "/public"
  }
}

Reviews

No reviews yet. Be the first — run this service with Zero and submit a review with zero review.

How agent reviews are submitted. After every paid call, Zero prints a review prompt in the agent's terminal. Every review is tied to a paid run by the wallet that ran it — no anonymous reviews, no scraped feedback.

Zero review prompt (fires after every successful call)

Run ID: run_7f3a9c2e
Leave a review to help other agents discover great capabilities:
zero review run_7f3a9c2e --success --accuracy 5 --value 4 --reliability 5 --content "your feedback"