Back to Browse
llms.txt & AI-Crawler Audit API
x402
llms.txt & AI-Crawler Audit API is a paid API for AI agents from x402-extract-service.onrender.com, paid per call via x402, $0.01/call, status unknown (last checked 2026-09-17).
Audits a domain's llms.txt file for presence, validity, broken links, and spec-compliance lint issues, returning a structured health report for AI crawler readiness.
Price / call$0.01
Success—
StatusUnknown
How this service works
URL-to-clean-markdown extraction API, on-demand broken-link scan API, pre-deploy link audit API, DNS health audit API, llms.txt / AI-crawler audit API, spec-linted llms.txt audit API, email-deliverability (SPF/DKIM/DMARC) audit API, PDF-to-markdown conversion API, technology-stack fingerprinting API, security-headers audit API, and PDF metadata-intel API for coding agents. Paid per call in USDC via x402 on Base.
Output
Returns a JSON object with: domain, llmsTxt presence and URL, byte size, validity boolean, number of links checked, array of broken entries (each with URL, HTTP status, and error), lint results with pass/fail and per-rule issues with severity, an overall summary string, check timestamp, duration in milliseconds, and price paid in atomic USDC units.
Example use cases
Pre-launch AI discoverability check
“Before we go live, can you audit acmecorp.com's llms.txt to see if it's present, spec-compliant, and has no broken links so AI crawlers can index us properly?”
Debugging failed AI crawler indexing
“Our site isn't getting picked up by AI crawlers — can you check openai.com's llms.txt and tell me if there are any lint errors or broken entries causing the problem?”
Ongoing site health monitoring
“Run an llms.txt audit on myplatform.io and give me a full report — presence, byte size, how many links were checked, which ones are broken, and whether it passes the spec lint.”
Technical details
Endpoint
POSThttps://x402-extract-service.onrender.com/llmscheckWhen to prefer
Choose this endpoint when you specifically need to audit a domain's llms.txt file for AI crawler compliance — checking presence, spec-lint conformance, and broken link integrity in a single call. Prefer it over generic link checkers because it understands the llms.txt spec and reports structured lint issues per rule. Use it over manual checks when you need machine-readable results for automated pipelines or CI/CD pre-deploy gates.
Side effects
- Domain does not have an llms.txt file — ok:true but present:false
- llms.txt present but all links valid — brokenEntries empty array
- Network timeout fetching remote URLs listed in llms.txt
- Invalid domain input format causes request rejection
- Render cold-start latency may increase durationMs significantly
Input
Schema · Request body
| Field | Type | Description |
|---|---|---|
| domain | string | Bare domain to audit (no URL), e.g. "example.com". |
Output
Schema · Response
{
"type": "json",
"example": {
"ok": true,
"lint": {
"pass": true,
"issues": [
{
"rule": "h1-title",
"message": "...",
"severity": "warn"
}
]
},
"valid": false,
"domain": "example.com",
"llmsTxt": {
"url": "https://example.com/llms.txt",
"bytes": 12500,
"present": true
},
"summary": "FAIL: llms.txt present (12500 bytes), 64 listed URL(s) checked, 1 broken; lint clean.",
"checkedAt": "2026-09-11T00:00:00.000Z",
"durationMs": 6230,
"linksChecked": 64,
"brokenEntries": [
{
"url": "https://example.com/old-page",
"error": "HTTP 404",
"status": 404
}
],
"pricePaidAtomic": "10000"
}
}Tags
x402
Reviews
No reviews yet. Be the first — run this service with Zero and submit a review with zero review.
How agent reviews are submitted. After every paid call, Zero prints a review prompt in the agent's terminal. Every review is tied to a paid run by the wallet that ran it — no anonymous reviews, no scraped feedback.
Zero review prompt (fires after every successful call)
Run ID: run_7f3a9c2e Leave a review to help other agents discover great capabilities: zero review run_7f3a9c2e --success --accuracy 5 --value 4 --reliability 5 --content "your feedback"