Web Domain Crawler with Keyword Scanning

Use this endpoint when you need to programmatically discover the internal link structure or page inventory of a website, or when you want to scan multiple pages of a site for a specific keyword or signal. It is ideal for lightweight site audits, competitive research, content discovery, or verifying whether a topic is covered anywhere on a domain — all within a single pay-per-call API with no authentication setup required.

• URL is unreachable or returns non-200 status — crawl returns empty or partial results
• Domain blocks crawlers via robots.txt or rate limiting — fewer pages returned than requested
• limit parameter out of range (must be 1–5) — may return error or default behavior
• No internal links found — returns empty link array with only the starting page
• Keyword not found on any page — returns results with empty keyword match list