Robots.txt Generator
Generate a valid robots.txt file: a default rule group for all crawlers, an optional rule group for one specific bot (Googlebot, GPTBot, etc.), Crawl-delay, and one or more Sitemap URLs. Produces spec-correct output ready to copy or download.
Authorizations
Bearer authentication header of the form Bearer <token>, where <token> is your auth token.
Body
Header Comment (optional)
Disallow paths
Allow paths (optional)
Crawl-delay (seconds, optional)
x >= 0User-agent
, Googlebot, Googlebot-Image, Bingbot, YandexBot, DuckDuckBot, Baiduspider, GPTBot, ChatGPT-User, Google-Extended, CCBot, anthropic-ai, ClaudeBot, PerplexityBot, Twitterbot, AhrefsBot, SemrushBot, Applebot, __custom__ Custom user-agent
Disallow paths
Allow paths (optional)
Crawl-delay (seconds, optional)
x >= 0Sitemap URL(s)
Response
Tool output
The tool's slug, echoing the {slug} in the request path.
Output-contract version for this tool.
Credits this call consumed, after any settlement refund. 0 when metering is disabled.
Credits left in the current monthly allowance, or null when metering is disabled.
Correlation id, also sent as x-request-id.