# robots.txt for Agent-Ready Check (https://github.com/vibeprogrammer/agent-ready-check) # Content signals: https://contentsignals.org/ . Every group repeats them, because a crawler # reads only the group that matches it best (RFC 9309, section 2.2.1). # Everyone: read every page, but do not start scans by following links User-agent: * Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / Disallow: /scan Disallow: /api/scan Disallow: /r/ Disallow: /badge # AI search and training crawlers: welcome, same rules User-agent: OAI-SearchBot User-agent: Claude-SearchBot User-agent: PerplexityBot User-agent: Amzn-SearchBot User-agent: MistralAI-Index User-agent: GPTBot User-agent: ClaudeBot User-agent: Google-Extended User-agent: Applebot-Extended User-agent: CCBot User-agent: MistralAI-Training User-agent: Meta-ExternalAgent Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / Disallow: /scan Disallow: /api/scan Disallow: /r/ Disallow: /badge # Assistants fetching for a person right now: they may also open scan results User-agent: ChatGPT-User User-agent: Claude-User User-agent: Perplexity-User User-agent: Amzn-User User-agent: MistralAI-User Content-Signal: search=yes, ai-input=yes, ai-train=yes Allow: / Sitemap: https://isainative.dev/sitemap.xml Agentmap: https://isainative.dev/.well-known/ard.json