GEO FixAI search readiness check

AI search fix

ClaudeBot blocked — how to fix it

When ClaudeBot is blocked, Anthropic's crawler cannot fetch your public HTML for Claude-related retrieval. Blocks typically come from robots.txt disallow rules, SEO plugin templates, or Cloudflare WAF challenges. Add an explicit Allow section for ClaudeBot in robots.txt, align CDN skip rules, and confirm HTTP 200 on priority URLs in logs. Unblocking improves crawl eligibility; it does not guarantee Claude will cite your site.

After fixes, compare CMS robots editor output with the live robots.txt browsers and crawlers fetch — staging and production often diverge.

Unblock ClaudeBot

  1. Review live robots.txt for disallow affecting ClaudeBot.
  2. Add User-agent: ClaudeBot with Allow for public marketing and docs paths.
  3. Cloudflare: skip Bot Fight and managed challenges for ClaudeBot user-agent.
  4. Re-test after Wordfence or security plugin updates.
  5. Verify repeated 200 responses in logs over 24–72 hours.

Free · 2 minutes · no card

See what AI crawlers hit on your site

Technical blockers, missing context, weak AI-readiness signals — in one HTML report.

No card and no payment for the free check. Prices for Starter and Pro exclude VAT as shown on the site. GEO Fix does not guarantee AI visibility or citations.

You'll get an HTML report on ClaudeBot access and robots.txt on your domain.

Frequently asked questions

Should I also allow anthropic-ai and Claude-Web?

Review Anthropic documentation. ClaudeBot is the primary crawl string for page fetching; other strings may differ.

Does unblocking ClaudeBot require llms.txt?

No. llms.txt is optional after crawl access works.

Will ClaudeBot appear in Google Analytics?

Usually not. Use server or CDN bot logs instead.

Related questions

Updated