How to set up llms.txt for Claude (ClaudeBot crawl first)

Setting up llms.txt for Claude starts with crawl access: ClaudeBot must be allowed in robots.txt and not blocked by Cloudflare or Wordfence rules. Then publish a lean /llms.txt using the llmstxt.org layout with links to your priority public pages. Anthropic documents ClaudeBot as a crawler; llms.txt is optional orientation for AI systems — not a switch that forces Claude to cite you. Google does not require llms.txt for AI Overviews either. Verify with server logs or Bot Activity after fixes.

If Claude cannot fetch your homepage, fix claudebot-blocked issues before editing llms.txt. Keep the file aligned with what you show human visitors — no keyword stuffing or hidden URLs.