AI search fix
llms.txt missing — how to fix it
A missing /llms.txt at your domain root leaves many AI workflows without a standard site briefing — no quick summary of your brand, services, or priority URLs. Crawlers may still reach pages if robots.txt and your WAF permit them, but assistants must piece together structure from HTML alone. Google’s guidance for AI Overviews is clear: llms.txt is optional there; crawlability and helpful, non-commodity content matter most. For ChatGPT, Perplexity, Claude and similar tools, adding a lean markdown map at /llms.txt is still sensible GEO preparation once crawl access is sorted.
Think of llms.txt as a complement, not a substitute. It sits alongside robots.txt, Cloudflare allow rules, and strong on-page copy — never in place of them. Follow the llmstxt.org pattern: markdown headings, list links, no HTML tables or scripts. Refresh the file whenever pricing, routes, or core messaging shift so AI-facing summaries stay accurate.
How to add llms.txt when it is missing
- Open https://yourdomain.com/llms.txt and confirm it returns 404 or an empty response.
- Draft an H1 title, a one-line blockquote summary, and a short context paragraph.
- Add sections such as Docs, Services, and Optional with markdown list links.
- Serve the file at /llms.txt with Content-Type text/plain or text/markdown.
- Add an Updated date line and re-check after your next site deploy.
You'll get an HTML report showing whether llms.txt is present and whether robots.txt or WAF rules block AI crawlers.
Run the diagnosticRelated questions
Updated