AI search fix
llms.txt missing — how to fix it
When llms.txt is missing at your domain root, many AI workflows lack a standard short briefing about your brand, services, and priority URLs. Crawlers may still fetch pages if robots.txt and your WAF allow them, but assistants have to infer structure from HTML alone. Google's guidance for generative AI features in Search says you do not need llms.txt or other special AI files there — crawlability and helpful, non-commodity content come first. For ChatGPT, Perplexity, Claude and similar systems, publishing /llms.txt is still a practical GEO signal: a markdown map with a one-line summary, brief context, and linked sections for your most important pages.
llms.txt does not replace robots.txt, Cloudflare allow rules, or strong on-page content. It complements technical access with a readable site map some LLM pipelines use. Keep the file lightweight: markdown headings and list links per llmstxt.org, without HTML tables or embedded scripts. Regenerate it when pricing, routes, or core messaging change so AI-facing summaries stay current.
How to add llms.txt when it is missing
- Open https://yourdomain.com/llms.txt and confirm it returns 404 or an empty response.
- Draft an H1 title, a one-line blockquote summary, and a short context paragraph.
- Add sections such as Docs, Services, and Optional with markdown list links.
- Serve the file at /llms.txt with Content-Type text/plain or text/markdown.
- Add an Updated date line and re-check after your next site deploy.
You'll get an HTML report showing whether llms.txt is present and whether robots.txt or WAF rules block AI crawlers.
Run Express CheckRelated questions
Updated