AI search fix
Is llms.txt used by ChatGPT?
OpenAI documents GPTBot and related user-agents for fetching public web content used in ChatGPT-class products. llms.txt is a community convention at /llms.txt; OpenAI does not publish a guarantee that ChatGPT always reads or prioritises it. Practical order: allow GPTBot in robots.txt and at your CDN, serve crawlable HTML, then publish optional llms.txt with accurate links. None of this guarantees citations in ChatGPT answers.
Teams often ask about llms.txt before fixing GPTBot blocks. Reverse the order: prove crawl in logs, then add a lean llms.txt map if you want extra orientation for parsers.
What to verify for ChatGPT-related crawl
- GPTBot and related strings allowed in production robots.txt.
- No 403 or challenge loops at Cloudflare for those user-agents.
- Public 200 HTML on pages you care about for answers.
- Optional /llms.txt returns 200 with current links only after the above.
Free · 2 minutes · no card
See what AI crawlers hit on your site
Technical blockers, missing context, weak AI-readiness signals — in one HTML report.
You'll get an HTML report on GPTBot access and whether /llms.txt is reachable.
Frequently asked questions
Does ChatGPT require llms.txt?
No documented requirement. Crawl access and helpful public content come first.
Is llms.txt the same as robots.txt for ChatGPT?
No. robots.txt controls fetch permission. llms.txt is an optional site summary file.
Will llms.txt force ChatGPT to cite my site?
No. Citations depend on relevance, trust, and OpenAI retrieval policies.
Related questions
Updated