AI search fix
How to publish llms.txt on WordPress, Wix, Webflow and Docusaurus
To publish llms.txt on a CMS, create a plain-text or markdown file, serve it at the root path https://yourdomain.com/llms.txt with a 200 response, and confirm trusted AI crawlers are not blocked by your platform's security settings first. The exact steps differ by stack — WordPress fights with security plugins, Wix and Webflow limit robots.txt control, and static-site generators like Docusaurus publish it as a build asset — but the order is the same everywhere: crawl access, then the file, then verification.
Shopify has its own dedicated guide (see related links) because it has real, measured search demand — this page covers the platforms where the fix pattern is close enough to share one page.
llms.txt on each platform: where the file goes
| Platform | How to publish |
|---|---|
| WordPress | Root-level file via theme, plugin or hosting panel; verify against SEO/security plugin overrides. |
| Wix | Publish through available routing options; platform limits robots.txt control more than other stacks. |
| Webflow | Page or site-wide custom code, published on the production domain — not the webflow.io staging host. |
| Docusaurus | Static file in the assets path, served from root after build and deploy. |
| Any platform | ClaudeBot and other AI bots must already be allowed in robots.txt before the file matters. |
Free · 2 minutes · no card
See what AI crawlers hit on your site
Technical blockers, missing context, weak AI-readiness signals — in one HTML report.
You'll get an HTML report on /llms.txt reachability and whether AI crawlers are blocked on your domain.
Crawl access comes before the file, on every platform
Publishing llms.txt before AI crawlers can reach your pages is a wasted step, regardless of CMS. Check robots.txt and your CDN or security plugin for the bots you want to allow — ClaudeBot is a useful example: if Anthropic's crawler cannot fetch your homepage, fix that first, because llms.txt only orients crawlers that can already read your site. Once access is confirmed in server logs, publish a lean file that matches what you show human visitors — no keyword stuffing, no hidden URLs.
WordPress: plugins and security layers often fight each other
WordPress sites frequently publish a valid llms.txt while Wordfence, Cloudflare, or an SEO plugin's robots editor still blocks the AI bots the file is meant to help. Confirm "discourage search engines" is off in Settings → Reading, review your security plugin's bot-blocking list, and fetch https://yourdomain.com/robots.txt directly rather than trusting the CMS dashboard — cached or CDN-served robots.txt can differ from what the editor shows. SEO plugins can also auto-generate an llms.txt-shaped file; treat that output as a first draft to verify, not a finished file, since a generator has no way to know which of your pages you actually want AI systems to prioritise.
Wix and Webflow: platform limits and staging domains
Wix offers more limited robots.txt control than self-hosted platforms, so verify the rendered file on your live domain rather than assuming editor settings match production output; publishing llms.txt itself usually needs a custom route or a Velo function if your plan supports one. Webflow's common failure mode is different: a webflow.io staging subdomain stays indexed and public while the real production domain has a stricter policy, so crawlers and reviewers end up looking at the wrong host. On both platforms, add JSON-LD and llms.txt only after confirming the production domain — not staging or preview — returns public 200 HTML.
Docusaurus and other static-site generators
On Docusaurus and similar static-site stacks, add llms.txt as a static asset so the build places it at the site root, then verify the production URL after deploy rather than trusting a local build. Keep links pointed at stable, canonical documentation URLs — versioned docs routes can silently break llms.txt links across a version bump, so re-check after every release.
Verify after every deploy, not just once
The same five checks apply across every platform above, in this order: request https://yourdomain.com/llms.txt directly and confirm a 200 response with no login wall or redirect loop; confirm the body is readable markdown with a real H1, one-line summary and working links; click a sample of the linked URLs and fix anything broken or redirected off-domain; compare the file's Updated date against your last pricing, product or route change; and re-run all four checks after any plugin, theme or CMS update, since those are the most common source of a file that quietly stops resolving or a robots.txt rule that quietly reappears.
Frequently asked questions
Does llms.txt replace robots.txt on any of these platforms?
No, on every platform. robots.txt controls crawl access; llms.txt only adds orientation once bots can already reach your pages.
Can my CMS or an SEO plugin auto-generate llms.txt?
Some can, but always verify the live output and headers. A short, hand-checked file is easier to keep accurate than an unreviewed auto-generated one.
Do I need llms.txt on every platform I publish to?
No. It is optional GEO preparation. Prioritize the platform where crawl access is confirmed working and content is strongest.
Will publishing llms.txt on my CMS guarantee AI citations?
No. It can help some AI systems orient after crawl access works, but citations still depend on content quality and each platform's policies.
Do versioned docs break llms.txt links on Docusaurus?
They can. Point llms.txt at stable, canonical URLs rather than a specific version path, and re-check links after each documentation release.
Is llms.txt required for ClaudeBot or any other AI crawler?
No platform requires it, and none guarantees the file will be read every time it is fetched — llms.txt is optional GEO preparation, not a switch that forces a citation. ClaudeBot and other trusted crawlers must already be able to reach your pages; the file only helps orientation once that access already works.
How do I know a bot like ClaudeBot actually reached my site?
Check server or CDN access logs for the exact user-agent string (ClaudeBot, GPTBot, PerplexityBot and so on), or a bot-activity dashboard if your CDN provides one. Express Check also flags common robots.txt blocks on your domain without needing raw log access.
Related questions
Updated