AI search fix
Website not visible in AI search on WordPress, Shopify, Wix or Webflow — how to fix it
A website built on WordPress, Shopify, Wix or Webflow can rank well on Google and still be invisible to AI search, because each platform blocks AI crawlers its own way: WordPress through security plugins and stacked robots.txt overrides, Shopify through a platform-managed robots.txt and third-party apps, Wix and Webflow through staging domains and limited robots.txt control, and WooCommerce through the WordPress layer plus missing Product schema. The fix starts the same way on every platform — confirm the live production robots.txt and any CDN or WAF allow the AI crawlers you want, then add accurate structured data on the pages you want cited. None of the steps below guarantee AI citations or sales; they remove technical and structural barriers you control directly.
Treat platform defaults as something to audit, not a policy you chose: most CMS and app defaults were written before AI crawlers existed, so check them explicitly rather than assuming they already welcome GPTBot, PerplexityBot or ClaudeBot.
Where AI visibility usually breaks, by platform
| Platform | Where AI visibility usually breaks |
|---|---|
| WordPress | Security plugins (Wordfence and similar) and stacked SEO-plugin robots.txt overrides that block AI user-agents even when Googlebot works. |
| WooCommerce | The WordPress layer above, plus missing Product schema and thin or duplicate descriptions on category and faceted URLs. |
| Shopify | A platform-managed robots.txt you cannot fully edit, plus security or SEO apps that block "bad bots" broadly. |
| Wix | Password protection or an unpublished page state, limited robots.txt control, and widgets that hide copy from HTML. |
| Webflow | Indexed staging (webflow.io) subdomains, custom robots.txt edits, and JSON-LD never added to page or site custom code. |
Free · 2 minutes · no card
See what AI crawlers hit on your site
Technical blockers, missing context, weak AI-readiness signals — in one HTML report.
You'll get an HTML report showing which AI crawlers are blocked on your CMS — WordPress, Shopify, Wix, Webflow or WooCommerce.
WordPress and WooCommerce: plugins and stacked robots.txt
WordPress sites frequently block AI crawlers through security plugins, cached or virtual robots.txt overrides, or Cloudflare rules copied from generic hardening guides — GPTBot, PerplexityBot and ClaudeBot can receive 403 responses even when Googlebot works normally. Check both the Settings → Reading "discourage search engines" flag and your SEO plugin's robots editor (Yoast, Rank Math and similar): they stack, and either one alone can silently override the other.
WooCommerce inherits every WordPress blocking issue and adds product-specific gaps: missing Product schema, duplicate manufacturer-boilerplate descriptions, and faceted URLs that pollute crawl budget while core products stay thin. Fix crawler access on the WordPress layer first, then add accurate Product JSON-LD with visible price and availability on top-selling SKUs — AI assistants cannot reliably recommend a store when GPTBot is blocked or product pages lack clear facts in HTML.
Shopify: a platform-managed robots.txt and blocking apps
Shopify serves a platform-managed robots.txt you cannot fully rewrite, and third-party SEO or security apps often add their own disallow rules on top of it — open https://yourstore.com/robots.txt and read the live output rather than a template editor. AI crawlers such as GPTBot and PerplexityBot also need explicit allow policy at the edge if you run Cloudflare in front of the store, since app-level blocks and CDN rules act independently.
Disable or reconfigure any app that blocks "bad bots" broadly, confirm product and collection pages return a public 200 response without a storefront password, and add Product and Organization JSON-LD through your theme or an app where the data is accurate. Shopify Markets and multi-locale setups can also confuse bots with unexpected redirects — test AI crawler access per public market domain, not only the default storefront.
Wix and Webflow: page builders limit robots.txt control
Wix and Webflow both limit how much of robots.txt you can customize compared with a self-hosted CMS, and both have a platform-specific false lead: on Wix it is password protection or a page still marked "not published"; on Webflow it is an indexed webflow.io staging subdomain sitting alongside the production domain. Confirm the site is published publicly on its real production domain before touching bot policy at all.
On Wix, open the Wix SEO settings and allow indexing on money pages, review the Wix-generated robots.txt for unexpected disallows, and add structured data through the Wix SEO panel where supported. On Webflow, set the staging subdomain to noindex, edit robots.txt only with explicit allow rules for AI bots you trust, and embed JSON-LD through page or site-wide custom code — hidden tabs and animations should not be the only place core copy lives, since bots read the static HTML.
Every platform: align Cloudflare with whatever the CMS serves
If you front any of these platforms with Cloudflare or another CDN, mirror the same AI bot allow rules you configured in the CMS — a request can pass the platform's own robots.txt and still be challenged at the edge before it gets that far. Cloudflare's Bot Fight Mode and Managed Rules bundle AI crawlers into generic automation heuristics by default on every platform equally, so an allow rule written for WordPress does not carry over to a Shopify or Webflow domain on the same account.
Structured data and citable copy only help once crawl access works — fix the platform's robots.txt and any CDN block first, then invest in schema and clearer product or service descriptions on the pages you most want AI search to notice.
Frequently asked questions
Can Wix edit robots.txt freely for AI bots?
No. Wix provides more limited robots control than WordPress. Use the available SEO settings and support docs for your plan, and verify the live output rather than assuming a setting applied.
Does Webflow block AI crawlers by default?
Not usually, but custom robots.txt edits, password-protected pages, or Cloudflare in front of Webflow can block them — check all three before assuming Webflow itself is the cause.
Does WooCommerce include Product schema by default?
Some themes and SEO plugins add it, but many stores need explicit configuration. Verify the live HTML source on product URLs rather than assuming the theme handles it.
Does disallowing /wp-admin/ block AI crawlers on WordPress?
No. Disallowing /wp-admin/ is normal and should not block public posts. A broad Disallow: / rule elsewhere in robots.txt is what actually blocks AI crawlers from content.
Can I upload a fully custom robots.txt on Shopify?
Not entirely — Shopify controls the base file. Some rules are editable through apps or Shopify Search & Discovery settings; verify what your plan allows on the live URL.
Will Shopify Markets or a Webflow staging subdomain confuse AI bots?
They can. Multiple market domains or an indexed staging subdomain give bots inconsistent signals — test AI crawler access on the exact public domain buyers and bots should use.
Can caching or security plugins and apps hide bots from logs?
Yes. Misconfigured cache, firewall, or bot-blocking apps can return challenges or hide requests from origin logs — check edge or app-level logs as well as the server.
Does fixing crawler access guarantee AI citations or sales?
No. Unblocking crawlers and adding accurate schema only remove technical and structural barriers. Whether a platform cites or recommends your pages still depends on content quality and each platform's own policies.
Related questions
- AI crawlers blocked — how to fix GPTBot, PerplexityBot, ClaudeBot and Google-ExtendedDiagnose robots.txt and CDN/WAF blocks per bot, plus the Google-Extended policy decision.
- GEO checklist for e-commerce storesProduct crawl, Product schema, platform robots limits, and AI bots.
- Schema markup for AI search: Organization, Product, FAQPage and BreadcrumbJSON-LD reference and validation steps for the schema types AI parsers read most.
Updated