AI search fix
OAI-SearchBot: what it is and how to allow it
OAI-SearchBot is an OpenAI user-agent used when OpenAI systems fetch public web pages for search-oriented product flows — separate from GPTBot (broad ChatGPT retrieval crawl) and ChatGPT-User (often tied to user-initiated browsing features). You may see OAI-SearchBot in server logs, CDN bot dashboards, or Microsoft Clarity Bot Activity. To permit it, add an explicit Allow in robots.txt and ensure your WAF does not challenge that string. Blocking OAI-SearchBot while allowing GPTBot is a valid policy choice — list each user-agent on its own line. Allowing OAI-SearchBot does not guarantee ChatGPT search or browse features will cite your site.
When auditing OpenAI access, export robots.txt and log filters for all three strings: GPTBot, OAI-SearchBot, and ChatGPT-User. A single Disallow: * under User-agent: * can override narrower allows if your CMS merges rules incorrectly.
| User-agent | Typical role |
|---|---|
| GPTBot | Crawler for ChatGPT-class retrieval and indexing of public pages. |
| OAI-SearchBot | Search-oriented fetch flows — separate robots.txt control. |
| ChatGPT-User | May appear for user-driven browsing features — policy separately. |
| GEO prerequisite | Reachable HTML, honest copy, schema.org JSON-LD where appropriate. |
Free · 2 minutes · no card
See what AI crawlers hit on your site
Technical blockers, missing context, weak AI-readiness signals — in one HTML report.
You'll get an HTML report listing GPTBot, OAI-SearchBot, and related OpenAI strings in robots.txt.
Frequently asked questions
Do I need to allow both GPTBot and OAI-SearchBot?
Only if your policy allows all OpenAI fetch types you care about. Many sites allow GPTBot and OAI-SearchBot together; others allow one and block the other — document the choice in your robots.txt.
Is OAI-SearchBot the same as ChatGPT-User?
No. They are different user-agent strings for different product behaviors. Review OpenAI's current publisher documentation before changing allow rules.
Will OAI-SearchBot show in Google Analytics?
Usually not as normal browser sessions. Use origin or CDN logs, Clarity Bot Activity, or Express Check to confirm the bot can reach your pages.
Related questions
- GPTBot user-agent: what it is and how to allow itSibling OpenAI crawler — compare policy before merging rules.
- ChatGPT cannot crawl my website — how to fix itEnd-to-end fixes when OpenAI bots never reach HTML.
- robots.txt for AI crawlers — how to write itTemplate sections for OpenAI and other AI user-agents.
Updated