AI search fix
Bingbot and Copilot: how Microsoft AI crawls your site
Bingbot is Microsoft's crawler user-agent for Bing Search indexing — the crawl layer behind many Microsoft AI experiences. Copilot and other Bing-powered assistants ground answers on web content discovered and indexed through that pipeline, plus freshness signals when pages change. If Bingbot is blocked in robots.txt or halted at Cloudflare, you restrict both classic Bing visibility and the pages Microsoft systems can retrieve for AI answers. Permitting Bingbot does not guarantee Copilot will cite your site or send AI-referred traffic.
Once Bingbot is permitted, push meaningful URL updates via IndexNow so Bing and partner engines pick up fresh pages sooner. Review Bing Webmaster Tools AI Performance for citation trends — but only after crawl and indexing look healthy.
Microsoft AI crawl stack (simplified)
- Bingbot — primary Search crawler; look for User-Agent containing Bingbot in logs.
- Indexing — pages enter Bing's index when crawl, quality and policy permit.
- Grounding — Copilot-class products retrieve web content before generating answers.
- Citations — optional measurement in Bing WMT AI Performance or Clarity Citations.
- GEO Fix focus — robots.txt, CDN/WAF, schema and answer-first copy before chasing citation KPIs.
You'll get an HTML report on Bingbot and other AI user-agents in robots.txt and at your edge.
Run the diagnosticRelated questions
- What is AI search visibility (and how to measure it)Crawl, grounding, citations and visits — without guarantee claims.
- robots.txt for AI crawlers — how to write itPermitting Bingbot alongside GPTBot, ClaudeBot and others.
- How to optimise your website for AI searchIndexNow, schema and copy patterns for Microsoft and other channels.
Updated