In one week, AI crawlers hit our site 3,427 times. Three weeks earlier the same bots logged 604. That is a 5.7× swing between two windows three weeks apart — not a growth curve.
Most generative engine optimization statistics you will find are aggregated, undated, or copied from another roundup. This page is the opposite: exact request counts from our own Cloudflare logs, 15 named crawlers across four seven-day windows, with the raw file attached so you can check the arithmetic yourself.
These figures come from Cloudflare logs for getgeofix.com — one website, not an industry panel. We counted HTTP requests attributed to named AI crawler user-agents across four non-contiguous seven-day windows: 26 June – 2 July, 6–12 July, 15–21 July, and 28 July – 3 August 2026. Coverage is not continuous; there are gaps between the windows. Counts are exact — nothing rounded, nothing smoothed.
It is a single-site sample, and we would rather say so plainly than dress it up. What it shows is how AI crawler traffic behaves week to week on a real business site. What it cannot show is a market-wide average. Raw data, CC BY 4.0: CSV and JSON — 60 crawler-by-week observations, the same ones in the table below.
AI crawler requests: 15 bots across four weeks
Request counts by crawler and window on getgeofix.com:
Bot
26 Jun – 2 Jul
6–12 Jul
15–21 Jul
28 Jul – 3 Aug
OAI-SearchBot
159
2,254
57
87
ClaudeBot
179
261
252
244
ChatGPT-User
99
452
15
73
Bytespider
0
14
536
52
GPTBot
107
161
83
174
Amazonbot
2
89
66
8
PerplexityBot
36
46
8
58
meta-externalagent
3
55
44
40
Claude-User
15
3
15
18
anthropic
0
29
0
17
cohere
0
28
8
7
Applebot
0
21
0
21
CCBot
0
11
4
21
Google-Extended
0
1
0
17
Perplexity-User
4
2
0
8
Total
604
3,427
1,088
845
What the numbers actually show
Four patterns come out of that table, and none of them looks like a tidy “AI traffic grew N%” headline.
Spikes, not a line
The quietest window totalled 604 requests, the busiest 3,427 — about 5.7× higher. The two weeks after the peak fell back to 1,088 and 845. Average those four numbers and you get roughly 1,491 requests a week, a figure that describes none of the four weeks and hides the only interesting event in the sample.
OAI-SearchBot: down about 96% after its surge
OpenAI’s crawler documentation describes OAI-SearchBot as the agent that surfaces websites in ChatGPT’s search features. In our logs it went from 2,254 requests in 6–12 July to 87 in 28 July – 3 August — a fall of about 96% in three weeks. It was the least predictable crawler in the sample. On a monthly rollup, that entire story disappears.
ClaudeBot: the steady one
ClaudeBot turned up every single week at a similar level: 179, then 261, 252, 244. It was the most consistent major crawler here. Spiky bots make the headline; the steady one is the one you can actually plan around when you are reviewing logs or writing a security rule.
Bytespider: arrive, binge, fade
Bytespider logged nothing in the first window, 14 in the second, 536 in 15–21 July, then 52. It appeared from almost nowhere, worked one heavy week, and cooled off. Blend every bot into one “AI traffic” line and that week reads as a site-wide trend. It was one crawler having a busy week.
AI crawler activity on a real site looks like bot-by-bot spikes, not a smooth rising line. When a roundup says AI traffic grew by N%, ask which bots, which weeks, and whether the average erased the swing.
Which of these bots reach your site?
Our numbers are ours. The free Express Check gives you yours, in about a minute.
Crawls content that may be used in training OpenAI’s foundation models
Training data collection — not live ChatGPT search
OAI-SearchBot
Surfaces websites in search results in ChatGPT’s search features
Indexing for search-style answers
ChatGPT-User
Visits a page when a user asks ChatGPT or a CustomGPT a question
A person triggered this fetch — and OpenAI notes robots.txt rules may not apply
The third row is the one worth sitting with. ChatGPT-User is not a crawler working through a queue — it is a fetch that happened because a person asked the assistant something. On our site it ran 99, then 452, 15, and 73 across the four windows. GPTBot can be busy while ChatGPT-User is silent, or the other way round.
For the robots.txt patterns and the training-versus-search split, see our GPTBot guide. For why a clean SEO audit still misses these agents entirely, see AI crawlers SEO platforms miss.
What this data does not tell you
These logs measure reachability and crawl volume. They do not measure outcomes, and the gap between the two is where most published GEO statistics quietly overclaim.
What people read into crawler logs
What the logs actually support
“We are being cited by AI”
A page was fetched. Citation is a separate step.
“We rank in ChatGPT”
Nothing about ranking. Retrieval and ranking are not the same.
“AI traffic is up across the industry”
One site’s logs. This is not a benchmark.
“A spike means we won visibility”
It may be one crawler’s schedule, and often is.
“A quiet week means we lost visibility”
Crawl cadence drops; previously fetched pages do not vanish.
You do not need our volume. You need the same bot-level view of your own logs.
Name the bots in your logs — OAI-SearchBot, GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot and the rest, separately. A single “AI” filter would have hidden every pattern on this page.
Compare weeks, not only months. A 5.7× swing like ours only appears when you keep the windows apart.
Check access before you argue about content. If Cloudflare or robots.txt blocks the crawlers that matter, no content change fixes reach.
GEO Fix works on that readiness layer: the files and CMS steps that decide whether AI systems can reach and parse your site, not only a score that tells you they cannot. Crawl access still does not equal citations — it removes one common technical blocker on the way there.
We will update this dataset monthly. One-off snapshots get forgotten; a dated series is something people can build on.
Cite this dataset
GEO Fix, “Generative Engine Optimization Statistics”, August 2026. https://getgeofix.com/en/blog/generative-engine-optimization-statistics
Dataset: GEO Fix AI Crawler Dataset — Summer 2026. Raw files, CSV and JSON, published under CC BY 4.0 — reuse them, including commercially, with attribution.
FAQ
As a practical discipline, yes. Generative engine optimization is the work of making a business reachable and quotable inside AI answers rather than only in classic rankings. Crawler access is one technical piece of it, not the whole job. For the business-level definition, see what generative engine optimization means for business.
It depends entirely on the week and the bot. On our site the weekly total moved between 604 and 3,427 requests inside six weeks, then settled lower. A quiet week does not mean the assistants lost interest, and a busy one may be a single crawler surging. Check your own logs by user-agent and by week — a monthly figure will not show you either.
Per OpenAI’s documentation, GPTBot crawls content that may be used to train models, OAI-SearchBot surfaces sites in ChatGPT’s search features, and ChatGPT-User fetches a page when a person asks ChatGPT a question. Allowing or blocking one does not do the same for the others — see our GPTBot article.
No. A fetch means the page was reachable. Whether an engine then retrieves it for a given question, and whether it names or links you in the answer, are separate decisions the platform makes. Anyone selling crawler hits as proof of AI visibility is selling you step one as though it were step three.
Yes. The CSV and JSON are published under CC BY 4.0 — use them, including commercially, as long as you credit GEO Fix and link back. If you republish the table, keep the date windows attached to the numbers.
On getgeofix.com, weekly AI crawler volume swung 5.7× between the quietest and busiest window — spikes by bot, not an industry curve.
OAI-SearchBot fell about 96% in three weeks; ClaudeBot barely moved. Averaging them together erases both facts.
Treat GPTBot, OAI-SearchBot and ChatGPT-User as three signals, and remember robots.txt may not apply to the third.
Crawl volume is access data, not citation proof — quote it with its date window or it goes stale fast.
Free · 2 minutes · no card
See which of these crawlers can reach your site
The free Express Check tests whether AI systems can fetch your pages — crawler access, robots.txt, llms.txt and structure gaps. A clear HTML report by email in under a minute. No card, no account.
