Should llms.txt directory include all blog posts?

Usually no. Include cornerstone pages and key resources, not every URL.

Is llms.txt directory required by Google?

No. Google does not require llms.txt for AI features in Search.

Can a directory improve AI visibility by itself?

Not by itself. Crawl access, page quality, and trust signals remain primary.

AI search fix

llms.txt directory: what to include and how to organize it

An llms.txt directory should be a compact, human-readable index of priority public pages, grouped by sections like Docs, Services, and Policies. It is not a replacement for sitemap.xml and should not include every URL on your site. The goal is to help AI systems orient quickly after crawl access works. Keep links current and remove stale routes. A good directory improves clarity, but it does not guarantee citations.

The most common mistake is overloading llms.txt with hundreds of links. A smaller, curated directory is usually more useful for both maintenance and machine parsing.

Build an llms.txt directory in five steps

Define 3-5 sections that reflect how users navigate your site.
List only priority public URLs that stay stable over time.
Use short labels and avoid tracking parameters in links.
Validate /llms.txt returns 200 and plain text or markdown.
Review monthly and remove stale links after site updates.

You'll get an HTML report on /llms.txt availability and crawler access on your domain.

Run the free check

Updated 2026-05-28

llms.txt directory: what to include and how to organize it

Build an llms.txt directory in five steps

Related questions