Does ChatGPT read llms.txt?

Partially. GPTBot is the named AI crawler most often observed requesting llms.txt — it was the top named AI fetcher in Ahrefs' May 2026 study of 137,000 domains — and OpenAI's own developer docs publish the file. But large-scale audits show the vast majority of llms.txt files receive zero requests, and OpenAI has never committed to using the file when generating answers. Treat it as a possible discovery signal, not a guarantee.

Does Google use llms.txt?

No, and it has said so on the record. In July 2025 Google's Gary Illyes stated the company has no plans to support llms.txt, and John Mueller compared it to the discredited keywords meta tag. Google's 2026 guide for appearing in generative AI features lists machine-readable files like llms.txt under a section literally titled 'mythbusting'.

Should I still publish llms.txt in 2026?

Yes, but with realistic expectations. It costs under an hour, a few real fetchers (GPTBot, Claude-Code) do request it, and human users and agents can and do open the file directly. Just don't expect a ranking or citation boost: independent studies found no measurable traffic change for 8 of 9 sites after adding it. Publish it, validate it, and move on to your content.

AI visibility · Crawler research · August 2026

Which AI Engines Actually Read llms.txt?

llms.txt is pitched as the file that helps ChatGPT, Claude, Perplexity and Gemini understand your site. Two years of independent crawler audits tell a more honest story — here is the evidence, straight from server logs, and what it means for your site.

The Short Answer: Almost No One Fetches It

If you publish an llms.txt today, the realistic outcome is that nothing automated will ever request it. In May 2026, Ahrefs analyzed 137,000 domains that publish the file: 97% of those files received zero requests during the entire month — no crawler, no agent, no human. The 3% that were fetched were hit almost entirely (96%) by bots, and only about a fifth of those fetches came from named AI tools.

This is not one outlier study. Independent audits keep arriving at the same number: zero.

What 500 Million Bot Requests Reveal

Two large log-based studies are worth knowing about because they measure what bots actually do, not what press releases say:

Weekerp (2025): across two production sites, 68,759 AI bot requests in a month — and /llms.txt received 0 requests. In the same period, robots.txt was fetched 1,938 times and sitemaps 1,224 times, proving AI bots do fetch supporting files when they care about them.
Limy (2026): analysis of 515 million LLM bot traffic events found that GPTBot, ClaudeBot, PerplexityBot, OAI-SearchBot and Google-Extended "overwhelmingly skip the file and crawl HTML directly."

Crawler	Owner	Fetches llms.txt?	Evidence
`GPTBot`	OpenAI	Rarely — the most common named fetcher	Ahrefs 2026: top named AI fetcher
`ClaudeBot` / `Claude-Code`	Anthropic	Training crawler: not observed; Claude-Code: second most common	Ahrefs 2026; Weekerp: 0 requests
`PerplexityBot`	Perplexity	Not observed	Weekerp, Limy audits
`OAI-SearchBot`	OpenAI	Not observed	Weekerp, Limy audits
`Google-Extended` / Googlebot	Google	No — officially unsupported	Illyes, July 2025

Google's Position Is On the Record

Google is the only major player that has answered the question publicly and unambiguously. In July 2025, Search Relations' Gary Illyes said Google has "no plans to support llms.txt", and John Mueller compared the file to the keywords meta tag — a self-declared signal that was abandoned because it was trivially gameable. Google's 2026 guidance for generative AI visibility lists llms.txt by name as a tactic that does not help.

Yes, some Google properties (like ai.google.dev) still serve the file — Mueller confirmed that is an internal content system artifact, not an endorsement. We covered the full nuance in Does llms.txt Affect SEO?

Adoption Is Growing Anyway

Despite the crawl data, adoption keeps climbing. Casey Burridge's crawl of the top 10,000 sites found 5.61% with a valid llms.txt in June 2026, up from 1.04% in July 2025 — roughly 5.4x growth in a year, or about 39,000 sites across the top million. SE Ranking measured 10.13% across 300,000 domains, and Rankability found 8.7% of the top 1,000 (15.8% of the 549 it could actually reach). Documentation platforms like Mintlify generate the file automatically, which is why the v2 spec and so many dev-tool sites adopted it early.

So who does read your file, realistically? Three groups:

Occasional fetchers. GPTBot and Claude-Code are the only named AI tools with meaningful request records (Ahrefs).
Humans and agents on demand. Users and AI assistants that are asked to summarize your site will often read llms.txt directly — this is the file's most reliable audience.
Future adopters. The ecosystem is young; the v2 spec, subpath files and markdown link relations were added specifically to make adoption easier.

Why Bots Skip Your llms.txt

Training runs from datasets, not live fetches. Most model training uses pre-built corpora like Common Crawl, so there is no crawl step where llms.txt would matter.
It costs crawl budget. Probing /llms.txt on every domain adds a request per host for little confirmed benefit, so most crawlers don't bother.
It is still unofficial. No LLM lab has committed to honoring the file, which makes it a speculative investment for crawler teams.
robots.txt already works. Access control is handled by a 30-year-old standard every crawler respects — llms.txt controls nothing and blocks nothing.

What This Means for Your Site

The honest playbook in 2026: publish a spec-compliant llms.txt because it is cheap and because the file's on-demand audience is real — then spend your remaining SEO time on things with proven returns, like crawlable HTML content, a clean robots.txt, and site speed. There is no evidence that llms.txt moves rankings, traffic or citations today.

Generate a valid file with the llms.txt generator and validate it with the checker.
Keep it small (5–15 links) and point links at markdown versions per the v2 spec.
Check your own logs once a month to see if anything requests it:

grep -c "llms.txt" /var/log/nginx/access.log
grep -i "GPTBot" /var/log/nginx/access.log | head -20

If the second command shows hits, your file is being discovered — but don't expect it. Ship the file, keep it valid, and let the ecosystem catch up. You're ready for the agentic web either way.