Do AI crawlers like GPTBot and ClaudeBot execute JavaScript?

On the published evidence, no. Vercel's network analysis with MERJ — monitoring nextjs.org and the Vercel network — reported that none of the major AI crawlers render JavaScript: OpenAI's GPTBot, OAI-SearchBot and ChatGPT-User, Anthropic's ClaudeBot, Meta-ExternalAgent, Bytespider and PerplexityBot all fetch the raw HTML and stop. Those crawlers still download JavaScript files — 11.50% of ChatGPT's requests and 23.84% of Claude's — without executing them. Gemini inherits Googlebot's rendering and Applebot runs a browser-based crawler, so those two can see more; Googlebot's own rendering is a separate system. Treat this as the current baseline and re-test periodically, because crawler behaviour changes.

My llms.txt is valid — why would rendering still matter?

Because llms.txt is an index of URLs, not a copy of the content. If the pages it lists are built client-side, a crawler that does not render receives whatever the server sent in the first response: a shell with a root element and script tags. We probed 54 URLs listed inside nine production llms.txt files on October 12, 2026 and 53 returned readable text, because the strong files point at markdown twins or server-rendered pages rather than at client-rendered routes. The one failure was not a shell but a dead link: squareup.com/us/en/licenses, listed as 'Licenses' in Square's llms.txt, returned 404 to GPTBot, ClaudeBot, PerplexityBot and a desktop Chrome user agent alike.

What is the fastest fix for a client-rendered site?

Ship a machine-readable twin and link that. Serve the page's Markdown at the same URL with a .md extension, declare it with rel=alternate type=text/markdown, and point the llms.txt entry at the .md URL so the crawler never has to render anything. Linear, React and Square all do this in production — Linear's entries resolve to text/markdown, React's to text/plain .md files, Square's to Markdown on its CDN. Where a twin is impractical, server-render or pre-render the high-value pages, and avoid dynamic rendering that whitelists Googlebot only.

GEO · October 2026

Does llms.txt Work on JavaScript-Rendered Sites?

An llms.txt file is an index of URLs — and every URL in it is a promise that a crawler can read something at the other end. On October 12, 2026 we fetched /llms.txt from twelve product domains with the GPTBot user agent and then probed 54 of the URLs those files list. Here is what a client-rendered page actually returns to a crawler that never runs JavaScript, and the two commands that tell you whether your own site has the problem.

No Major AI Crawler Renders JavaScript

Vercel's crawl analysis with MERJ, run across its own network and nextjs.org, reported that none of the major AI crawlers execute JavaScript: OpenAI's GPTBot, OAI-SearchBot and ChatGPT-User, Anthropic's ClaudeBot, Meta-ExternalAgent, Bytespider and PerplexityBot all stop at the raw HTML response. They fetch script files anyway — 11.50% of ChatGPT's requests and 23.84% of Claude's were JavaScript — without ever running them, which is why a JavaScript-heavy site looks busy in server logs and invisible in answers.

The exceptions are infrastructure, not effort. Gemini rides on Googlebot, and Applebot runs a browser-based crawler, so both can render. That is the trap for teams that solved this problem once already: Google's rendering investment applies to Google. A site that fixed visibility with dynamic rendering scoped to Googlebot is still serving an empty document to everyone else. The full picture of which engines read what is in which AI engines actually read llms.txt.

What Twelve Production llms.txt Paths Returned

Every request below used the GPTBot user agent on October 12, 2026, and followed redirects:

Host	Status	Content-Type	Bytes
webflow.com	200	text/plain	13,293
framer.com	200 (308 from non-www)	text/plain	9,636
linear.app	200	text/plain	9,159
clickup.com	200	text/plain	8,836
notion.so	200 (301 to www)	text/plain	6,780
asana.com	200	text/plain	5,691
trello.com	200	text/html	160,977
figma.com, airtable.com, miro.com, canva.com, discord.com	404 (each 400-byte HTML error body)	text/html	400

Six hosts serve a real plain-text file. Five do not serve one at all. And one — Trello — answers /llms.txt with 160,977 bytes of the site's own HTML: the string llms appears zero times in that body. That is the JavaScript-rendering problem wearing a different hat. A crawler asking for a text file receives a React page instead, gets no index out of it, and every downstream page that was supposed to be discoverable through the file stays invisible.

The same class of delivery bug is catalogued in llms.txt not working: seven delivery-level fixes, where a catch-all rewrite answering text/html is the first entry.

The Best Files Point at Markdown, Not at App Routes

We then fetched up to six URLs from each of the nine files that resolved, 54 in total. 53 returned readable text, and the pattern behind that hit rate is worth copying:

linear.app — entries resolve to .md twins served as text/markdown (208 to 1,024 words each). No rendering required.
react.dev — index links carry the .md suffix and arrive as text/plain, from 318 to 10,020 words.
svelte.dev — the index is six lines pointing at llms-small.txt, llms-medium.txt, llms-full.txt and per-section files.
squareup.com — product entries point at Markdown documents on its asset CDN rather than at site routes.
notion.com, asana.com, clickup.com, webflow.com, framer.com — HTML, but server-rendered: 437 to 6,491 words of text in the first response.

The single failure is a reminder that a curated file still needs its links checked: squareup.com/us/en/licenses, listed as Licenses, returned 404 to GPTBot, ClaudeBot, PerplexityBot and a desktop Chrome user agent alike. Markdown twins and how they are connected to the index are covered in Markdown for Agents vs llms.txt.

Where the Empty Shells Actually Appear

We measured fourteen app, login and dashboard URLs with the same user agent, counting words of text in the raw response:

URL	Status	Words in raw HTML
app.shopify.com	200	1,577
app.slack.com	200	914
github.com/lukuime	200	586
app.asana.com	200	48
vercel.com/dashboard	200	47
dashboard.stripe.com	200	34
tiktok.com	200	5
instagram.com, notion.so/login	200	1 each
x.com (402), app.webflow.com (404), app.linear.app and app.framer.com (no response)	failure	—

Six of the fourteen answered 200 OK with fewer than 60 words of extractable text. Most of those are login walls that are not meant to be indexed, and that is fine. The measurement matters because it is the same architecture: whatever is rendered on the client arrives at a non-rendering crawler as an almost empty document. The reassuring half of the dataset is the twelve public marketing homepages we measured in the same pass — every one returned an <h1> and between 616 and 2,849 words in the first response. Marketing sites are usually static or server-rendered by now. Public content routes inside an app shell are where the gap survives.

Two Commands That Reveal the Problem

Do this for the five pages you most want an AI engine to quote:

# 1. Fetch exactly what an AI crawler receives, with no JavaScript executed
curl -sS -o page.html \
  -A 'Mozilla/5.0 (compatible; GPTBot/1.2; +https://openai.com/gptbot)' \
  -w '%{http_code} %{content_type} %{size_download}\n' \
  https://www.example.com/your-page/

# 2. Count the words a parser would find, and look for one sentence you know is on the page
sed -e 's/<script[^>]*>.*<\/script>//g' -e 's/<[^>]*>/ /g' page.html | wc -w
grep -c 'the exact sentence your page is supposed to answer with' page.html

# 3. If you publish a Markdown twin, prove it resolves
curl -sS -o /dev/null -w '%{http_code} %{content_type}\n' https://www.example.com/your-page.md

A word count in the hundreds means the crawler got your content. A count under about sixty, or a grep that returns 0 for a sentence you can see in the browser, means the page exists only after JavaScript runs. The .md check decides your fix: if the twin answers 200 with text/markdown, the cheapest repair is to point the llms.txt entry at the twin.

The Four Fixes, in Order of Effort

Link the twin. Serve Markdown at the page URL plus .md, declare it with rel="alternate" type="text/markdown", and list the twin rather than the route. This is a content-pipeline change, not an architecture change, which is why Linear, React and Square already ship it.
Server-render the pages you list. Pre-render at build time where the content is stable; server-render at request time where it is not. What matters is that headings, body text, tables and internal links are present in the first response.
Stop scoping fixes to Googlebot. Dynamic rendering configured for one user agent leaves the AI crawlers on the shell, and they do not fall back to anything.
Then verify both layers. A file that passes a structural validation can still list dead or unreadable targets.

Build the index in the generator, run the link and structure pass in the checker, then fetch the pages it points at the way a crawler would — which is the part of llms.txt validation that involves no file at all.

The verdict: llms.txt works perfectly on a JavaScript-rendered site — as long as the URLs inside it serve readable HTML or Markdown to a client that never runs a script. Six of fourteen app routes we measured did not, one major domain answers /llms.txt itself with an HTML shell, and a curated file still managed to list a 404. Rendering is the invisible half of AI visibility: fix the response before you tune the file.