AI Agents & Discovery · September 2026

llms.txt vs MCP: The Read Layer vs the Action Layer

Two protocols get mentioned in the same breath and solve opposite problems. llms.txt answers "what is on this site?" — a static map an agent reads once. MCP answers "what can I do here?" — a live interface an agent calls. Confusing them leads to the wrong investment: a file you never needed to make dynamic, or a server nobody can find.

Two Questions, Two Protocols

Ask a chat assistant to "summarise how this company handles refunds" and it needs content: which page, in what form. Ask an agent to "find the cheapest plan for eight seats" and it needs capabilities: a query it can run against your data or your UI. llms.txt is the accepted answer to the first, MCP to the second, and a third, browser-level answer — WebMCP — is emerging underneath both.

llms.txt Is the Read Layer

/llms.txt is a Markdown file at your site root, proposed by Jeremy Howard in September 2024 and maintained by the community as a convention rather than a formal standard — the current shape of it is documented at llmstxt.org. It carries an H1 name, a summary blockquote and curated sections of links with one-line descriptions. Nothing executes; nothing authenticates.

Its value is proportional to how bad your website is to parse. An agent landing on an HTML marketing page burns context on navigation, cookie banners and markup before it reaches the answer; a curated file replaces that with a handful of links. That is why documentation-heavy sites adopted first — and why independent crawl studies put overall adoption at roughly 10% of domains, far below robots.txt or XML sitemaps.

Treat it as the cheap, static, cacheable layer. The format guide covers every element and the generator turns a sitemap into a first draft. Google has said repeatedly that the file has no effect on Search rankings — see what Google actually says. That makes it an agent-facing asset, not a ranking asset.

MCP Is the Action Layer

The Model Context Protocol, launched by Anthropic in late 2024 and now maintained as an open specification at modelcontextprotocol.io, defines how a client (an assistant or IDE) connects to a server that exposes tools, resources and prompts. Instead of reading your documentation, the agent calls it: a search tool, a pricing lookup, a status endpoint.

The 2026-07-28 revision is the biggest change since remote MCP shipped. The initialize handshake and the Mcp-Session-Id header are retired: requests are stateless, each carrying its protocol version, client identity and capabilities in request metadata. A capability inventory is now an optional server/discover call, and the new Mcp-Method and Mcp-Name headers let a gateway route on the operation without parsing the body:

POST /mcp HTTP/1.1
MCP-Protocol-Version: 2026-07-28
Mcp-Method: tools/call
Mcp-Name: search
Authorization: Bearer <token>
Content-Type: application/json

The practical consequence: with no session store to share, a server can run behind a plain round-robin load balancer — or, as Cloudflare describes in its MCP v2 write-up, entirely inside a single Worker. Remote servers are expected to use OAuth 2.0 with PKCE, and updated TypeScript, Python, Go and C# SDKs shipped with the spec.

MCP has one weakness llms.txt does not: the protocol has no discovery. Agents connect only to servers they were configured with, so yours can be excellent and unknown. This is where the read layer earns its keep: a curated file naming your MCP endpoint, what it exposes and who it is for is the most reliable "here is my server" signal you control.

WebMCP: The Third, In-Browser Layer

WebMCP moves the same idea into the page. Rather than a server the agent connects to, the browser exposes tools the site declares: declaratively by annotating existing HTML forms, or imperatively with navigator.modelContext.registerTool. The agent then operates the interface the user is already looking at, inside the visitor's own session — no separate endpoint, no separate auth.

WebMCP is still experimental. Chrome's documentation for Lighthouse's Agentic Browsing category — which verifies tool registration, checks the accessibility tree, measures layout stability and inspects llms.txt discoverability — requires Chrome 150 or later, and its WebMCP audits require the origin trial. Not a ranking factor, but a clear signal of where browser vendors are heading.

Side by Side

Layer What it is Who consumes it Cost to ship
llms.txt Static Markdown index at the site root Reading crawlers and assistants An afternoon, then occasional edits
MCP server Live tools and resources over a stateless protocol Configured clients: IDEs, assistants, agents Engineering work, auth, ongoing maintenance
WebMCP Browser-exposed tools from an existing page Agents acting in a live session Form annotations today, experimental API

The Adoption Order That Makes Sense

Layer these in sequence, and only move down when the layer above is measurably working:

The short version: llms.txt is how an agent learns what you have, MCP is how it acts on it, and neither replaces the other. Ship the file, instrument it, and let real request data — not vendor enthusiasm — decide whether a server comes next.