What is llms.txt?
The complete guide
llms.txt is the "content map" that AI engines read to understand your website. This guide explains what it is, why it matters, and how to create and deploy it in minutes.
What is llms.txt?
llms.txt is a Markdown file placed in the root of your website, proposed in 2024 by AI researcher Jeremy Howard as an open convention for large language models (LLMs). Just as robots.txt gives search-engine crawlers directions, llms.txt gives AI engines a structured summary and index of your site's content — so they can understand and cite it accurately instead of guessing.
The canonical specification lives at llmstxt.org.
Why llms.txt matters now
AI is becoming a primary entry point to the web. Users increasingly ask ChatGPT, Claude, Perplexity and Gemini questions instead of typing keywords into Google. These AI engines need an efficient, reliable way to understand your site:
- Without llms.txt: AI agents crawl your entire site, guess at what's important, and may miss or misread your core content.
- With llms.txt: AI reads your curated content map directly and cites your pages accurately — the foundation of Generative Engine Optimization (GEO).
The llms.txt format
An llms.txt file is deliberately simple — plain Markdown. A minimal valid file looks like this:
# Site Name
> One-sentence description of the site (optional).
## Important links
- [Page title](https://your-site.com/page): Short description
- [Another page](https://your-site.com/other): Another descriptionThe rules per the llmstxt.org spec:
- The H1 heading (
# Site Name) is the only mandatory element. Everything else is optional. - Links must be absolute URLs (
https://…) — no relative paths. - Group links under H2 headings (
## Section); use Markdown list items for links. - Optionally add a blockquote tagline (
> …) describing the site. - For documentation-heavy sites, you can reference a full version at
/llms-full.txt.
llms.txt vs robots.txt vs sitemap.xml
They complement each other — most sites should have all three.
| File | Audience | Purpose |
|---|---|---|
robots.txt | Search engine crawlers | Allows / blocks crawling |
sitemap.xml | Search engine crawlers | Lists all URLs for indexing |
| llms.txt | AI engines (ChatGPT, Claude, Perplexity…) | Structured content summary for LLM understanding & citation |
How to create an llms.txt
- Use our free generator — enter your site name, tagline and URLs (or import a sitemap) and it produces a spec-compliant file instantly.
- Download the file and upload it to your website root, next to
robots.txt. - Verify it's publicly accessible at
https://your-domain.com/llms.txt. - Run it through the validator to confirm the structure and get an AI-readiness score.
- Update it whenever your site structure changes significantly.
Frequently asked questions
Does llms.txt affect my Google ranking?
No, not directly. Traditional search engines like Google do not use llms.txt. Its value is in how AI engines (ChatGPT, Claude, Perplexity, etc.) understand and cite your content — the GEO channel.
How often should I update it?
Whenever your site structure changes in a meaningful way — new sections, renamed pages, removed content. For most sites, reviewing it monthly is plenty.
Is llms.txt an official standard?
Not yet formally. It's a community-driven open convention that isn't standardized by the W3C or IETF, but adoption is growing quickly — tools, frameworks and AI products increasingly recognize it.
Where should the file live?
In the root of your website, exactly like robots.txt. The canonical URL is https://your-domain.com/llms.txt. AI engines look for it there first.
Ready to get started?
Generate your first llms.txt in under a minute — free, no sign-up.
⚡ Generate your llms.txt