llms.txt tutorial · 10-minute setup

How to Create an llms.txt File: Step-by-Step Guide

An llms.txt file tells AI engines — ChatGPT, Claude, Gemini, Perplexity, and coding agents — exactly which pages of your site are worth reading. This llms.txt tutorial walks you from zero to a live, validated file in about 10 minutes, no matter what your site is built with.

Why Create an llms.txt File in the First Place?

Every day, AI assistants answer questions with content from your site. Without an llms.txt file, those models have to guess what to crawl, what to skip, and which of your pages actually matter. The result: they often land on your marketing pages instead of your documentation, or miss your best content entirely. An llms.txt file fixes that by giving agents a small, curated Markdown index of your most important pages — the LLM equivalent of robots.txt, but for what should be read instead of what shouldn't.

The payoff is concrete: Chrome's Lighthouse now audits sites for an llms.txt file, documentation platforms like Mintlify and GitBook generate one automatically, and major AI labs publish their own. Sites with a valid file get referenced more accurately — and more often — in AI-generated answers. Creating one is one of the cheapest SEO moves available: a single text file, no plugins, no redesign. Here's the exact process.

Step 1: Pick Your Site Name and Write a One-Line Summary

The first two lines of your file are the most important. Line one is the H1: your site or project name, prefixed with a single #. Line two is a blockquote (a line starting with >) summarizing what the site is and who it's for, in one or two sentences. This is the only context an agent gets before deciding whether to read further, so be concrete: what you do, for whom, and what makes you different.

# Acme Corp

> Acme Corp builds time-tracking software for small teams. Free plan available for up to 10 users.

Keep the summary short and factual — a one-sentence pitch works better than marketing fluff. The H1 here is the only element the llmstxt.org v2 specification marks as required, so never skip it.

Step 2: List Your 5–15 Most Important Pages

An llms.txt file is a curated index, not a sitemap. Resist the urge to list everything — a well-chosen 5–15 links beats 200 links that drown agents in noise. Include pages an AI should actually quote: product overviews, documentation, API references, pricing, setup guides. Skip boilerplate, thin blog posts, login pages, and anything behind an auth wall. Write down the full absolute URLs for your candidates:

# Candidate pages
- https://acme.dev/product.md         → Product overview
- https://acme.dev/pricing.md         → Plans and pricing
- https://acme.dev/docs/quickstart.md → Quick start guide
- https://acme.dev/docs/api.md        → REST API reference
- https://acme.dev/blog/              → Skip: low value for AI answers

Prefer clean Markdown versions of your pages when they exist (most documentation sites serve .md variants of every page — same URL with .md appended). Plain HTML pages work too, but Markdown is cheaper for LLMs to parse and less likely to be truncated.

Step 3: Organize Everything in the llms.txt Format

Now turn your list into the spec structure: H1 at the top, blockquote summary, then ## sections (H2 headings) each containing a Markdown link list. Every link is written as [text](url): short description — the description after the colon is optional but recommended, because it tells agents what's behind each link without fetching it. Use a ## Optional section at the end for secondary links agents can skip when context is short. Here's the complete skeleton:

# Acme Corp

> Acme Corp builds time-tracking software for small teams.

## Important links

- [Product overview](https://acme.dev/product.md): What Acme does and why teams choose it
- [Pricing](https://acme.dev/pricing.md): Plans, per-seat pricing, and the free tier

## Documentation

- [Quick start](https://acme.dev/docs/quickstart.md): Set up Acme in 10 minutes
- [API reference](https://acme.dev/docs/api.md): Complete REST API reference

## Optional

- [Changelog](https://acme.dev/changelog.md): Release notes for all versions

Three rules keep the file machine-readable: all URLs must be absolute (https://…, never /relative/paths), there is no comment syntax (a line starting with # is a heading, so don't use hash comments), and H2 sections must contain lists of links — not prose. Prose belongs in the free-form notes area between the blockquote and the first ##.

Step 4: Generate the File Automatically (or Write It by Hand)

If you've already followed Steps 1–3, writing the file by hand takes two minutes — it's plain Markdown, nothing more. That said, hand-writing is exactly where mistakes creep in: missing H1s, relative URLs, sections without descriptions. If you want a spec-perfect file in seconds, paste your site URL into our free llms.txt generator — it crawls your most important pages, builds the formatted file for you, and lets you tweak the section names and descriptions before downloading:

# Generated by llmstxtgenerator.dev
# Acme Corp

> Acme Corp builds time-tracking software for small teams.

## Important links

- [Product overview](https://acme.dev/product.md): ...
- [Pricing](https://acme.dev/pricing.md): ...

Either path produces the same plain-text file. The generator is fastest for large or frequently changing sites; writing by hand gives you full control over tone and link order. For a first file, we recommend the generator, then manual edits — the combination of speed and control is hard to beat. Save the result as llms.txt (lowercase, no extension, UTF-8 encoding).

Step 5: Put the File at Your Website's Root

The file must be publicly reachable at https://your-domain.com/llms.txt. Exactly where that file lives in your project depends on your platform — here's where to place it for the most common setups:

On a static host, the deployment step is usually as simple as dropping the file next to your index.html and redeploying — for example:

cp llms.txt public/llms.txt
# then rebuild / redeploy as usual

Step 6: Verify the File Is Actually Reachable

A file that isn't publicly accessible is the same as no file at all — and this is the step most people skip. Open a browser (or an incognito window) and visit https://your-domain.com/llms.txt. You should see your plain-text file, not a 404, a redirect, or a login page. From the terminal, the equivalent check is:

curl -I https://your-domain.com/llms.txt
# Expect: HTTP/2 200 and content-type: text/plain

A 200 status with a text/plain content type is exactly what you want. If you get a 404, double-check the file name (it's llms.txt, not llms.txt.txt or Llms.txt) and the deploy path from Step 5. A redirect is acceptable on platforms like Shopify, but a 404 isn't.

Step 7: Validate the File With the llms.txt Checker

The last step is a quality gate. Paste the URL or file contents into the free llms.txt checker, which validates the structure against the llmstxt.org v2 specification — required H1, absolute URLs, correctly formed sections — and scores AI-readiness:

Checking https://your-domain.com/llms.txt ...
✔ H1 heading present
✔ Blockquote summary found
✔ 3 sections, all with link lists
✔ All 9 links use absolute URLs
✔ 0 errors — structure score: 10/10

Fix anything the checker flags, redeploy, and re-check. Once you see zero errors, you're done: agents will start finding the file the next time they crawl your domain. That's the whole process — seven steps, ten minutes, one small text file.

Hand-Written or Generated: Which llms.txt Should You Use?

Both approaches are valid, and the file format is identical either way — so this is a workflow choice, not a quality one. Hand-writing gives you total control over wording and ordering, which matters if your descriptions need a human editorial touch. The generator wins on speed, completeness, and consistency: it never forgets the H1, never emits a relative URL, and regenerates in seconds when your site changes. The pragmatic pattern is a hybrid: generate an initial file, review and polish the descriptions, then keep the generator around for monthly refreshes. If you're creating files for multiple sites or products, the generator's consistency alone is worth it.

Common llms.txt Creation Mistakes to Avoid

Most problems in hand-written files come down to a handful of recurring mistakes:

• Missing H1. Starting the file with prose or an H2 means the required title element is gone. Always begin with # Site Name.
• Relative URLs. - [Docs](/docs) breaks parsers and agents. Every link must be an absolute https://… URL.
• Treating it like a sitemap. Listing hundreds of pages dilutes the file's value. Curate: 5–15 links that actually help agents answer questions about you.
• Hash comments. A line like # TODO: add pricing is parsed as a heading. There is no comment syntax — use HTML comments (<!-- ... -->) in the notes area if you must annotate.
• Links without descriptions. Bare links force agents to fetch each page to learn what it is. Add a : short description to every item.
• Never re-checking. Links go stale, pages get renamed, sections disappear. Set a monthly reminder to re-run the checker, or regenerate from scratch after any redesign.

llms.txt Creation FAQ