What is llms.txt and do you need one?

What is llms.txt?

llms.txt is a plain-text Markdown file at the root of your domain — yoursite.com/llms.txt — that tells AI crawlers what your site is about and links to the pages that matter most. Think of it as a curated map for language models: a short, human-readable index of your best content.

The format is deliberately simple: an H1 with your project name, a one-line summary in a blockquote, then Markdown sections of links. Any writer can produce one without touching code, and any model can parse it without special tooling.

It was proposed in 2024 by Jeremy Howard as a way to give language models a clean, token-efficient entry point to a site instead of forcing them to crawl bloated HTML. You can see a working example at our own /llms.txt.

For the one-line definition and how it sits next to robots.txt and sitemaps, see the llms.txt entry in our glossary.

What is llms.txt NOT?

llms.txt is not access control, not a ranking hack, and not a guarantee. It does not block or grant crawlers anything — that is robots.txt's job — and no search engine has confirmed it as a ranking factor. It is a discovery aid, not a lever you pull to move up results.

It is not robots.txt. robots.txt tells crawlers what they may and may not fetch; llms.txt assumes they are already welcome and simply points them at your best material. Putting a page in llms.txt does not hide it, protect it, or lock it.

It is not a confirmed ranking factor. No major search engine has said llms.txt changes rankings, and treating it as a growth hack sets you up for disappointment. Its value is discovery and clarity, not leverage.

It is not guaranteed to be read. Adoption is growing but voluntary — some crawlers honor it, others ignore it. Publish it because it is cheap and helpful, not because every model is contractually bound to it.

Do you actually need an llms.txt file?

You need llms.txt if AI answers matter to your business and your best content is spread across a large or messy site. It is a low-cost, low-risk file that takes minutes to write. If you have five pages and clean navigation, the upside is small — but it rarely hurts.

You benefit most if you run documentation, a large blog, a knowledge base, or a product with many pages — anywhere your best answers are buried under navigation, marketing, and boilerplate. llms.txt lets you say 'start here' in a single file.

You benefit least if your site is tiny and already well-structured. A five-page site with clear headings and clean schema is easy for a model to read without a manifest.

Either way, the effort is measured in minutes, and it pairs naturally with the work of getting cited by ChatGPT, Perplexity and AI Overviews — llms.txt helps engines find the pages that earn those citations.

What goes in an llms.txt file? (copy-paste template)

An llms.txt file is Markdown with four parts: an H1 with your name, a blockquote summarizing what you do, and H2 sections listing your key pages as links with one-line descriptions. An optional '## Optional' section holds secondary links a model can skip when its context is tight.

Here is a minimal template. Replace the names, URLs and descriptions with your own, keep each description to one plain sentence, and list only pages you would be proud to have quoted.

Two rules keep it useful: link to canonical, self-contained pages rather than category shells, and write descriptions for a reader, not a keyword stuffer. The file lives at your domain root, exactly like our example.

• # Acme Robotics
• > Industrial robot arms and the software that runs them, for small manufacturers.
• ## Docs
• - Quickstart: Unbox, wire, and run your first job in 20 minutes.
• - API reference: Every endpoint, parameter, and code sample.
• ## Product
• - Pricing: Plans, unit limits, and what each tier includes.
• - Safety guide: Setup, guarding, and compliance basics.
• ## Optional
• - Changelog: Release notes — safe to skip for core questions.

How does llms.txt fit the rest of your AI-search strategy?

llms.txt is one tile in a bigger picture, not the whole board. It helps AI engines find your best pages, but they still judge what they read. Pair it with clean schema markup, quotable answer-shaped content, and a healthy crawlable site — that combination is what actually earns citations.

llms.txt handles discovery — it tells engines where to look. Schema markup handles understanding — it tells them what each page means. You want both, because a model that finds a page still needs unambiguous facts to quote.

Content quality decides the rest. Answer-shaped, quotable sections are what actually get lifted into AI Overviews and chatbot answers, whether or not a manifest pointed the way.

And none of it works on a site crawlers cannot read. Before you polish your llms.txt, make sure the fundamentals pass with a technical SEO audit — broken rendering and blocked resources undo every manifest you write.