llms.txt explained: what it does and what it doesn’t

What is llms.txt?

llms.txt is a plain Markdown file served at /llms.txt. The idea, first published by Jeremy Howard on 3 September 2024 and revised in August 2026, is to give language models and agents a short, curated map of a site instead of making them parse pages designed for people. The proposal notes that thousands of sites now publish one, including the major AI labs.

It is not robots.txt. It does not grant or block access, and no crawler is obliged to read it. Think of it as a table of contents written for machines.

The appeal is easy to see. Web pages carry navigation, scripts, cookie banners and layout that a model has to strip away to find the content. A hand-picked list of the pages that matter, with a line on what each one answers, is a cleaner starting point for any agent that chooses to read it.

What format does llms.txt use?

The specification fixes the order of the sections. Only the first heading is required; everything else is optional but should appear in this order.

• An H1 with the name of the site or project (required).
• A blockquote with a short summary of what the site is.
• Optional paragraphs or lists with extra context, without headings.
• H2 sections, each holding a list of links written as [name](url), optionally followed by a colon and a note.
• An H2 called “Optional” for secondary links that can be skipped when context is short.

Does llms.txt improve AI citations?

Not on current evidence. CitePulse’s August 2026 review summarises two large studies. Ahrefs looked at 137,210 domains and found that 97% of llms.txt files receive no requests at all; of the requests that did arrive, 96% came from bots and 77% of that bot traffic was SEO auditing tools, with named AI retrieval bots at roughly 1%. SE Ranking tested about 300,000 domains and found no statistically significant relationship between having the file and being cited.

Google is explicit. Its May 2026 guide to generative AI features says special AI text files are not needed, and coverage of the guide quotes Google that such files are treated like any other text file, with no special treatment or preferred indexing pathway.

The verified readers are coding assistants and agents that fetch documentation on request, such as tools a developer points at your docs. That makes the file most useful for products with an API or technical documentation, and much less useful for a brochure site hoping for search traffic.

Should you still publish one?

Yes, if it takes you less than an hour. It costs nothing to serve, it forces you to decide which pages matter most, and it helps any agent that does read it. Just do not report it as an AI visibility win, and do not let it take time from the work that does move citations: pages crawlers can read in raw HTML, answers that match real questions, and mentions on other sites.

The free llms.txt generator on this site builds a valid file from your site name, summary and key links, in the order the specification requires.

What does a simple llms.txt look like?

For a small business or a personal site, the whole file can fit on one screen. Start with the site name as the only H1, follow it with a one- or two-sentence blockquote that says what the site offers and for whom, then add one or two H2 sections of links. A portfolio might have a section for case studies and one for writing; a software company might have sections for documentation, pricing and guides.

Each link should point to a page that works on its own: a reader who lands there from the list should understand it without reading the home page first. Write the note after each link as the question the page answers, which makes the list useful to people as well as to agents. If you have Markdown versions of key pages, the proposal suggests linking those, but a normal HTML URL is fine.

• # Site name
• > One-sentence summary of what the site offers and who it is for.
• ## Case studies — [Project name](URL): the result in one line.
• ## Optional — links that are useful but can be skipped.

What should you do instead, if you want AI visibility?

Spend the time where the evidence points. Make sure the pages you want quoted are readable in raw HTML, because most AI crawlers do not run JavaScript. Rewrite those pages so the answer to the buyer’s question comes first. Then work on mentions elsewhere: reviews, partner pages, community answers and original data that other writers cite. The 2026 studies that found no effect from llms.txt found large effects from brand mentions and authority.

Finally, measure. Run a fixed set of buyer questions through the AI engines you care about every week and track how often you are cited. That number, not the presence of a file, tells you whether the work is paying off.

What should a good llms.txt include?

Keep it short and current. List the pages a newcomer would need to understand the site: what you do, your main products or services, pricing, documentation, case studies, and contact. Give each link a one-line note that says what the page answers. Put anything nice-to-have under Optional. Update it when you add or retire a key page, the same way you would update a sitemap.