What Is llms.txt? Does Your Website Need One? (2026)

What is llms.txt?

llms.txt is a plain-text file, written in Markdown, that sits at the root of a website (yoursite.com/llms.txt) and gives AI tools a short, curated summary of the site with links to its most useful pages. Think of it as a hand-written contents page for large language models (LLMs).

It was proposed by Jeremy Howard of Answer.AI on 3 September 2024. The reasoning was sensible: web pages are cluttered with navigation, scripts, cookie banners and adverts that an AI tool has to strip away before it can use the content. A clean summary would help AI tools find the right information faster.

The proposal reached version 2 on 10 August 2026. It is still a convention, not an official standard. No search engine or AI company is obliged to read it, and, as the data below shows, most don’t.

What goes in an llms.txt file?

The format is deliberately simple, and only one element is required. In order, a file contains:

An H1 heading with the name of the site or project. This is the only required part.
A blockquote with a one- or two-sentence summary.
Optional paragraphs with any extra context.
H2 sections containing lists of links, each with a short description.

Here’s the start of our own llms.txt file:

# The AI SEO Agency > The AI SEO Agency is a UK agency that helps businesses get found, cited and recommended in AI search. ## Services - [AI Visibility Audit](https://theaiseoagency.co.uk/services/ai-visibility-audit/): Find out if ChatGPT, Perplexity and AI Overviews mention you. - [Entity SEO](https://theaiseoagency.co.uk/services/entity-seo/): Make your brand unambiguous to Google and AI models.

llms.txt vs llms-full.txt vs .md pages

You’ll see two related ideas mentioned alongside llms.txt:

llms-full.txt puts the full text of a site’s key content in one file, rather than just links. It’s popular with documentation platforms, but it isn’t part of the core proposal.
.md pages are clean Markdown copies of individual pages. The proposal suggests offering them at the same address as the page with .md added.

Both are mainly useful for software documentation, where developers point AI coding assistants at a product’s docs.

llms.txt vs robots.txt vs sitemap.xml

These three files are often confused, but they do very different jobs. Only one of them affects whether AI tools can read your site.

robots.txt tells crawlers what they may and may not access. Major AI crawlers such as GPTBot, ClaudeBot, PerplexityBot and Google-Extended check it. This is the file that actually controls AI access.
sitemap.xml lists the URLs you want search engines to discover. Google and Bing use it, which matters because ChatGPT search draws on web search results.
llms.txt suggests what’s most useful to read, in plain Markdown. No crawler is required to fetch it, and the evidence shows very few do.

If your robots.txt, CDN or firewall blocks AI crawlers, no llms.txt file will help. Check that first.

What the data says: 3 studies, 475,000+ domains

Three large studies have now tested llms.txt, and none found it makes a measurable difference to AI visibility.

Ahrefs, June 2026 (137,210 domains): 28% of sites published a valid llms.txt, but 97% of those files received zero requests in May 2026. Of the requests that did arrive, only 19.5% came from AI bots. The largest single group was SEO audit tools (21.7%). AI bots made no requests at all for llms.txt files that didn’t exist. They don’t go looking for them.
Trakkr, March 2026 (37,894 AI-cited domains): sites with llms.txt averaged 6.8 AI citations, against 6.7 for sites without. Only 6% of the 50 most-cited domains had one. Adoption was highest among SaaS and developer tools (24.1%).
SE Ranking, November 2025 (around 300,000 domains): about 1 in 10 sites had llms.txt, and the study found no statistically significant link between the file and AI citations.

The most-cited websites on the internet are mostly ones without llms.txt. They earn citations through authority and content, not a file.

Does Google or ChatGPT use llms.txt?

Google has said plainly that it doesn’t. Asked about llms.txt in June 2026, Google’s John Mueller said: “the file has existed for years, yet none of the AI systems use it”, calling any benefit “purely speculative for now”.

You may read that Google’s Lighthouse tool now checks for llms.txt. It does, but the check is narrow. It flags the file only if it returns a server error, and marks a missing file as “Not Applicable”. That confirms the file works if you have one. It isn’t a ranking signal.

The v2 proposal notes that OpenAI, Anthropic and Google publish llms.txt files for their own developer documentation. Publishing a file isn’t the same as reading yours, though. In Ahrefs’ data, AI assistants made up just 2.5% of llms.txt requests, and AI retrieval bots (the kind that fetch pages to answer a live question) made up 1.1%.

Do UK AI SEO agencies use llms.txt?

Almost half do, far more than the evidence would justify. On 26 September 2026, we checked 22 UK agencies that appear in “AI SEO agency” search results and “best AI SEO agencies UK” lists. Of the 20 we could reach, 9 (45%) published a valid llms.txt file. That’s well above the 10–28% rates found in the large studies above.

This is a small, single-day sample picked from search visibility, so treat it as an indicator rather than an industry statistic. The pattern is still telling. The businesses selling AI SEO are among the keenest adopters of a file that AI crawlers largely ignore.

Does llms.txt help SEO or AI visibility?

Not measurably, based on the evidence available today. llms.txt won’t improve your Google rankings, and there’s no data showing it increases how often ChatGPT, Perplexity or Google AI Overviews cite you.

What does affect AI visibility:

Crawler access: make sure robots.txt, your CDN and your firewall aren’t blocking AI search crawlers.
Indexable HTML: AI tools read your pages directly, so key content needs to be in the page, not hidden behind scripts.
Answer-first content: clear, self-contained answers that are easy to quote.
A clear brand entity: consistent facts about who you are across your site and profiles. See our entity SEO service.
Mentions on trusted sites: reviews, directories, press coverage and comparison articles that AI answers draw on.

So should you add one? If it takes ten minutes, go ahead. It’s harmless, and it’s most useful for software and documentation sites. Just don’t spend budget on it, and be wary of anyone selling it as a way to get recommended by ChatGPT.

How to create an llms.txt file

Pick your 10–30 most important pages: services, key guides, pricing, about and contact.
Write an H1 and a one-sentence summary of what your business does and who it serves.
Group the links under H2 headings, each with a one-line description.
Save it as llms.txt and upload it to your site’s root folder, so it loads at yoursite.com/llms.txt.
Check it loads without errors, and keep it up to date. Several SEO plugins and site builders can now generate it automatically. Ours rebuilds itself from our site content every time we publish.

Want to know what actually affects your visibility in AI search? Our AI visibility audit checks crawler access, llms.txt, entity clarity and how AI tools answer your customers’ questions today, then ranks what to fix first.

FAQs (add each one in the FAQs field)

Q: Is llms.txt an official web standard?

A: No. It’s a community proposal first published by Jeremy Howard of Answer.AI in September 2024, now on version 2. No search engine or AI company is required to read it, and Google has said none of the AI systems use it.

Q: Does llms.txt replace robots.txt?

A: No. robots.txt controls which crawlers may access your site, and major AI crawlers respect it. llms.txt only suggests which content is most useful. If robots.txt blocks AI crawlers, llms.txt can’t help.

Q: Will adding llms.txt improve my Google rankings?

A: No. Google has said it doesn’t use llms.txt, and large studies of hundreds of thousands of domains found no measurable link between having the file and being cited by AI tools.

Q: Can llms.txt stop AI companies using my content?

A: No. To control AI crawler access, use robots.txt rules for specific bots such as GPTBot, ClaudeBot or Google-Extended. llms.txt is an invitation, not a restriction.

Q: Where should I put my llms.txt file?

A: In your website’s root folder, so it loads at yoursite.com/llms.txt. It should be a plain text file written in Markdown, starting with an H1 heading.

What Is llms.txt? And Does Your Website Need One?