Ranking Fixer

llms.txt: What It Is, Who Reads It, and Whether You Need One

llms.txt is a plain Markdown file at the root of a site, at the address /llms.txt, that summarizes the site and lists its most useful links for large language models. Jeremy Howard of Answer.AI proposed it in September 2024. It is cheap and harmless, but it is not a standard the major search engines use: Google has said Search does not read it.

Short answer: add one if your site is technical documentation, or if the basics are already done. Do not expect rankings or AI citations from it. This site publishes one, and the example below is taken from it.

What an llms.txt file looks like

The proposal keeps the structure simple: a title with the site name, a blockquote with a short summary, then sections that list links, each with a one-line description. This is the start of the file on rankingfixer.com, shortened:

# Ranking Fixer

> Ranking Fixer (rankingfixer.com) is a technical SEO consultancy run by
> Calebe Araujo (…) It finds the technical problems that keep websites
> from ranking on Google and implements the fixes in the code.

## Services

- [Technical SEO audit](https://rankingfixer.com/technical-seo-audit): crawling,
  indexing, rendering, speed and structured data, ranked by impact.
- [Google indexing fixes](https://rankingfixer.com/google-indexing-issues):
  Search Console page indexing errors.

## Guides

- [Soft 404 errors in Search Console](https://rankingfixer.com/guides/soft-404):
  pages that return 200 but look missing.

The full file is at rankingfixer.com/llms.txt. The proposal also describes an optional llms-full.txt with the full text of the pages in one file.

llms.txt vs robots.txt vs sitemap.xml

All three live at the root of the site and get mixed up, but they do different jobs. Only llms.txt is not an established standard.

The three files at the root of a site
FileWhat it doesWho reads itA standard?
robots.txtTells crawlers which URLs they may fetchSearch engines and most AI crawlersYes, for decades
sitemap.xmlLists the URLs you want discoveredGoogle, Bing and other search enginesYes
llms.txtSummarizes the site and points AI tools to its most useful linksSome tools, usually when someone hands them the linkNo: a proposal without confirmed support from the major search engines

Who actually reads llms.txt

The real use today is software documentation. Documentation platforms generate the file automatically, and developers paste the link into AI coding assistants so the tool reads the whole documentation at once. There the file works because someone hands it to the model on purpose.

Search is a different story. Google has said it does not use llms.txt, and no major answer engine has confirmed relying on it to choose sources. AI Overviews and AI Mode work from Google's regular search index. ChatGPT search uses OpenAI's own crawlers and search partners such as Bing.

Should you add one?

What to fix before llms.txt

How to write an llms.txt file

  1. Start with a title and a summary. The first line is "# Site name", followed by a blockquote (a line starting with ">") that says what the site is and who it is for, in one or two sentences.
  2. Group the key pages in sections. Use "##" headings such as Services, Guides and About. Under each, list the pages as Markdown links with a short description after a colon.
  3. List only live, indexable pages. Every link should return 200 and be allowed in search. A broken link or a noindexed page in llms.txt is a sign the file was written once and forgotten.
  4. Publish it at the root as plain text. Serve it at /llms.txt on the main host, next to robots.txt. A static file is enough; it does not need to be in the sitemap.
  5. Keep it in sync with the site. Update it when you publish or remove an important page. The summary has to match what the pages actually say.

Questions about llms.txt

Does llms.txt help SEO?

No. Google has said Search does not use llms.txt, so it does not change rankings or your chances of appearing in AI Overviews. The file is cheap and harmless, but what gets a site found and cited is still indexing, pages that answer the question and a source search engines trust.

Is llms.txt the same as robots.txt?

No. robots.txt controls which crawlers may fetch which URLs, and search engines and most AI crawlers respect it. llms.txt only describes the site; it neither allows nor blocks anything.

Does ChatGPT read llms.txt?

OpenAI has not confirmed using llms.txt in ChatGPT search. For ChatGPT to cite your pages, robots.txt has to allow OAI-SearchBot, its search crawler, and the pages have to be well indexed, including in Bing.

What is llms-full.txt?

An optional companion file from the same proposal that holds the full text of the important pages in one Markdown file. It is mostly used by software documentation, so a developer can give an AI coding assistant the whole documentation at once.

Not sure your site needs an llms.txt?

Email the site address. I check what matters more first, like indexing and crawler access, and tell you whether the file is worth adding in your case.

Ask about your site

Page updated on .