llms.txt

advancedspreading

A simple text file in the root of a website that concisely shows language models and AI agents what the most important content on the page is.

In plain terms

Imagine a hurried guest entering a huge department store. They do not want to walk through every floor, so you hand them a one-page guide at the entrance: who we are, where to find what, and which is the most important department. The llms.txt is exactly this guide, but on your website, and the guest is an AI system.

This is needed because websites are made for people: menus, ads, pop-ups and decorative code surround the substance. A language model can process only a limited amount of text at once, and it easily gets lost among all the clutter. llms.txt gives it the essentials, along with links to the most important subpages, in a clean, clutter-free format.

It is important that this is not an official standard, but a community proposal. No one compels AI companies to read it, nor have they promised to take it into account. It is more of a polite gesture toward machines than a guaranteed entry to AI responses.

How it works

The website owner creates a file named llms.txt and places it in the root of the site, making it accessible to anyone at pelda.hu/llms.txt.
The file is prepared in Markdown format, meaning simple text readable by both humans and machines: the page or company name at the top, followed by a short summary.
Thematic sections with link lists follow: the title, URL, and a one-sentence description of important subpages. Content that can be omitted due to space constraints may go into a separate 'Optional' section.
Many sites also create an llms-full.txt file alongside, which contains not only links but the entire text content in one block. This is not part of the original proposal but has spread from the practice of documentation tools.
When an AI agent or a coding agent searches for information on a site, instead of crawling numerous subpages, it can read this single file if the creators configured it that way or if you provide the address directly.

An everyday example

You struggle with a software user manual and ask your AI assistant to help configure it. If the software manufacturer published an llms.txt file and the assistant reads it, it can quickly find the relevant documentation page and provide a more accurate, up-to-date answer than if it guessed from memory.

Who is affected

marketerIt is one of the frequently mentioned, low-cost tools for appearing in AI search engines, but its impact should be assessed soberly. → Marketing

developerThe llms.txt file on documentation pages helps the coding agent work from up-to-date descriptions rather than outdated knowledge. → Code

entrepreneur, executiveYou can decide what to highlight about your company for machine readers, while more customers gather information via AI assistants.

A common misconception

MythIf I upload an llms.txt file, ChatGPT and other AI search engines will recommend me.

In realityNothing guarantees that AI systems will even read the file, and Google representatives have publicly stated that their search engine does not use it. Inclusion in AI answers still depends mainly on the quality and popularity of the content.

Limits and risks

It is not a standard but a recommendation: AI companies decide voluntarily whether to take it into account, and they rarely disclose this.
It does not protect or block anything: do not write what you do not want the AI to see into this file; instead, block it with another tool.
It must be maintained: an outdated file leads the machine reader to broken links and old information.
It can also be abused: the file can contain text other than what appears on the page that people see, so systems may treat it with caution.

Similar, but not the same

robots.txt – robots.txt tells robots what they may and may not crawl (compliant robots follow this, but technically it excludes no one), while llms.txt does not prohibit but recommends and summarizes.
sitemap (sitemap.xml) – A sitemap lists subpages for search robots in a mechanical way, while llms.txt provides a curated, annotated summary for language models.
generative search engine optimization – Generative search optimization is a full methodology for appearing in AI-generated answers; llms.txt is merely one of its tools, with debated impact.

Related concepts

Broader

generative search engine optimization, search engine optimization, web standards

Narrower

llms-full.txt

On the edge

robots.txt, sitemap, AI search bot, Markdown, context window

A brief history

The idea of llms.txt was published by Jeremy Howard, co-founder of Answer.AI, in September 2024. The proposal spread quickly across software documentation sites because multiple documentation generation tools automatically began producing the file.

Where things stand

At the end of 2026, many technology companies and documentation sites publish such a file, and coding agents often find it useful. However, it has not become an official standard, and the major AI search engines have not committed to using it.

Status: 2026 November

Also known as

llms.txt · LLMs.txt · /llms.txt · llms-full.txt · llms.txt file · llms.txt standard · llms.txt standard · Text file for LLMs

Sources

This entry was machine-edited and has undergone machine expert review; the “In the news” section updates automatically from Agent Age stories.