# What is llms.txt?

import { Aside } from '@astrojs/starlight/components';

`llms.txt` is a proposed standard for a single Markdown file, placed at the root of your site, that gives large language models a curated map of your documentation. Jeremy Howard of Answer.AI [proposed it on 3 September 2024](https://llmstxt.org/). This guide explains what the file is, what problem it solves, and how it relates to standards you already know.

## The problem it solves

When an AI assistant answers a question about your product, it often needs to read your documentation at inference time. But a full documentation site is hard for a model to consume: navigation, scripts, and styling wrap each page, and the whole site is far too large to fit in a model's context window.

`llms.txt` addresses this by offering a clean, curated index. Instead of crawling and parsing your entire site, a model can read one structured file that points to the pages that matter, each described in plain language.

## How llms.txt differs from robots.txt and sitemap.xml

`llms.txt` is often confused with two files it sits alongside. They serve different purposes.

| File | Audience | Purpose |
|------|----------|---------|
| `robots.txt` | Crawlers | Controls which paths a crawler may access. |
| `sitemap.xml` | Search engines | Lists every indexable URL for discovery. |
| `llms.txt` | Large language models | Provides a curated, human-readable overview for use at inference time. |

The distinction is curation. `sitemap.xml` aims to be complete and lists all of your pages. `llms.txt` is selective: it highlights the content most useful to a model and leaves out the rest.

<Aside type="note">
`llms.txt` does not control access and does not replace `robots.txt`. A model can still read any page your `robots.txt` allows. The file is an invitation to your best content, not a permission boundary.
</Aside>

## The file format

The specification defines a precise structure built from standard Markdown, so both people and programs can read it:

1. An **H1 heading** with the name of the project or site. This is the only required element.
2. A **blockquote** with a short summary of the project, capturing the key information needed to understand the rest of the file.
3. Zero or more **Markdown sections** of free-form detail (paragraphs, lists) with no headings.
4. Zero or more **H2 sections**, each containing a list of links. Each list item is a Markdown link, optionally followed by a colon and a note.

A minimal example:

```markdown
# Example project

> A short description of what the project does and who it is for.

Additional context about the project in plain prose, with no heading.

## Docs

- [Quickstart](https://example.com/quickstart.md): Get running in five minutes
- [API reference](https://example.com/api.md): Full endpoint reference

## Optional

- [Changelog](https://example.com/changelog.md): Release history
```

<Aside type="tip">
A section named **Optional** has special meaning: it marks links a model can skip when it needs a shorter context. Put secondary material there.
</Aside>

## Companion conventions

Two related conventions often accompany an `llms.txt` file:

- **Markdown versions of pages.** The proposal recommends serving a clean `.md` version of each page at the same URL with `.md` appended (for example, `quickstart.md`). The links in `llms.txt` point to these Markdown versions so models receive content without site chrome. See [Serve a Markdown version of every page](/guides/serve-markdown/).
- **`llms-full.txt`.** Many tools also generate an `llms-full.txt` file that inlines the full text of the documentation in one file, rather than linking out. This is a widely adopted companion to `llms.txt`, not part of the original specification.

## Adoption: a realistic view

`llms.txt` is a community proposal, not an official or ratified standard, and adoption is still evolving.

- A growing number of documentation platforms generate the file automatically, and many sites now publish one.
- Support among AI vendors is not universal, and there is ongoing public debate about how many major systems actually read `llms.txt` today versus crawling sites directly.

<Aside type="caution">
Treat `llms.txt` as low-effort and forward-looking rather than guaranteed. Publishing one is cheap and positions your docs well, but do not assume a specific assistant reads it. Avoid promising that adding the file changes how any particular AI product behaves.
</Aside>

## Next steps

- [Add an llms.txt file to your docs](/guides/llms-txt/add-llms-txt/) — create, place, and maintain the file.