llms.txt generator

What is llms.txt?

llms.txt is a plain-text file you place at the root of your website — https://yoursite.com/llms.txt — that tells large language models which pages on your site matter and what each one contains. It was proposed by Jeremy Howard in 2024 and the format is documented at llmstxt.org.

The problem it solves is simple. When an AI assistant tries to answer a question about your business, it has to work out which of your pages to read. HTML is full of navigation, cookie banners, scripts and boilerplate; context windows are finite. An llms.txt gives the model a curated map instead — here are the pages, here is what each one is about, in that order of importance.

How is it different from robots.txt or sitemap.xml?

They answer different questions. robots.txt says what a crawler is allowed to fetch. sitemap.xml lists every URL that exists, in machine-readable XML, with no indication of which ones matter. llms.txt is editorial: it is a short, human-readable, prioritised summary written for something that is going to read and reason about your content rather than index it.

What does the format look like?

The spec is deliberately minimal. An H1 with your site name, an optional blockquote summary, then H2 sections containing lists of links with short descriptions:

# Acme Bakery

> Independent bakery in Leeds supplying sourdough
> and pastries to cafés across Yorkshire.

## Main
- [Home](https://acme.example/): What we bake and who we supply
- [Wholesale](https://acme.example/wholesale): Trade ordering and lead times
- [Our bakers](https://acme.example/team): The people behind the bread

## Optional
- [Privacy policy](https://acme.example/privacy)

The ## Optional section is part of the spec and has a specific meaning: those URLs can be skipped when a shorter context is needed. Our generator puts legal and navigational pages there automatically.

Does it actually do anything yet?

Honest answer: adoption is still early. No major search engine has committed to using llms.txt as a ranking input, and you should be sceptical of anyone claiming otherwise. What is true is that the file costs nothing to publish, several AI developer tools and documentation platforms already read it, and it forces a genuinely useful exercise — deciding which twenty pages on your site actually explain what you do.

Where do I put it?

The root of your domain, served as text/plain: https://yoursite.com/llms.txt. That is the same place your robots.txt lives, and the same place an IndexNow key file goes — so if you can do one, you can do the others.