manifesto.md

Conventionv2Agentic StackRepo Conventions

The /llms.txt file (llms.txt)

A proposal for a Markdown file at /llms.txt that gives language models a curated overview of a website and links to clean Markdown versions of its pages.

Canonical text: https://llmstxt.org/, read against version 2. In use on this site: manifesto.md/llms.txt.

The proposal

  1. We propose adding a /llms.txt markdown file to websites to provide LLM-friendly content.
  2. Context windows, while larger than they were, are still too small for most websites in their entirety.
  3. The llms.txt file is unusual in that it uses Markdown to structure the information rather than a classic structured format such as XML.
  4. An H1 with the name of the project or site. This is the only required section.
  5. A blockquote with a short summary of the project, containing key information necessary for understanding the rest of the file.
  6. Zero or more markdown sections (e.g. paragraphs, lists, etc) of any type except headings.
  7. Zero or more markdown sections delimited by H2 headers, containing file lists of URLs where further detail is available; each item is a markdown link, optionally followed by a colon and notes.
  8. The "Optional" section is used, by convention, for secondary information that can be skipped if a shorter context is needed.
  9. Provide a clean markdown version of those pages at the same URL as the original page, but with .md appended.
  10. llms.txt is designed to coexist with current web standards.
  11. While sitemaps list all pages for search engines, llms.txt offers a curated overview for LLMs.
  12. It can complement robots.txt by providing context for allowed content.

Facts

Maintainer
Jeremy Howard
Repository
https://github.com/AnswerDotAI/llms-txt
Licence of the text
Apache-2.0
First published
2024-09-03
Status of the text
stable
Listing
listed, not yet claimed

Read with

Badge

manifesto.md: listed

[![manifesto.md](https://img.shields.io/endpoint?url=https://manifesto.md/badge/llms-txt.json)](https://manifesto.md/llms-txt/)

Maintainers claim a listing with a pull request setting listing: claimed, or a yes on the listing issue.