Last updated
What is llms.txt?
llms.txt is a proposed standard: a markdown file at the root of a website that gives language models a clean, structured summary of the site and links to its most useful pages. It was proposed in late 2024, and no major AI provider has publicly confirmed that it reads the file.
What it contains
A heading with the site's name, a one-line summary of what the organisation does, and then curated sections of links with a short note on each. The point is to hand a model the map a human would draw, rather than making it infer the structure from navigation menus and scripts.
How it differs from robots.txt
robots.txt is a set of prohibitions: which crawlers may fetch which paths. llms.txt is the opposite gesture, an invitation that says here is what matters and here is how it fits together. The two are unrelated files serving unrelated purposes, and one does not substitute for the other.
The honest position on whether it works
There is no public confirmation from OpenAI, Anthropic, or Google that they consume it, and at least one prominent search engineer has compared it to the old keywords meta tag. It costs almost nothing to generate from content a site already has, so the case for it is cheap optionality rather than demonstrated return. Anyone selling it as a proven ranking factor is ahead of the evidence.
Start with the free audit.
Get a free auditSend the link. We read the site by hand and write back inside five business days.
No call, no commitment