llms.txt: everyone ships it, no engine admits reading it
No proposed standard in search has been adopted faster on thinner evidence than llms.txt. Nearly one in eleven of the web's biggest sites now serves one. Google says, on the record and repeatedly, that nothing reads it. Both facts are measurable, both are current, and the gap between them is the story.
What llms.txt actually is
The llms.txt proposal, from Answer.AI's Jeremy Howard in September 2024, is a markdown file at your site root that hands language models a curated map: what the site is, which pages matter, where the clean text lives. The pitch is reasonable. Context windows are finite, HTML is noisy, so give the model a menu instead of making it crawl the kitchen. The question was never whether the idea is tidy. It is whether anything on the other side actually reads the menu.
The adoption is real
Rankability's June 2026 crawl of the Tranco top 1,000 found 87 sites, 8.7%, serving a valid llms.txt. For a voluntary standard with no confirmed consumer, that is remarkable uptake. The trouble starts when you look at the server logs on the other side of those files:
The evidence is not
The public record from the engines is unusually blunt. Google compared the format to the keywords meta tag in April 2025, confirmed that no AI system currently uses it in June 2025, and closed the loop this June: the file neither helps nor harms your rankings. The behavioral data agrees with the statements: in Ahrefs' study of live server logs, 97% of llms.txt files had never been requested by any bot at all.
One wrinkle keeps the file from being a pure cargo cult. OpenAI's crawler has been observed fetching llms.txt files, without any statement about what, if anything, the fetches feed. And OpenAI itself publishes an llms.txt for its own developer docs, aimed at coding agents that read documentation, not at search ranking. That is probably the honest future of the format: a docs index for agents that were explicitly pointed at it, rather than a ranking input any engine discovers on its own.
What to do with it
Treat llms.txt like a business card in a bowl at a conference: nearly free to leave, unlikely to be picked up, occasionally exactly how someone finds you. If you run developer documentation, ship one, because agents get pointed at docs deliberately and the format genuinely helps there. For everyone else, generate it once, never let it displace an hour of work on the things engines measurably reward: liftable passages, evidence density and format, the record on which is covered in our entity-first brief. And if a vendor is charging you monthly for llms.txt optimization, the server logs above are the only due diligence you need.