Yes, for one audience, and no for the audience most people asking have in mind. llms.txt works for coding agents reading developer documentation, which is why OpenAI, Stripe and the AI labs maintain theirs by hand. llms.txt does nothing for a business trying to be cited in an AI answer, and Google Search says so in writing. The more useful question is what is actually inside these files, and almost nobody has looked.
The short version:
- Google Search’s own guidance says creating one will “neither harm nor help your site’s visibility or rankings in Google Search, as Google Search ignores them“
- Ahrefs read the server logs of 137,210 domains: 28% publish an llms.txt, 97% of those files received zero requests in May 2026, and no requests arrived for files that do not exist
- The “the evidence says it does nothing” articles published in August 2026 rest partly on a study dated 7 November 2025 — before Google’s Lighthouse audit existed and before version 2 of the specification
- I read nine llms.txt files line by line. Every one written by a person is good. Every one generated by a plugin is broken, each in a different way
- All in One SEO’s own file breaks the specification in its first line, runs to roughly 3,529 links, and never mentions the plugin the company sells
- Rank Math ships an llms.txt generator to four million sites and publishes no llms.txt of its own
- Google’s Lighthouse gives the file two seconds to respond. A correct file can fail on speed while a wrong one passes
- Yoast put a discount code inside their llms.txt “to help us measure interest” — the leading SEO plugin vendor does not know whether anything reads it either
- My own llms.txt listed nine pages, carried no descriptions at all, and did not mention my blog. It passed Google’s check
What is llms.txt, and what does the specification actually require?
llms.txt is a plain-text Markdown file at the root of a website that lists the pages worth reading, for the benefit of AI agents rather than search engines. Jeremy Howard proposed it in September 2024. The current specification is version 2, modified on 10 August 2026. The bar it sets is extremely low: a single H1 heading is the only required element in the entire file.
A file that follows the specification properly contains, in this order: an H1 with the name of the site, a blockquote summarising what the site is, optional paragraphs of context, then H2 sections holding lists of links. Each link takes the form [name](url): notes — and those notes after the colon are the whole point, because a list of URLs without them tells an agent nothing it could not get from a sitemap.
llms.txt is not robots.txt, and it is not a sitemap
These three files get confused constantly, and the confusion makes the rest of the argument impossible to follow. robots.txt tells crawlers which paths they may fetch, and the major crawlers obey it. An XML sitemap lists every indexable URL on a site, and search engines genuinely consume it. llms.txt is neither enforced nor comprehensive: it is a curated, advisory reading list, and the evidence below shows it is almost entirely unread.
What changed in version 2
Version 2 of the llms.txt specification, published on 10 August 2026, made four changes worth knowing. Sites can now advertise Markdown versions of pages using standard HTML link relations rather than a naming convention. Markdown URLs may either append .md or replace the existing extension. llms.txt files can live in subpaths, where the most specific file wins. And the expansion tool of version 1 was dropped in favour of agents reading the file directly and following the links they need.
Most of the sceptical articles about llms.txt were written before those changes or have not registered them, which matters because several of them describe behaviour the specification no longer asks for.
Does anything actually read llms.txt?
Almost nothing reads llms.txt, and the best available evidence comes from server logs rather than opinion. Ahrefs examined 137,210 domains in May 2026 and found that 97% of the llms.txt files those sites published received no requests at all. Of the requests that did arrive, 96% came from bots and 19.5% from named AI tools. No requests at all arrived for files that do not exist.
What 137,210 domains’ server logs show
The Ahrefs study, published on 15 June 2026 by Louise Linehan and Xibeijia Guan, is the strongest evidence in this argument because it measures requests rather than correlations. Two of its own caveats travel with it and should not be dropped. The authors state that their users skew more technical than the web at large, so their 28% adoption figure is an upper bound. They also point out that a file being fetched is not the same as a file being read, which makes their numbers a ceiling on consumption rather than a measurement of it.
The finding I keep returning to is the quietest one: no AI bot requested an llms.txt from a site that did not have one. Nothing is out there speculatively checking. A file only ever gets fetched by something that already knew to look, which tells you how much of the current discussion is about a file almost nobody is asking for.
Why the “it does nothing” articles are quoting a 2025 study
SE Ranking analysed roughly 300,000 domains and found no correlation between having an llms.txt and being cited by AI engines — their model actually improved when llms.txt was removed as a factor. It is good work and the conclusion is probably right. The study is dated 7 November 2025. That is before Google’s Lighthouse llms.txt audit existed and before version 2 of the specification, and none of the August 2026 articles leaning on it mention the date.
This is not a reason to dismiss the finding. Naming the date is simply what separates a claim you can defend from one you are repeating.
The one audience that does use it
Coding agents reading developer documentation are the exception, and they are the reason the file exists at all. The specification’s own authors say the file is meant for inference rather than training, consulted on demand when an agent needs information while helping a user, and note that coding agents use them reliably. Ahrefs’ 19.5% figure for named AI tools points the same way. That is why OpenAI, Stripe and the AI labs maintain theirs carefully while almost nobody else’s file is ever fetched.
What does Google say about llms.txt?
Google says two different things depending on which product team you ask, and both are true. Google Search’s guidance on generative AI features states that site owners “don’t need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search”, and that doing so “will neither harm nor help your site’s visibility or rankings in Google Search, as Google Search ignores them”. Chrome, meanwhile, audits for the file.
Chrome’s Lighthouse documentation describes llms.txt as “an emerging convention used to provide a machine-readable summary of a website’s content, specifically designed for LLMs and AI agents”, and has checked for one since May 2026. I went through what that audit does and does not test in what Agentic Browsing in PageSpeed Insights actually checks, including why a missing file costs you nothing. The short version for here: the audit checks the format of a file you already have, not whether you have one.
What is actually inside the llms.txt files on the web?
I read nine llms.txt files line by line on 16 and 17 September 2026, and the pattern was clean enough to be uncomfortable: every file written by a person follows the specification, and every file generated by a plugin breaks it. Nine out of nine, no exceptions. Every URL below is public and at a site’s root, so any reader can check the same files in about ten minutes.
| Site | Who wrote it | H1 first | Blockquote | Notes on entries | Follows the spec |
|---|---|---|---|---|---|
| developers.openai.com | By hand | Yes | Yes | Yes, after a colon | Yes |
| stripe.com | By hand | Yes | Yes | Yes, after a colon | Yes |
| semrush.com | By hand | Yes | Yes | Yes, after a colon | Yes |
| yoast.com | By hand | Yes | Yes | Yes, after a colon | Yes |
| ai.google.dev | Generated | Yes | No | Yes, after a colon | Partly |
| platform.claude.com | Generated | Yes | No | No link list at all | Partly |
| aioseo.com | Its own plugin | No | No | Hyphens, not colons | No |
| rankmath.com | — | — | — | — | No file at all |
| theshiva.co (mine, before) | Yoast plugin | Yes | Yes | None | Partly |
The plugin vendor whose own file breaks the specification
All in One SEO publishes an llms.txt on aioseo.com generated by their own Pro plugin, and it goes wrong in the first line. The specification puts the H1 first; All in One SEO prints “Generated by All in One SEO Pro v5.0.0.1, this is an llms.txt file, used by LLMs to index the site.” above the H1. There is no blockquote. The sitemap entry uses the specification’s colon, and every other entry uses a hyphen instead, so one file carries two separators and only one of them is right.
The more consequential problem with All in One SEO’s file is what it covers. The file has two sections: one sitemap link, and roughly 3,529 links to blog posts. There is no product section, no pricing section and no features section anywhere in it. The company sells a WordPress plugin, and its own machine-readable summary describes it as a publisher of blog articles.
The descriptions in All in One SEO’s file are present but were not written for this job. They are the posts’ meta descriptions, lifted whole, marketing voice intact — “Dominate SEO with AIOSEO’s Expert Blog!”, “Unlock the secrets of SEO with our in-depth beginner’s guide”. Copy written to win a click from a human in a search result, doing duty as a description of what a page contains.
The plugin vendor with no file at all
Rank Math ships an llms.txt generator and publishes no llms.txt on rankmath.com. The URL returns a 404 page, and the site header above that 404 reads “4 Million+ Happy Users”. I checked twice on 17 September 2026 in case of a caching accident. Rank Math’s generator is also the one named in the Lighthouse bug report further down this page, which makes the absence more interesting rather than less.
The three AI labs write it three different ways
If anyone should agree on the format, it is the companies whose models the file exists to serve. OpenAI’s file is the best one I read: a proper blockquote, sections, a description on every entry, and a separate llms.txt for each product rather than one enormous list. Google’s Gemini file has an H1 and descriptions but no blockquote, across roughly 230 flat entries. Anthropic’s has no blockquote and no link list at all — prose, and a heading called “Root URL”. The specification’s own link to Anthropic’s file points at a URL that now redirects somewhere else.
And one file I cannot show you
One client site in this set is a professional services firm of around 200 pages, and its llms.txt is the clearest example of the whole problem. The file is auto-generated by an SEO plugin. It lists blog posts only — not one service page, not one person’s profile — so it describes a firm that sells expertise as though it were a magazine. At least four entries carry a description belonging to an entirely different post. It passes Google’s Lighthouse check without complaint.
I am not naming that client, or their country, or showing their file, and the same applies to every client example on this site. What I can say is that their file and All in One SEO’s have the same shape, produced the same way, for the same reason.
What does my own llms.txt look like?
Mine went through three versions, and only the third is worth having. Version one was whatever Yoast generates on its own. Version two was Yoast in manual mode, configured by me, which quietly removed my entire blog. Version three is hand-written and served from a code snippet. I ran all three on my own site in September 2026, and the differences are larger than the settings screen suggests.
| Version | Who chose the contents | Blog posts included | Descriptions | Passes Google’s check |
|---|---|---|---|---|
| 1 · Yoast, automatic | The plugin | Yes, plus a categories section | None | Yes |
| 2 · Yoast, manual | Me | No, and it cannot be done | None | Yes |
| 3 · Hand-written snippet | Me, with the article list automated | Yes, added on publication | On every entry | Yes |
Version one: what Yoast produces on its own
Yoast’s automatic mode builds a reasonable skeleton without being asked. Switching it on produced four sections — Pages, Posts, Categories and Optional — with both published articles under Posts and the AI Visibility category under Categories. Yoast’s documentation says automatic mode takes the five most recently updated pages, posts and custom post types with cornerstone content prioritised, and that is broadly what appeared.
What automatic mode does not produce is a single description. Every entry is a bare link carrying the page’s SEO title as its text, pipes and ampersands included, because Yoast writes a description only when a page has a custom excerpt and none of mine had one. The file also announces which version of the plugin generated it, on a line that cannot be removed through the usual WordPress filter for hiding version numbers.
Version two: manual mode, and the mistake was mine
I switched to manual selection and added my nine service pages one at a time, and in doing so I deleted my blog from the file without knowing it. Yoast’s manual picker only offers pages. Search it for one of my own post titles and it returns “No pages found”. Choosing manual selection is therefore also choosing to have no posts and no categories in your llms.txt, and nothing in the interface says so.
Manual mode is the version that sat on my site while I wrote this article. It listed nine pages, carried no descriptions, and did not mention the blog I launched on 14 September or either post on it. It passed Google’s Lighthouse llms.txt audit the whole time, because that audit checks whether the file can be fetched and parsed, not whether it is true.
The uncomfortable part is that manual mode is the careful choice. I chose it because I wanted control over what an AI agent saw, and the automatic setting I overrode was the one that would have included my writing.
Version three: hand-written, with the volatile part automated
The file live now is produced by a small code snippet rather than by the SEO plugin. It has four sections of my own naming, a written description after every entry saying what is on that page and who it is for, and prices stated as plain facts. The parts that rarely change — services, about, the sitemap — are written by hand. The list of articles is built automatically from the latest published posts, so the failure I have just described cannot happen twice.
That split is the recommendation, and it is the only part of this article I would defend as advice. A hand-written file says something useful and then rots, because nobody remembers it. A generated file never rots and never says anything. Writing the stable parts and generating the volatile ones is the arrangement that survives a real publishing schedule.
Can a correct llms.txt fail Google’s check?
Yes, and on something almost nobody is looking at: speed. Google’s Lighthouse gives the llms.txt fetch a two-second budget and fails anything slower. In June 2026 a site owner reported that a spec-compliant file returning HTTP 200 was being marked as failed. A Lighthouse maintainer answered plainly: “we use a timeout of 2s for these fetches, but this llms.txt takes more.”
The file in that report measured 2.342 seconds. It was being generated on every request by a WordPress plugin assembling 882 links from the database, with no caching at the edge. Once the site owner cached it, the response fell to around 119 milliseconds and the site scored 3/3. Google declined to raise the timeout or make it configurable, on the reasonable grounds that a slow response is a real problem worth surfacing.
So the audit that everyone treats as a check on llms.txt quality is, in its only failure state, a check on your hosting. A slow correct file fails. A fast wrong file passes. Mine passed throughout, while listing nine pages and describing none of them.
How can I tell whether anything reads my own llms.txt?
Put something in the file that exists nowhere else, and watch whether it is ever used. Yoast did exactly this, and it is the most honest thing I found in a week of reading these files. Inside yoast.com/llms.txt there is a discount section offering 15% off, with the line: “This code is unique to this listing to help us measure interest.”
Read that in context. The company that shipped the first llms.txt generator in an SEO plugin does not know whether anything consumes the file, so rather than guess, they put a unique token inside it and waited to see. That is not an embarrassment, it is the correct response to an open question, and you can copy it this afternoon.
The method, in three steps:
- Put a token in the file that appears nowhere else — a discount code, or a link carrying campaign parameters that no other page on your site uses. The link in my own file is my contact page with
utm_source=llms.txtattached - Check your server logs for requests to /llms.txt, and note which user agents arrive. This is the measurement Ahrefs made at scale and you can make on one site
- Wait, and check again at 30 and 90 days. A file nothing requests is not a file worth maintaining carefully
One qualifier, because it matters. A public text file can be read by a person as easily as by an agent, and a discount code in a public file can end up on a coupon site. A token that gets used is an indication that something read the file, not proof that an AI agent did. Treated that way it is still far better evidence than anybody’s opinion, including mine. I am running this on my own site now and will report what it shows. If you want the wider version of this approach, I set it out in how to measure whether AI actually cites your business.
When is an llms.txt worth writing?
An llms.txt is worth writing when your buyers arrive through a coding agent, and is close to pointless when they arrive through a recommendation. The distinction is not about company size or industry, it is about which system answers the question your buyer asks. That system is a directory, a Google Business Profile, a marketplace listing or a third-party list far more often than it is an agent reading files at your domain root.
| Your situation | Is llms.txt worth the effort? | Why |
|---|---|---|
| You publish developer documentation or sell an API | Yes | Coding agents genuinely fetch and follow these files. This is the audience the format was designed for |
| You run a documentation-heavy software product | Probably | Cheap to maintain, and your users are the ones using agents |
| You are a local service business, a law firm, a tour operator | No | Your buyers are answered from directories, Google Business Profiles and third-party lists. None of them reads this file |
| You already have one because a plugin made it | Fix it or remove it | It costs nothing to be correct, and a file describing the wrong half of your business is worse than none |
What llms.txt cannot do is worth stating plainly, because it is sold as though it can. An llms.txt will not get you cited by ChatGPT, Claude or Google’s AI Mode. It will not affect your Google rankings — Google Search ignores it. It will not add your site to a model’s training data. And it will not put you on a shortlist when somebody asks an assistant to recommend three businesses like yours. The difference between training data and the live search that produces citations is set out in how AI engines actually find, read and cite a website.
What should I do about llms.txt today?
If a plugin has already made you one
Open yourdomain.com/llms.txt and read it, which will take two minutes and is the only part of this that is not optional. Check four things: that the H1 is the first line, that a blockquote summarising the business follows it, that every entry has a real description after a colon, and — the one that catches almost everybody — that the file describes what you sell rather than what you last published.
If you do not have one
Nothing is wrong. A missing llms.txt is marked Not Applicable by Google’s Lighthouse rather than failed, so it costs you no score, and Ahrefs found that no bot goes looking for files that do not exist. If your audience is developers, write one properly. If it is not, this belongs near the bottom of your list, below anything that affects how your pages themselves are written.
What not to do
Do not buy an llms.txt as a deliverable, and treat anyone selling one as the core of an AI visibility offer as having told you something useful about their depth. Do not rebuild a site for agents on the strength of this file. And do not let a plugin generate one and assume the job is done, which is precisely the mistake I made on my own site while writing about other people making it.
Your questions, answered
Does llms.txt help my business get cited by ChatGPT?
No. There is no published evidence that ChatGPT consults llms.txt when answering a question about a business, and SE Ranking’s analysis of roughly 300,000 domains in November 2025 found no correlation between having the file and being cited. Being cited is decided by what is on your pages and what other sites say about you, not by a list of links at your domain root.
Does Google use llms.txt?
Google Search does not. Google’s own guidance states that creating one “will neither harm nor help your site’s visibility or rankings in Google Search, as Google Search ignores them”. Chrome’s Lighthouse does check the file as part of its Agentic Browsing category, which is a different product team answering a different question, and neither position contradicts the other.
Do I need an llms-full.txt as well?
No. Version 2 of the llms.txt specification does not require one, and the current proposal is that pages offer clean Markdown versions of themselves rather than that a site publish one enormous combined file. If almost nothing is fetching your llms.txt, a larger second file will not be fetched either.
Will a broken llms.txt hurt my site?
A broken llms.txt will not hurt your Google rankings, because Google Search ignores the file entirely. It can fail Google’s Lighthouse llms.txt audit, which happens when the file returns a server error or takes longer than two seconds to respond. The real cost is different: a file that describes the wrong half of your business is a misleading summary sitting at a predictable URL.
Should I let my SEO plugin generate my llms.txt?
Only if you then read what it produced. Every plugin-generated file in the set I examined was defective, including the ones published by the plugin vendors themselves on their own websites. A generator can pick your pages; it cannot know which of them matter or describe what is on them.
Is fixing llms.txt the same as an AI visibility audit?
No, and the gap between the two is large. An AI visibility audit establishes why AI engines do or do not cite a business, which sources they draw on instead, and what would change that. llms.txt is one file, it takes an afternoon, and on most sites it is not the reason the engines are not citing you.
The short of it
llms.txt is a reasonable idea with one real audience and a great deal of noise around it. Coding agents reading developer documentation use these files; almost nothing else does, and Google Search says in writing that it ignores them. The argument about whether the format works has been conducted almost entirely without anyone opening the files, and the files turn out to be the interesting part: written by a person they are careful and useful, generated by a plugin they describe the wrong business, and they pass Google’s check either way.
My own file did all of that for three days while I was writing this. If you have one, go and read it. That is the whole recommendation.
