Most SaaS sites do not need llms.txt yet. It is a proposed Markdown file that lists a site's key pages for AI models, but no AI service has confirmed using it, and Google says no special AI file is required. Add one if you run substantial documentation; otherwise fix crawl access first.
That verdict holds for sites judged on rankings and AI citations. It changes when you publish large docs, an API reference or a help centre that AI assistants are likely to be asked about.
The file costs about an hour to make, so adding one is cheap. Nothing in the sources shows it changes whether AI systems cite you.
If you would rather have someone check how AI systems read your site before you touch any files, our LLM SEO service covers llms.txt, crawler access and entity consistency.
What Llms.txt Is And Where It Came From
llms.txt is a proposed plain Markdown file, published at a site's root, that gives AI models a short, curated list of the pages that matter most on that site. The llmstxt.org proposal, dated 3 September 2024, asks sites to publish it at a fixed path as a concise guide for AI agents.
Think of it as a hand-written table of contents for machines. A normal website is built for people. It has navigation, scripts, banners and pages that mean little without context. The file strips that away and says, in plain text, "these are the pages to read, and this is what each one is for."
The proposal describes the file as something a model uses on demand, at the moment it answers a question. It also says training runs could use it. That is the whole pitch. The file does not change your pages, it does not block anything, and it does not rank anything. It is a pointer.
The format is deliberately simple. The file is named llms.txt and sits at the site root or a subpath. The only required element is an H1 with the site name. Everything else is optional, including a blockquote summary, prose and H2 sections of links.
One thing to be clear on early. This is a proposal. Publishing it does not make you compliant with anything, because there is nothing to comply with. If you want the wider frame for why sites are doing this at all, start with what AI search visibility means and come back.
Do Any AI Systems Actually Read Llms.txt?
No AI service has publicly confirmed using llms.txt. That is the finding that matters, and most guides skip it because it spoils the pitch.
Start with the proposal itself. It names no crawler or provider that is required to read the file. There is no list of bots that promise to fetch it, and no commitment from any AI company inside the document. A format that nobody has agreed to read is a hopeful format.
Then look at what Google has said. Google's AI features documentation, updated on 10 December 2025, states that site owners do not need to create new machine-readable files, AI text files or markup to appear in AI Overviews or AI Mode. That is Google's own guidance for its own AI features, and it is plain.
On 17 April 2025, Google's John Mueller compared llms.txt to the keywords meta tag, said no AI services had confirmed using it, and said server logs suggest bots do not check for it.
So what should you conclude? Not that llms.txt is useless, and not that it is a secret lever. Conclude that the evidence is thin. Two dated statements from Google and a proposal that names no reader. We make no claims here about any other AI provider fetching or publishing the file, because nothing we can verify supports one.
If you see a guide stating that a named crawler reads llms.txt, ask for the date and the source. If neither exists, the claim is a guess. The honest position is that you would be adding the file on speculation, which is fine when the cost is an hour and a bad idea when it replaces real work. For the work that does move AI answers, look at our Generative Engine Optimization service.
Llms.txt Vs Robots.txt Vs Sitemap.xml
People confuse these three files because all of them sit at the site root and all of them mention URLs. They do different jobs for different readers.
| File | Job | Who it speaks to | Status |
|---|---|---|---|
| robots.txt | Controls which URLs crawlers may fetch | Crawlers | Formal standard, RFC 9309, dating from 1994 |
| sitemap.xml | Lists the URLs a site wants search engines to know about | Search engines | Search engine convention |
| llms.txt | Curates a short list of key pages with descriptions | AI models and agents | Proposal, no formal standard |
robots.txt controls crawling; llms.txt curates content. They work on different layers, so one does not replace the other.
That has a practical consequence. llms.txt cannot block anything. If you want an AI crawler kept out, that is a robots.txt decision, and a robots.txt block on an AI crawler takes precedence over anything you list in llms.txt. Listing a page in the file does not invite a crawler you have already turned away.
Keep one more point in mind. robots.txt is a crawl directive, not a security boundary. It asks well-behaved bots to stay out. It does not protect anything private. If a page must not be read, put it behind a login, and keep login-walled pages out of llms.txt as well.
A sitemap is closer in spirit to llms.txt, since both list pages. The difference is scale and intent. A sitemap is exhaustive and written for search engines. An llms.txt file is short, selective and written with descriptions for a model that has to choose.
What Goes In An Llms.txt File
Here is an illustrative file for a fictional analytics product. The brand and every URL are made up and use example.com only. Copy the shape, not the content.
# Acme Analytics
> Acme Analytics is a web analytics platform for B2B SaaS teams. This file lists the pages that explain what the product does, how to set it up and what it costs.
The documentation is written for developers and marketers who already use the product.
## Documentation
- [Getting started guide](https://example.com/docs/getting-started): Install the tracking snippet and verify your first event.
- [API reference](https://example.com/docs/api): Endpoints, authentication and rate limits.
- [Integrations overview](https://example.com/docs/integrations): Supported CRMs and data warehouses.
## Product And Pricing
- Pricing: Plans, limits and billing terms.
- [Security and trust centre](https://example.com/trust): Data handling, certifications and subprocessors.
## Optional
- [Company blog](https://example.com/blog): Product news and long-form articles.
- [Changelog](https://example.com/changelog): Release notes by date.
The H1 with the site name is required; everything else in the file is optional.
In order, the structure runs like this. The H1 names the site. A blockquote gives a short summary. Optional prose adds context. H2 sections group the links. A section named Optional marks content a model can skip when it is short on space.
Each link is a title and a URL, with an optional description. Descriptive titles work better than generic ones, so "API reference" beats "Docs". The description should say what the reader gets, in a few words.
Include: The pages a model would need to answer a buyer's questions accurately. Documentation, pricing, integrations, key guides and the trust centre are the usual candidates. If you are writing those guides, write them to be quoted, as we cover in our piece on AEO-ready SaaS blog content.
Leave out: Login-walled pages, generic titles and copied navigation. A model cannot read a page behind a login, and a list that repeats your menu adds nothing a crawler could not find itself.
On size, the file is a curated list, not a full sitemap. Guidance puts a typical file at roughly twenty to forty links. A file with several hundred entries defeats the purpose, because the model gets no help deciding what matters. If you cannot cut your list down, you have not yet decided what your site is for.
Does Llms.txt Help SEO?
No. llms.txt is not a ranking factor. It will not directly help classic Google rankings, and Google says site owners do not need new machine-readable files or AI text files to appear in its AI features. Google's John Mueller said in April 2025 that no AI services had confirmed using the file.
The only upside is on the generative side. If some AI system does read the file, a clean list of your best pages could make it easier for that system to find and quote them. That is possible, and it is unproven. Do not put it in a forecast.
It also helps to separate two goals that get blurred. Ranking in classic results and being cited in an AI answer are different outcomes with different levers. We break the difference down in how AISO differs from classic SEO.
The levers that do move AI answers are mostly the old ones done well. Clear pages, accurate facts, honest sourcing and visible expertise. If you want to know whether E-E-A-T still matters now that answers are generated, that is the better place to spend your reading time than a text file.
Does Your SaaS Site Need Llms.txt? A Decision Rule
Fix four things before you think about the file. Crawl access, so the bots you want can actually fetch your pages. Text availability, so your content is real text rather than something hidden behind scripts. Internal linking, so key pages are reachable. Governance, so someone owns what the site says.
Then apply this rule. Add llms.txt when you publish substantial documentation, an API reference, a trust centre or a support hub, and inaccurate AI answers about those pages would cost you pipeline or support load. Pilot it when your site has some documentation but the case is not strong, and run it as a contained test with a before-and-after check. Skip it for now when your site is a brochure of a few pages.
| Site type | Call | Reason |
|---|---|---|
| SaaS with large documentation or an API reference | Add | Wrong AI answers about setup or limits cost support time and trust |
| SaaS with a trust centre or support hub | Add | Buyers ask AI assistants about security and policy, and errors affect pipeline |
| SaaS with a blog and a small docs section | Pilot | Some value is possible, so test it cheaply and measure the result |
| Brochure-style site of a few pages | Skip for now | Few pages to curate, and a crawler finds them without help |
| Any site with blocked crawlers or script-hidden text | Fix first | The file cannot repair access problems underneath it |
Be honest about what this is. The file is a low-cost enhancement for sites with substantial documentation. It is not a ranking tool. It matters most for complex B2B sites, where an inaccurate AI answer affects pipeline or support, and least for brochure sites. Treat it as a cheap bet with unproven upside, never a proven lever.
If the rule says skip, do not feel behind. Put the hour into an AEO strategy for B2B SaaS instead, or into our Answer Engine Optimization work if you want help being cited as the answer.
If this decision is one piece of a bigger question about where AI search fits your SEO plan, the GEO + SEO Growth Programme is how we run it. We start with whatever stops your key pages being read, then decide which files are worth the time.
How To Create And Publish Llms.txt
The job takes about an hour for a small site. Most of that hour is deciding what to include, not typing.
- Decide which pages to feature. List the pages a buyer or a support agent would need to get a correct answer. Cut the list to roughly twenty to forty. Drop anything behind a login.
- Write the llms.txt file in Markdown. Start with an H1 and the site name, add a one-paragraph blockquote summary, then group links under H2 headings. Give every link a descriptive title and a short description.
- Upload the file at the site root. The path is yoursite.com/llms.txt. The proposal also allows a subpath, but the root is the sensible default.
- Check that the file loads as plain text. Open the URL in a browser. You should see raw Markdown, not a styled page or a 404. Confirm every link inside it resolves.
- Add the file to your content release checklist. Each time you retire, rename or add a key page, review the list. A file that is never revisited goes stale.
Generators and validators exist for this. A generator can draft the file from your sitemap or page list, which saves time on a large site. A validator can catch formatting slips. We do not review or rank any of them here. Neither tool can decide which pages deserve a place, and that curation is the part that takes thought. For a site that fits the add or pilot rows above, writing it by hand is often just as quick.
Llms-Full.txt And Markdown Versions Of Pages
llms-full.txt is a companion file name used alongside llms.txt. Where llms.txt is a short index of links, llms-full.txt is the idea of putting the full text of those pages in one place.
A related practice is serving pages as plain Markdown, so a model can read a clean version without the page furniture. Both are optional. Both are general practice rather than anything a named vendor has promised to use, and neither is better evidenced than llms.txt itself. If you cannot yet justify the main file, skip these.
How To Check Whether Llms.txt Is Doing Anything
Publishing the file is the easy part. Knowing whether it did anything is where almost every guide goes quiet. Three checks cover it.
- Run the Lighthouse audit. Chrome's Lighthouse includes an llms.txt audit under its agentic browsing documentation, with sections on how the audit works and how to fix a failure. A pass does not tell you any AI system used the file, and its fix guidance points to the spec and gives no example file.
- Read your server logs. Search the logs for requests to /llms.txt and note which user agents asked for it. Requests from a known AI bot would be the first real evidence that your file is being fetched. Requests only from your own tools or random scanners tell you much less.
- Test a fixed set of buyer questions. Pick ten questions your customers ask. Run them in the AI assistants you care about before you publish, record which pages get cited, then run them again after a few weeks. Compare the cited pages. Our guide on how to audit your brand's visibility on LLMs lays out the manual method.
A log with no requests for /llms.txt is a valid result. It is also the answer you may well get, given Mueller's remark that server logs suggest bots do not check for the file.
If you want site health and rank tracking in one place before you blame a missing file for a visibility problem, Semrush covers the site audit and tracking side of this work.
Running the before-and-after test by hand gets tedious across many questions. If you want tooling to track the result, read how to choose an AI visibility platform, or start from the shortlist of tools that monitor SaaS brand visibility.
The Limits And Risks Of Llms.txt
The main risk of llms.txt is false confidence. A team publishes the file, ticks the box and stops looking, while three real problems stay hidden. Crawlers are still blocked. Key content still sits behind scripts. Thin pages still say too little to be quoted.
There are quieter risks too. A stale file pointing at retired pages is worse than no file, because it sends a reader to dead ends. The file adds a maintenance duty that somebody has to own. And curation is a judgement call. Leave out a page that matters, and the list quietly steers attention away from it.
None of this is a reason to avoid the file. It is a reason to keep it small, owned and reviewed, and to keep fixing the basics underneath it.
FAQs
Is Llms.txt An Official Standard?
Where Should The Llms.txt File Live?
Does Llms.txt Block AI Crawlers?
Should A Small SaaS Site Create Llms.txt?
Can Llms.txt Hurt My Site?
How Many Links Should An Llms.txt File Have?
Do I Need Llms-Full.txt As Well?
Do I Need An Llms.txt Generator?
How Often Should I Update Llms.txt?
Still unsure whether llms.txt is worth an hour of your team's time? Send us your site and the question through the contact page, and we will tell you what we would do first.
📑 Disclosure!
Some links in this article are affiliate links. If you sign up through them, The Rank Masters may earn a commission at no additional cost to you. This does not influence which tools we recommend or what we say about them.
