If you manage a website, you have probably seen the recommendation: add an llms.txt file so AI systems can understand your content. Some plugins now generate one automatically. Some agencies have added it to their launch checklist. Others have never heard of it.
It is worth understanding what llms.txt actually is, what problem it was designed to solve, what it does not currently guarantee, and how to make a sensible decision for your own site — without overstating its value.
What llms.txt is
llms.txt is a proposed convention: a plain-text Markdown file placed at the root of a domain (yoursite.com/llms.txt) that offers large language models a concise, curated map of the site’s most important content.
The reasoning behind it is straightforward. A modern web page is a mess of navigation, scripts, banners, cookie notices and sidebars wrapped around a small amount of actual content. Language models also work within limited context windows. The file is meant to provide a clean summary — what the site is, what the organization does, and links to the pages that matter most — so a model does not have to infer all of that from raw HTML.
The format is deliberately simple: a top-level heading with the site name, a short description, then sections of annotated links. Some sites also publish an expanded llms-full.txt containing the actual content of key pages in Markdown.
What it is not
Several misunderstandings are worth clearing up.
It is not robots.txt. robots.txt controls crawler access. llms.txt does not grant or restrict permission, block training or enforce anything. If you want to control AI crawler behavior, that is done through robots.txt directives and your hosting or CDN rules — not this file.
It is not an official standard. It is a community proposal, not a specification ratified by a standards body, and not a requirement published by any major search engine.
It is not a confirmed ranking or citation factor. As of now, the major AI providers have not broadly documented llms.txt as a signal they consume. Some tools may read it; many do not. Claims that adding the file will make your site appear in AI answers should be treated with healthy skepticism.
It is not a substitute for good content. An AI crawler file pointing at thin, outdated or inaccessible pages simply advertises weak content more efficiently.
So why consider it at all?
Three reasonable arguments.
First, cost. A well-maintained file takes an hour to create and little effort to keep current. If adoption grows, you are ready; if it does not, you have lost almost nothing.
Second, clarity. Writing the file forces you to articulate what your organization is and which twenty pages actually matter. That exercise frequently exposes structural problems worth fixing regardless.
Third, positioning. For organizations whose accuracy matters — public agencies, healthcare providers, professional services — any additional clear, authoritative signal about what you publish has defensive value against outdated third-party sources.
What to include if you publish one
- Organization or site name and a one- or two-sentence description of what you do and who you serve
- A short list of your most important pages — core services, contact, about, key documentation — each with a brief annotation of what it contains
- Grouping by section so the structure is obvious
- Only public, canonical URLs — never anything gated, internal or sensitive
- Restraint: curated and short beats a dump of every URL, which is what your sitemap is for
Managing it in WordPress
For llms.txt WordPress implementations, you have three options: upload a static file to the web root, generate it dynamically from a plugin, or have your theme or a small custom endpoint serve it from selected content.
Two cautions. First, do not let two plugins generate the file — duplicated or conflicting output is a common problem as more SEO tools add the feature. Second, an auto-generated file that lists every post in reverse-chronological order defeats the purpose; curation is the point. Tools like the SETN Schema Builder that let you select which entities and pages to include will produce a more useful llms text file than a blanket export.
A sensible priority order
If your site has accessibility barriers, PDF-only documents, outdated service pages or no structured data, fix those first. Those improvements are documented, durable and benefit humans, search engines and AI systems alike.
Once that foundation is in place, publishing a curated llms.txt is a low-cost, low-risk addition to your AI website discoverability practice — as long as you keep it current and do not mistake it for a strategy on its own.
The honest summary
llms.txt is a promising, inexpensive convention with real uncertainty attached. Add it if you can maintain it. Do not expect it to move results by itself, and do not let it distract from the structural work that reliably does.
