This guide does two jobs that usually get done dishonestly when combined. The first is the actual recipe: the llms.txt spec is simple, a correct file takes twenty minutes, and if you want one you deserve instructions that don't wander. The second is the evidence: the measured reality of who fetches this file, which is a short and brutal story, and which should set your expectations before you spend even those twenty minutes. Recipe first, reality second, and a fair account of when writing one still makes sense.
What The File Is
llms.txt is a proposed convention from late 2024, credited to Jeremy Howard of Answer.AI: a markdown file at your domain root meant to hand language models a curated map of your site, the theory being that models working within context limits would prefer a clean index over crawling your navigation. It deliberately mimics robots.txt in location and spirit, a plain text contract at a known address, while doing the opposite job: robots.txt says keep out, llms.txt says here's the good stuff.
The spec is minimal, which is its best quality. One H1 naming the site. A blockquote immediately after, summarizing what the site is in a paragraph. Then H2 sections, each containing a link list where every entry is a markdown link followed by a colon and a one-line description. An H2 called Optional flags links a model can skip under pressure. That's the whole grammar. The companion convention, llms-full.txt, inlines complete page content instead of links, producing one large file meant for direct ingestion rather than navigation.
The Recipe, With A Worked Example
Here's a complete, correct file for a fictional B2B SaaS, annotated by section choice rather than line by line, because the format explains itself:
# Meridian Scheduling
> Meridian is appointment scheduling software for medical clinics.
> It handles patient self-booking, reminders, no-show prediction,
> and syncs with major EHR systems. Plans start at $99/month.
## Product
- [How Meridian works](https://meridian.example/product): booking flow,
reminder engine, and EHR sync explained
- [Pricing](https://meridian.example/pricing): all plans, current prices,
and what each includes
- [Security](https://meridian.example/security): HIPAA posture,
data handling, and compliance documentation
## Comparisons
- [Meridian vs paper scheduling](https://meridian.example/vs-manual):
the switching case with time-cost math
- [Alternatives guide](https://meridian.example/alternatives): honest
category overview including competitors
## Docs
- [API reference](https://meridian.example/docs/api): REST endpoints,
auth, and rate limits
- [EHR integration guides](https://meridian.example/docs/ehr): per-system
setup instructions
## Optional
- [Blog](https://meridian.example/blog): weekly articles on clinic operations
- [About](https://meridian.example/about): company history and team
The choices that matter: the blockquote states category, buyer, and price, the three facts you most want any model to carry; the link descriptions are written as answers, not labels ("all plans, current prices" beats "our pricing page"); comparisons get their own section because those are the pages that decide deals; and the blog goes under Optional, because an index that lists everything prioritizes nothing.
Deployment is the easy paragraph: save it as UTF-8 plain text, serve it at yourdomain.com/llms.txt over HTTPS with a text/plain or text/markdown content type, and it's live. No registration, no ping, no search console equivalent. Update it when your top pages change, which is also the beginning of the problem we should now discuss.
One interplay worth stating because it confuses people: llms.txt does not override robots.txt. A crawler blocked in robots.txt stays blocked regardless of what your index invites it to read, and nothing about publishing the file changes your crawl permissions in either direction. The two files answer different questions from different eras, and only one of them is reliably read.
Know exactly what AI says about your competitors.
RankControl's Recon Agent monitors competitor citations across ChatGPT, Perplexity, Claude, Gemini, Grok, and Google AI Mode. See where they show up and you don't.
Now The Reality, Measured
Having shown you the recipe, honesty requires the consumption data, and it points one direction with unusual consistency.
Ahrefs studied 137,000 domains with llms.txt files and found 97 percent of the files received zero bot requests in a month. Not low engagement: zero fetches. Google's John Mueller has been blunter than usual for a Google spokesperson, comparing the file to the keywords meta tag, noting that no major AI service checks it and that server logs confirm the absence. Google's own guidance labels the file unnecessary. And practitioners keep independently rediscovering the same null: in one r/AISearchLab thread asking the implementation question this guide answers, the highest-signal reply was simply an instruction to grep your access logs, because none of the AI bots are checking the file.
Meanwhile the adoption side runs backwards to the usage side, which is the strangest fact in this whole niche: llms.txt presence among top sites grew more than fivefold in a year, driven substantially by platforms like Shopify deploying it storefront-wide, while measured fetching stayed at approximately nothing. Millions of maps published; almost nobody consulting them. The adoption-versus-behavior data has its own deep dive, as does Google's position in full; the summary for a would-be author is that you are writing a document whose intended readers have, so far, declined the invitation.
I'll flag my own drift here for honesty's sake: a year ago I'd have called the file a cheap harmless hedge and left it at that. Watching a full year of null log data moved me to a sharper position, that the real cost was never the twenty minutes but the belief that the twenty minutes did something, because teams that felt covered by the file measurably deprioritized the structural work that actually moves citations. The file is harmless; the sense of completion it produces is not.
26 content formats. Published on your domain. Matched to your brand.
Guides, comparisons, listicles, case studies, and more. RankControl generates content that gets cited by ChatGPT, Perplexity, Claude, Gemini, Grok, and Google AI Mode.
Five Mistakes In The Files That Do Get Written
Since some readers will write the file regardless, the observed failure modes are worth cataloging, because a bad llms.txt manages to be worse than none in the one scenario where it gets read.
The auto-generated dump. Plugins that export your entire sitemap into llms.txt format produce exactly the un-curated sprawl the spec exists to prevent. An index of four hundred links prioritizes nothing and reads as noise.
Marketing-speak in the blockquote. The summary is the one part a model would quote, and files that fill it with award-winning-platform language waste the file's only real estate. State category, buyer, and price in plain declarative sentences, the way the worked example does.
Labels instead of answers in descriptions. "Our features page" teaches a model nothing; "booking flow, reminder engine, and EHR sync explained" is quotable. Every description should survive being lifted alone.
Link rot on a file nobody monitors. The file sits outside every CMS workflow, so renamed pages quietly break it, and a map with dead roads is worse than no map. If you ship it, put it in the redirect checklist.
llms-full.txt as a reflex. Inlining your whole site into one megafile multiplies the maintenance burden for a variant only coding-tool contexts consume. Build it for a known consumer or skip it.
The Maintenance Contract You're Signing
The recipe takes twenty minutes; the artifact takes a commitment, and unwritten commitments are how harmless files become embarrassing ones. If you publish llms.txt, add three lines to your operating cadence: the file joins the quarterly content audit (do the links still point at your best pages), it joins the redirect checklist (every URL rename checks the file), and it gets an owner whose name is written down. Cost, maybe an hour a quarter. If that hour isn't available, don't publish the file, because a stale index served at a canonical address is the one version of this artifact that could actively misinform the rare agent that does read it.
That sizing, an hour a quarter against a mostly-null readership, is also the fairest one-line summary of the whole llms.txt question in 2026, and it's why this guide could be honest and still include the recipe.
When Writing One Still Makes Sense
Four cases survive the evidence, stated without embarrassment. Documentation sites and developer tools: the one context with genuine consumption, since coding agents and their tooling do sometimes pull curated doc indexes, and llms-full.txt variants see real use in that world. If developers use your product through AI assistants, write the file and mean it. The audited hedge: an hour a quarter to maintain a file that might matter if some future engine adopts the convention is a defensible bet, provided the expectation is written down as a hedge, not a tactic. The Lighthouse peace treaty: an experimental audit in Lighthouse 13.3 checks for the file even as Google Search calls it unnecessary; if your org's tooling flags it and the argument costs more than the file, write the file. The clarity exercise: composing that blockquote and choosing those links is a genuinely useful audit of what your site's most important claims and pages are, and several teams have gotten more from the exercise than any bot ever got from the artifact.
What doesn't survive: writing it instead of the work that's measured to matter. If your money pages aren't extraction-ready, if your facts contradict across the web, if your citation share goes unmeasured, the file is a decoy task. Do it last, if at all, and never report it upward as AI optimization, because the engines have told us, in logs and in words, what they think of it.
The twenty-minute recipe is above and it's yours. Just cook it knowing who's coming to dinner: as of late 2026, almost nobody, with a small table of coding agents in the corner, eating happily.
Your competitors are getting cited by AI. You're not.
Every day without citation tracking is a day your competitors pull ahead in ChatGPT, Perplexity, and Claude.
