If you added an llms.txt file to your site this year, you've got plenty of company and almost certainly no readers. Those are the two datasets the llms.txt world produced in 2026, and they can't both flatter the file. One says the file's presence in the top 10,000 websites grew more than fivefold in twelve months. The other covers 137,000 domains and finds that 97 percent of llms.txt files got zero requests in a month.
A standard that's published eight times faster than it's read deserves a proper analysis rather than one more opinion. So we'll walk through both datasets and why they split apart, then finish with the falsifiable markers that would change our conclusion.
If you've read our earlier llms.txt pieces, you know where we stand on the file itself. The cost-benefit case and the requirements question both came out at "courtesy file, not strategy," and that position hasn't moved. What's new is the full adoption-versus-behavior picture, because 2026 finally produced enough data to draw it.
The adoption side: genuinely booming
The advocates' favorite numbers come from the most rigorous long-running source, an HTTP Archive analysis that has tracked millions of real-traffic sites since July 2025. When it started, 1.04 percent of the top 10,000 websites had a valid llms.txt. By June 2026 that share was 5.61 percent. That's roughly 5.4x in a year, with a steady climb through late 2025 that sped up early this year. Extrapolate across the top million sites and you get something on the order of 39,000 files.
At the very top of the web, Rankability crawled the top 1,000 sites in June 2026. It found the file on 8.7 percent of them, and on 15.8 percent of the ones its crawler could actually reach. Three details underneath tell you more.
Adoption is strangely flat across the rank curve. The spread between the top 1,000 and the top 1,000,000 is barely more than a percentage point. An elite practice trickling down from the biggest sites would look nothing like that. This is a broad fashion that landed everywhere at once, and when something spreads that evenly, you should suspect checklists before measured results.
Then there's Shopify. It deployed llms.txt across its storefronts in late April and early May 2026, which put adoption among Shopify sites near 78 percent almost overnight. That one vendor call probably accounts for a large slice of the year's growth. It also means an adoption chart mostly shows you a handful of platform teams making defaults, rather than tens of thousands of owners weighing a standard. On WordPress, where the choice gets made one plugin at a time, the figure is 8.7 percent.
The last detail is the one we keep coming back to: the major AI labs conspicuously leave llms.txt off their consumer sites, yet most of their documentation sites carry one. Nobody knows better than those labs whether engines read the file, and they publish it where agent builders look and skip it where ordinary users live. You can read that however you like. To us it looks like a message to developer culture far more than an input for a crawler.
One more finding should temper every growth chart you see this year. The dataset's most recent readings, May to June 2026, were flat, the first plateau since tracking began. One flat reading could be noise. It's also exactly what the top of a fashion cycle looks like, so keep an eye on the next few.
See your first AI citation report in under 5 minutes.
No setup calls. No onboarding meetings. Connect your domain and see where AI mentions your brand right now.
The behavior side: almost nobody's reading
The other dataset is brutally simple. Ahrefs analyzed 137,000 domains that publish llms.txt. In May 2026, 97 percent of those files received zero requests, and that zero includes every visitor, from the major engines on down. For each file somebody fetched, roughly thirty sat untouched. The fetches that did happen skewed toward a long tail of smaller agents and research tools, plus the occasional curious developer, which is about the modest audience the spec's authors imagined.
The debate keeps mixing up the rungs of a ladder, so it helps to lay it out. For llms.txt to matter for your visibility, an engine has to fetch the file, parse it, use it when it retrieves or selects sources, and then produce a citation effect you can measure. The Ahrefs data says 97 percent of files never clear that first rung.
The 3 percent that do get fetched run into a wall at rungs two and three, because no engine documents either one. You can read the crawler documentation that carefully specifies robots.txt handling and sitemap ingestion, and it says nothing about llms.txt anywhere that matters.
Rung four is where the controlled evidence lives, and it keeps coming back null. That includes our own twelve-site, ninety-day test, where we measured no citation lift on any engine. A machine-learning analysis of AI-visibility predictors this year found something almost comically deflating: the model's predictions improved once the llms.txt variable was removed, so the file had been feeding it noise.
What about the founders who post server logs showing AI bots fetching llms.txt on their developer-tool sites? Those logs are real, and they fit the ladder: developer-facing sites attract exactly the long-tail agents that read the file, and a fetch is curiosity, a long way short of a citation. Ask practitioners directly whether anyone has seen measurable impact and the threads follow a remarkably stable pattern, with enthusiasm in the abstract and silence once somebody asks for numbers.
Is anyone actually seeing measurable impact from adding an llms.txt file?
Curious whether it’s helping with AI crawling, citations, or visibility in ChatGPT/Gemini/Perplexity or if it’s still mostly experimental right now.
Explaining the gap: why publication outruns consumption
So how does a file grow 5.4x in a year while 97 percent of copies go unread? Name the forces behind it and the paradox goes away. We count four, and none of them needs the file to work.
Start with what we'd call checklist contagion. The file takes twenty minutes to add, it sits at your site root where competitors can see it, and nobody can disprove it quickly. That's the exact profile of a tactic that spreads through audits and listicles whether or not it does anything. It also explains the flat curve across ranks: contagion spreads evenly, while a measured advantage starts at the top and works down.
Platform defaults do the heavy lifting on volume. Shopify's storefront-wide deployment is the single biggest contributor to the adoption numbers, and it took exactly one decision by one team. Several documentation hosts auto-generate the file too. Once platforms are doing the publishing, adoption stops measuring belief entirely, and millions of these "adopters" have never heard of the file they're serving.
For a single site owner, the most defensible reason is a Pascal's wager priced at twenty minutes: if engines ever adopt the file, being early cost you nothing. That's a fine reason. It's also precisely why you can't read adoption data as evidence of anything, because the wager stays rational at near-zero cost even when the person making it has near-zero belief.
The fourth force is identity. In developer-tool culture, llms.txt has turned into a badge that says this site takes agents seriously. That's a legitimate function with a real audience even when crawlers don't care, just not the function anyone's invoice claims. You can see it in the AI labs' own pattern of putting the file on their docs sites and leaving it off their consumer sites.
Put the four next to each other and the gap stops being strange, because the two sides were never coupled. Adoption measures how cheap and fashionable it is to publish the file. Behavior measures whether any engine built a way to consume it, and 2026 is the year the data made the distance between them visible.
Your competitors are building backlinks while you read this.
Organic outreach, social mentions, and link exchanges, with managed backlinks available as an add-on. Grow your domain authority without running the campaign yourself.
Methods and caveats, briefly
We're trading on other people's measurements here, so you're owed a paragraph on their limits. The HTTP Archive series is CrUX-origin based, which means it samples real-traffic sites rather than random URLs. That's the right frame for this question, though it undercounts brand-new microsites, and its top-10k percentages rest on whatever subset its crawler reached in each run.
Rankability's figures depend heavily on reachability, which is where the gap between 8.7 and 15.8 percent comes from, and bot-blocking at the top of the web could plausibly bias that sample in either direction. The zero-requests finding covers one month, May 2026, and you could see different fetch behavior in other windows, though nothing in any adjacent month's anecdata suggests you would. Our own twelve-site test is small-n by design: a controlled probe rather than a census.
Why does the conclusion survive all of that? The caveats point in different directions, while the findings all point one way: publication is high and rising, requests are near zero, documented consumption is absent and controlled effects are null. For the gap to close, you'd need every dataset to be wrong in the same direction, and they were built with different methods.
How real standards actually arrived
If you want the harshest case against llms.txt, look at history. Every web standard that made it arrived in the same order, and llms.txt is running that order backwards.
Robots.txt emerged in 1994 because crawler operators agreed to honor it. Consumption came first, and publishing followed once publishing actually did something. Sitemaps became a standard in 2006, on the day Google, Yahoo and Microsoft jointly announced support. Every major reader committed at once, and publishing exploded afterwards because the payoff was documented.
Schema.org arrived in 2011 as a consortium of the consuming search engines, telling publishers exactly what they'd read and what it would affect. Even canonical tags followed the pattern. Engines announced how they'd handle them, publishers adopted, and you could observe the effects within months.
In every case the readers moved first or at the same time, and they did it in writing, so adoption tracked a documented payoff. llms.txt inverted the sequence. Publishers went first, at scale and on faith, while the would-be readers have stayed conspicuously silent for two years and counting.
You won't find a historical example of that inverted sequence turning into a real standard, and you'll find a rich history of it going the other way. The meta keywords tag is the classic ghost. Publishers kept stuffing it for a decade after engines stopped reading it, because it was cheap and visible and every checklist still asked for it. Its adoption statistics stayed impressive right up until everyone quietly agreed they'd always known better.
Segmenting the adopters: the true belief rate
The headline figure blends groups that mean completely different things, and separating them is where the analysis gets honest.
Take the platform-deployed adopters first: the Shopify storefronts at 78 percent. They made no decision; a vendor default made it for them. This group can grow by millions in a week, and all it tells you is that a platform team judged the file cheap enough to ship. Host-generated adopters are the same thing on a smaller scale, documentation sites whose hosting platforms emit the file automatically.
Owner-chosen adopters are the group that matters. On WordPress, at 8.7 percent, somebody had to install a plugin or write the file by hand, so this is the only population whose number measures belief. After two years of relentless checklist promotion, that rate sits in the high single digits, and that's the real size of the phenomenon.
Lay those groups over the growth curve and you'll see the story sharpen. The 5.4x year had its steepest stretch in spring 2026, exactly when the platform deployments landed. The owner-chosen group grew steadily but modestly, and the curve went flat the month the platform wave finished. Strip the defaults out and llms.txt looks like what it is: a niche practice among developer-tool sites and checklist followers, wearing a platform decision as a growth story.
The llms-full.txt variant, the everything-in-one-file cousin, shows the same shape one tier smaller: enthusiasts publish it, and no engine is documented consuming it.
The economics of the gap
One more force keeps the gap open, and it deserves its own ledger: the gap is profitable.
If you've ever bought agency work, you'll see why an llms.txt line item is close to the perfect deliverable. It ships in an afternoon and screenshots well in a report. Nobody can falsify it inside a quarterly review cycle, and its filename carries the urgency of the era's scariest trend.
Generator tools make money creating the file and audit tools make money flagging its absence, while monitoring products get paid to watch a file nobody fetches. None of this requires bad faith from anyone. It only requires the null result to be slower and quieter than the invoice, which it reliably is.
That's also why the discussion won't die. The evidence has pointed one way for two years, and the practitioners asking for numbers keep meeting silence. You'll still find the file at the top of AI-readiness checklists, because the people producing checklists are disproportionately the people selling afternoons. The meta keywords tag survived a decade on identical economics.
The lesson for you as a buyer is mercifully simple. Price any deliverable whose value can't show up in your own citation data within a month as decoration, whatever its filename says. For llms.txt, that price is the twenty minutes it takes you to write the file yourself.
You're getting AI traffic. But do you know where it comes from?
RankControl credits every visit to the assistant that sent it: ChatGPT, Perplexity, Claude, Gemini, Copilot, or Grok. Full source attribution, next to your Google traffic.
What real adoption would look like
We want this analysis to age into a method, so here are the three falsifiable markers that would change our conclusion, in rough order of arrival.
Documentation would come first. Every real crawler input got there the same way: a vendor documented consumption in its crawler or developer docs, as OpenAI, Anthropic, Google and Perplexity all do for robots.txt today. The day any major engine's documentation specifies how it handles llms.txt, the file changes categories, and a screenshot of an experimental audit that someone shows you doesn't count. This spring's Lighthouse episode already showed how much confusion a nod from a nearby team can generate without a single consuming system behind it.
After that, you'd see it in your logs. Consumption for visibility would show up as llms.txt fetches that correlate with answer-time retrieval, with user-triggered agents requesting the file alongside page fetches, at volume, across many sites. What exists now is occasional curiosity fetches at crawl cadence, and the real signature would be different enough that you wouldn't mistake it.
The final marker is a controlled citation effect. Matched-cohort tests, with sites that have the file and sites that don't in the same category over the same period, would show citation rates diverging. Every such test to date, ours included, shows none. The first credible test showing an effect would be the biggest news in this niche since the spec shipped. Until one of these markers shows up, adoption percentages will keep growing wherever platforms flip defaults, and you shouldn't read any of that growth as machines starting to read.
Test your own site in ten minutes
The nice thing about this debate is that your own logs can settle your own case. Filter your access logs for the file with grep "llms.txt" access.log, then break the hits down by user agent and cadence, the same way the broader crawler audit works. Nearly everyone lands in one of three outcomes.
Zero hits puts you in the 97 percent. The file is costing you nothing and doing nothing, which is fine as long as nobody's billing you for it.
If you see a trickle of long-tail agents, you're in the documented 3 percent, which is typical for developer-facing sites, and those fetches are courtesy traffic: pleasant and harmless, but not something you can measure. And if you find regular fetches from a major engine's documented user agent in answer-time patterns, you've found something the public data hasn't, and we'd genuinely like to hear about it.
Whichever outcome you got, then check the number that actually matters. That's whether engines cite your pages for the queries that come before your deals, measured weekly, per engine. It moves with rankings, structure, entity trust and mentions, and in every controlled look so far it has never once moved with the courtesy file at your site root.
The verdict, updated for the data
Squeeze 2026 into one line and you get a file that sites publish and almost nothing consumes. Adoption grew 5.4x because publishing the file is cheap, visible, platform-automatable and fashionable. Requests didn't follow, because no engine built the reading side, and the first plateau in the adoption curve suggests even the fashion may be cresting.
The file is still what its authors modestly proposed, a courtesy index for the small world of agents that choose to read one, and it's miscast in every other role the industry keeps auditioning it for. If your site serves that small world, spend the twenty minutes. Either way, give it zero strategic attention, and let the three markers above (documentation, answer-time fetches and controlled effects) be the tripwire that reopens the question.
We'll show you exactly where your brand stands in AI search.
No commitment. $0 due today, cancel anytime. See how ChatGPT, Perplexity, Claude, Gemini, Grok, and Google AI Mode talk about your brand today.
