Somewhere in the last two years, a rumor turned into a checklist item: add an llms.txt file and AI search engines will finally understand your business. It sounds plausible. It sits next to robots.txt like a sensible new sibling. And after two large independent studies, it turns out to do close to nothing.
What llms.txt actually is
llms.txt is a plain Markdown file you publish at the root of your domain, at yourdomain.com/llms.txt. Instead of controlling what crawlers can access (that's robots.txt's job) or listing every URL (that's sitemap.xml), it hands an AI model a short, human-written summary: what your business does, your most important pages, and links worth following.
The proposal is only about two years old, first floated in 2024 as a lightweight way to help large language models understand a site without crawling every page. It has never been adopted by a standards body. It isn't backed by the W3C, the IETF, or any recognized web standard. It's a community convention, nothing more, nothing less.
"You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them." (Google Search Central, updated June 15, 2026)
That's about as direct a "no" as a search engine ever gives.
The adoption numbers
Despite the lack of any official backing, llms.txt caught on with a real slice of the web. SE Ranking's study of nearly 300,000 domains, published November 2025, found 10.13% already had one in place. That's a meaningful number for an unofficial convention with no search engine behind it.
The breakdown by traffic tier is the interesting part:
- Low-traffic sites (0 to 100 visits a month): 9.88% adoption
- Mid-traffic sites (1,001 to 5,000 visits): 10.54% adoption
- High-traffic sites (100,001+ visits): 8.27% adoption
What the data actually shows
Two independent studies, one conclusion: the file isn't doing what its pitch promised.
Smaller and mid-size sites, exactly the businesses reading this article, adopted the file at a slightly higher rate than the largest, best-resourced ones. That's usually a sign of a tactic spreading through advice content and "quick SEO win" checklists faster than it spreads through evidence.
Does it change anything?
This is where the story gets clear. SE Ranking ran a statistical and machine-learning analysis (an XGBoost model) across the same domain set, checking whether having an llms.txt file correlated with getting cited more often by AI answer engines. It found no effect. Removing the variable from the model actually improved its accuracy, meaning the file was adding noise, not signal.
A separate cross-check looked at 94,614 AI citations and traced zero of them back to an llms.txt file. Zero.
Ahrefs took a different angle: server logs. Across 137,000 sites with an llms.txt file in place, 97% received no AI-bot traffic to that file at all in May 2026. The major crawlers mostly aren't requesting it in the first place, which explains why it can't be influencing anything downstream.
Search Engine Land ran the most direct test: add the file to real sites, watch what happens. Eight of nine test sites saw no measurable change in traffic afterward.
Four different methods, one conclusion.
Why Google and the AI labs aren't using it
The core problem is one Google has been consistent about for over a decade: any file the site owner writes and controls is trivially easy to game. John Mueller's comparison to the keywords meta tag is the right one. That tag let site owners declare their own relevance, so search engines stopped trusting it in the early 2000s, once it became obvious everyone would just stuff it with whatever they wanted to rank for. llms.txt has the identical incentive problem. A file where you get to describe your own business in the best possible light isn't a reliable signal for anyone trying to answer a question honestly.
There's also a more mundane, practical reason. Google, OpenAI, Anthropic, and Perplexity have all invested in crawling and understanding the actual content of a page: the headings, the schema, the text a real visitor reads. That content can't be curated separately from what's true. Asking a model to trust a separate, hand-picked summary file adds a second source of truth with no way to verify it against the first. Engineering teams tend not to build around signals like that, and the data above suggests they haven't.
What to do instead
None of this means AI search visibility doesn't matter, or that there's nothing you can do. It means the file everyone talked about isn't the lever. The things that actually correlate with AI citations are less novel and take longer, which is probably why llms.txt was so tempting as a shortcut:
- Fast, crawlable pages. If a bot can't render or load your page quickly, it can't cite it, no matter what any text file says.
- Real structured data. Organization, Product, Offer, and FAQ schema tell a model exactly what you sell and what it costs, in a format built for machine parsing rather than self-description.
- Content that answers real pre-sales questions. Plain-language pages that answer what a buyer actually asks before they call, not marketing copy about how great you are.
- Internal linking between related pages. Helps both classic crawlers and AI retrieval understand how your site's topics connect.
We go deeper on all four, plus a 30-day rollout plan, in GEO and AEO: how to get cited inside ChatGPT, Perplexity and Google AI Overviews. If you're also thinking about the agent side of this (bots that don't just cite you but try to complete a task on your site), see is your website ready for AI agents.
Should you still add one?
If it's genuinely a 30-minute job and someone on your team wants to check the box, go ahead. It costs almost nothing, does no measurable harm, and there's a small chance early adoption pays off if a major AI lab reverses course later. That's a defensible bet with an hour of your time.
It is not a defensible bet with a week of your time, or a line item you pay an agency for, or a reason to delay the changes that the data says actually work. Treat llms.txt as an optional footnote. Spend the real budget on schema, page speed, and content that answers the questions your buyers are actually typing into ChatGPT.
If you want a straight answer on where your site stands for AI visibility, not just llms.txt, but the schema, speed, and content fundamentals that the studies above show actually matter, talk to HiKit about an AI-readiness check or see how we approach it in website design.
FAQ
Questions, answered.
What we tell clients who ask if they need an llms.txt file.
It's a plain Markdown file at yourdomain.com/llms.txt that gives an AI model a short, curated summary of your site: what you do, your key pages, and links worth reading. The idea, proposed in 2024, was that AI models would fetch it before answering questions about your business, the same way search engines read a sitemap. It's a reasonable idea. It just isn't what's happening in practice.
No, and Google has said so directly. Google's Search Central guidance, updated June 15, 2026, states you don't need new machine-readable files, AI text files, or Markdown to appear in Google Search or its AI features, because Google Search doesn't use them. John Mueller compared llms.txt to the old keywords meta tag: a file the site owner controls, so search engines ignore it as a ranking signal by default.
There's no public documentation from OpenAI, Anthropic, or Perplexity naming llms.txt as a citation signal or a visibility requirement. Their published crawler guidance covers user agents and robots.txt access, not llms.txt. The 97% zero-traffic figure from Ahrefs' 137,000-site check tells the same story from the traffic-log side: the major AI crawlers mostly aren't requesting the file at all.
Not completely, just oversold. It costs almost nothing to add (half a day at most), it does no harm, and if AI crawlers ever do adopt it broadly, you're already there. Treat it as a cheap, optional footnote, not a strategy. The mistake is spending real hours on it while skipping the things that measurably move AI citations: clear service pages, real schema markup, and content that answers the exact questions buyers ask.
Fix what the citation data actually correlates with: fast, crawlable pages; Organization, Product, and FAQ schema; clear written answers to real pre-sales questions; and strong internal linking between related pages. We cover the full playbook, including a 30-day plan, in [GEO and AEO: how to get cited inside ChatGPT, Perplexity and Google AI Overviews](/articles/geo-aeo-get-cited-in-ai-search).
