August 27, 2026
llms.txt just picked up its first major update since 2024, adding formal ways for AI agents to discover LLM friendly markdown. Here is how the file actually functions, what changed in v2, and why Business to Agent (B2A) interactions are becoming a distinct audience worth designing for.
Two years ago, llms.txt was a quiet, largely ignored file that a small circle of coding tools bothered to read. That has not changed nearly as much as the marketing around AI search likes to suggest, but something underneath it has: the kind of visitor showing up at a website's front door increasingly is not a person, and increasingly is not just a search crawler either. It is software acting on someone else's behalf, comparing documentation, evaluating a vendor, or trying to finish a task, and none of it reads a page the way either of those older visitors did.
What follows covers what llms.txt actually does and the mechanism behind it, what changed with llms.txt v2 and why the update centers on discoverability, why Google's stance on search visibility and AI Overviews has not moved and is not likely to, where the emerging idea of Business to Agent, or B2A, interaction fits into the picture, how the file helps AI agents find and retrieve content once they decide to look, and how all of it ties into the wider question of AI agent readiness across coding assistants, MCP, and Google's own Agentic Browsing checks.
Every llms.txt file exists to answer two questions quickly for whatever reads it: what this site actually is, and which of its pages are worth fetching next. The file itself sits at a predictable root address, yoursite.com/llms.txt, written in plain markdown rather than a format that needs a special parser. Jeremy Howard, who cofounded Answer.AI and fast.ai, put the proposal forward on September 3, 2024, and this August it picked up its first real revision since then, the subject of the next section.
Reading the file is a different act entirely from retrieving through it, and that distinction is most of what the llms.txt function actually is. The file itself does no chunking, no embedding, no ranking, nothing a genuine retrieval pipeline would do. Think of it less as a search engine and more as a knowledgeable person stationed at the door, pointing an agent toward the two or three rooms that actually matter rather than leaving it to wander the whole building. An agent views or searches the file, then follows whichever links match its task, and those links should lead to LLM friendly content, ideally a clean markdown version of a page rather than HTML buried in navigation, scripts, and cookie banners. Put simply, that is what llms.txt for LLMs and the agents built on them is actually for: getting the right handful of pages in front of an agent quickly, not indexing or ranking anything on its own.
our complete guide to building an llms.txt file covers the file's structure, the editorial choices behind a genuinely useful one, and a step by step build process in full. What matters for this piece is the role llms.txt plays once agents, not just search crawlers, start showing up as a website's most demanding visitors: it hands them a shortcut past the noise of a normal web page, straight to a curated answer of what a site is and which pages are worth their limited context budget.
On August 10, 2026, Jeremy Howard published version 2 of the llms.txt specification, the first revision since the original 2024 proposal. Two years of real adoption surfaced one recurring request more than any other: given a page, how does an agent actually find its markdown version, or the llms.txt file that describes it, without guessing at a URL pattern? llms.txt v2 answers that discoverability problem directly, and the change is small enough that most existing files need only a minor addition rather than a full rebuild.
The update centers on two new link relations:
rel="alternate" type="text/markdown": placed on an HTML page, this points directly to that page's markdown version, so an agent already reading the page knows exactly where the cleaner copy lives.
rel="describedby": points from a page to the llms.txt file that covers it, since a single file can describe every page under its own path.
Both can be added as standard HTML link elements inside a page's head, or delivered through an HTTP Link response header instead. The header option matters more than it might look at first glance, since it works on resources beyond HTML too, can be set at the server or CDN level, and needs no template changes across an entire site.
A few smaller but genuinely useful clarifications came with it. Where v1 recognized only one URL pattern for a page's markdown twin, appending .md to the full address (page.html.md), v2 also allows simply swapping the extension (page.md), matching how several publishing tools already generate markdown versions in practice. Subpath files now have a formal rule too: when more than one llms.txt applies along a URL's path, v2 says an agent should defer to whichever one sits closest, which is what lets a project confined to a single directory, a GitHub Pages site being the clearest example, participate fully without controlling its domain's root.
One thing v2 quietly retired is worth knowing if an older file or guide still references it. v1 described a companion tool, llms_txt2ctx, for expanding a file into a ready made LLM context, and gave the Optional section special mechanical meaning tied to that tool. v2 drops the tooling from the proposal itself and, with it, that mechanical meaning. Agents are simply expected to view or search llms.txt and follow the links that matter, and Optional sections remain a useful convention for secondary material, just without a formal processing rule attached to them.
None of the v2 update changes Google's position, which has stayed consistent and explicit. Google's own guidance on showing up in generative AI features states plainly that Search, AI Overviews, and AI Mode all work without an llms.txt file, since Search itself does not use it. Publishing one does not help rankings, and skipping it does not hurt them either.
There is a wrinkle worth understanding rather than dismissing as a contradiction: Chrome's Lighthouse tool checks for llms.txt as part of its Agentic Browsing audit. That is not Google Search quietly reversing course. Search is asking whether a page has earned a place in the results. Lighthouse is asking something narrower and far more mechanical, does this one technical signal exist, yes or no, as a single item on a longer machine readiness checklist. Different tools, built by different teams inside the same company, answering different questions, and treating a Lighthouse checkbox as a ranking factor is exactly where that confusion tends to start. The fuller record on where Google and OpenAI actually stand on the file covers how little the major consumer facing AI assistants currently rely on the file for the answers they generate.
The practical takeaway holds regardless of which version of the file a site runs: llms.txt is not a lever for search visibility or citation frequency in generated answers. Its value, covered through the rest of this piece, sits somewhere else entirely, closer to infrastructure for the agents now doing quite a lot more than just answering questions.
Most of the language businesses use to describe their customers still assumes a person is on the other end: Business to Business, Business to Consumer. Business to Agent, or B2A, names something genuinely different, a growing share of interactions where the entity researching, evaluating, or acting on a company's behalf is software, not a person sitting at a keyboard. A shopper's assistant comparing prices, a developer's coding agent evaluating an API before recommending it, a procurement tool shortlisting vendors overnight, all of these are B2A interactions, and none of them read a page the way a human does.
Being legible to that audience is a different job than ranking well in Google, and llms.txt is one piece of the infrastructure that job requires, though far from the only one. What tends to separate a company that shows up well in B2A interactions from one that does not is whether its content, documentation, and product information are structured for a machine to parse quickly and act on confidently, rather than merely displayed for a person to scroll through.
A few situations show where llms.txt earns its keep most clearly inside a B2A approach:
Recommended Tool
Don't guess whether ChatGPT, Claude, Gemini, or Perplexity can access your content. Analyze your site in seconds.
Check AI Visibility Try Prompt Finder
No signup required • Instant results
A coding agent evaluating a vendor's API documentation before recommending it to a developer, where a curated llms.txt gets it to the right reference page in one hop instead of several
A procurement or research agent shortlisting SaaS vendors based on what it can actually parse from a site's documentation and policies, not marketing copy
A support minded agent trying to locate a specific help article or troubleshooting guide on a person's behalf
A shopping or booking agent working through emerging agentic commerce protocols , where llms.txt sits alongside product feeds and structured data as part of what makes a storefront legible to begin with
None of this replaces the fundamentals. A thin, poorly maintained llms.txt file will not rescue a company whose actual documentation is thin too. But for businesses with real technical depth, an API, a knowledge base, a product with a genuinely technical buyer, treating B2A as a distinct audience worth designing for, rather than an afterthought bundled into general AI web optimization, is becoming as ordinary a consideration as designing for mobile once was.
Strip away the specification language and the actual workflow an agent follows is fairly linear. Picture a coding agent trying to answer a question about a piece of software it does not know well:
It needs information about a domain or a specific page and checks for a describedby link under v2, or requests /llms.txt directly at the site's root, the same predictable address already home to robots.txt and sitemap.xml.
It reads the file, which is short enough to sit comfortably inside its context window alongside everything else it is already working on.
It matches its actual task against the curated link descriptions and picks the one or two pages that are genuinely relevant, skipping the rest.
Where a page publishes an alternate markdown link under v2, it follows that instead of the HTML version, arriving at a copy already cleared of menus, tracking scripts, and the rest of a page's visual scaffolding.
That last step is worth separating from a different, older mechanism that solves a similar problem: content negotiation, where a single URL returns markdown or HTML depending on the Accept header a client sends. Our guide to content negotiation for markdown covers that approach in full. llms.txt v2 takes a different path to a similar destination, a separate, explicitly discoverable web address instead of a negotiated response at the same one, which is easier to set up on static hosting and CDN configurations with no negotiation logic to run at all.
Stated plainly, llms.txt for AI agents is an orientation and retrieval efficiency mechanism, not a citation engine. Pointing an agent toward the right document faster is genuinely useful on its own terms. Whether any given sentence inside that document ends up quoted in a generated answer comes down to chunking, embedding, and reranking decisions made much further down an entirely different pipeline, none of which this file touches at all.
Agent readiness is less a single checkbox than a handful of related surfaces converging on the same idea: making a site's content something software can find, parse, and use without a person translating it first. llms.txt shows up across three of the most active ones.
AI coding assistants are where the file sees its most consistent, confirmed use today. Tools like Claude Code, Cursor, GitHub Copilot, and Windsurf increasingly treat a project or product's /llms.txt as the front door into unfamiliar documentation, following it to the right API reference or tutorial before generating code that depends on it. Documentation platforms and technical products publishing one are, in effect, optimizing directly for the coding agents developers already lean on daily, a narrower but genuinely real audience compared with mainstream AI search traffic.
MCP, the Model Context Protocol that Anthropic introduced in late 2024, is a related but distinct piece of this picture, worth being precise about since it is easy to conflate with WebMCP, a newer and separate standard built for letting a live page hand an agent specific actions to call. MCP works a layer further back, wiring an AI application up to backend tools and data sources it would otherwise have no way to reach, and dedicated MCP servers now exist purely to fetch and search llms.txt files for whatever agent asks. That gives an assistant queryable access to a file's contents instead of having to pull the raw text itself and parse it manually. Some technical products now pair a published llms.txt with a hosted MCP server for their documentation, offering agents both the curated file and a live, queryable interface into the same material.
Agentic browsing is the third surface, and Google's own tooling already treats llms.txt as part of it. Chrome's Agentic Browsing category inside Lighthouse and PageSpeed Insights checks for the file's presence alongside signals like accessibility tree integrity and WebMCP tool registration, one small entry on a longer checklist of whether autonomous software can operate a page at all. None of it is a ranking factor, as covered earlier, but it is a fairly direct signal of where Google expects the agentic web to head next.
AI Visibility Live preview
Check whether AI engines can reach, read and recommend your pages.
Taken together, these three surfaces are the practical, present day expression of the B2A shift described above. AI agent readiness is not a single specification to implement once. It is closer to an ongoing posture, keeping content structured and current enough that whichever surface an agent arrives through, a terminal, a browser, or a chat window, it finds something coherent waiting for it.
llms.txt v2 will not move a ranking report, and it is not the whole of what AI web optimization actually requires. But as one small, low cost piece of a genuinely B2A ready site, alongside clean structure, accurate documentation, and the agent readiness work covered above, it is worth getting right precisely because it is easy to get right. The harder part is knowing whether any of it is actually reaching agents out in the wild, and that calls for watching what happens after publishing rather than assuming the file is doing its job just because it exists.
llms.txt v2, published in August 2026, adds two formal link relations: rel="alternate" for pointing an HTML page to its markdown version, and rel="describedby" for linking a page back to whichever llms.txt file is responsible for it. It also allows a second URL pattern for markdown versions, formally defines how subpath files apply, and retires the old context expansion tooling, which is what removes the extra processing rule v1 had attached to the Optional section.
B2A stands for Business to Agent, the label for interactions where software, standing in for a person, is the one reading a company's content, comparing its documentation, or deciding whether to recommend it. llms.txt fits into B2A by giving that software a curated, machine readable way to orient itself on a site, sitting alongside structured data and clean documentation among the things that let a business be understood by code rather than only displayed to people.
Not urgently. v2 is additive rather than a breaking change, and an existing file keeps working exactly as before. The upgrade mainly benefits sites with substantial documentation that also publish markdown versions of their pages, since v2's link relations make those versions easier for agents to discover automatically, without needing a person or an agent to guess a URL pattern.
Yes, and it is arguably the strongest real world use case for the file today. Tools like these often check a project's llms.txt when they land somewhere unfamiliar, using its curated links to jump straight to the relevant reference or tutorial before writing code that depends on it, instead of working through an entire documentation site page by page.
No, though the two are easy to conflate since both come up whenever AI agents enter the conversation. llms.txt itself is a static file that points an agent toward a site's most useful pages. MCP is a protocol that gives an AI application a way to reach backend tools and data sources, and a handful of MCP servers are built purely to look up and search llms.txt content whenever an agent needs it, which makes the two complementary rather than interchangeable.
Understanding the Core Mechanisms, Role, and Functionality of llms.txt
What Changed in llms.txt v2: Connecting AI Agents to LLM Friendly Markdown
llms.txt and Google Search: No Direct Impact on Search Visibility or AI Overviews
Where llms.txt Matters: Enabling Business to Agent (B2A) Interactions
How llms.txt Helps AI Agents Find and Retrieve Relevant Web Content
llms.txt and Agent Readiness: Supporting AI Coding Assistants, MCP, and Agentic Browsing
Frequently Asked Questions
Summarize with ChatGPT Summarize with Claude Summarize with Perplexity
S
Satvik Mishra
Co Founder of Verseodin
Satvik Mishra is the Co Founder of Verseodin, an AI visibility platform that tracks brand citations across ChatGPT, Gemini, Claude, and Perplexity. He writes about generative engine optimization strategy and what actually works for brands trying to earn visibility in AI powered search.
Newsletter
Get the latest AI Visibility insights, GEO research, product updates, and SEO strategies delivered straight to your inbox.
Continue improving your AI visibility.
Discover what users are asking AI about your industry and uncover prompt opportunities.
Open Prompt Finder
Analyze whether AI search engines can discover and cite your website.
Run Free Report
Product Updates 13 min read
How to test whether a content update improved your brand's visibility in ChatGPT: record a baseline, change one thing, compare updated pages against similar unchanged pages, and read mentions and citations against the noise floor before deciding what to do next.
Read article
Product Updates 15 min read
A buyer prompt library is a maintained set of real buyer questions tracked against AI engines, not a one time keyword export. Learn how to build one with AEO tools, covering collection, journey mapping, clustering, updates, and retirement, plus how Verseodin's Query Universe and prompt tracking fit into the workflow.
Read article
Product Updates 19 min read
Traffic and rankings no longer show the whole picture. This guide walks through how to audit existing content for both SEO and AI search visibility, and how to decide, page by page, what to keep, update, merge, or remove.
Read article
Want more AI SEO insights?
Monthly research on AI search, GEO and citation trends. No noise.
[ partner program ]
Join our partner network, whether you're an agency, consultant, or reseller. Leave your email and we'll reach out with details.
Revenue share
Earn a cut of every referral you bring on, renewing month over month.
Co-marketing
Joint webinars, case studies, and content with the Verseodin team.
Priority support
Direct line to our team plus early access to new features.
How llms.txt Streamlines Web Content for LLMs and AI Agents | VerseOdin