Markdown for agents means answering a request that says Accept: text/markdown with a Markdown version of the page, while everyone else still gets HTML. It is ordinary HTTP content negotiation: one URL, two representations, and the request decides which one comes back.
Why bother? An agent reading your HTML has to wade through navigation, scripts, styles and layout markup to find the text. A Markdown copy of the same page is a fraction of the size and keeps only what matters: headings, paragraphs, lists, links and tables. That means fewer tokens, lower cost for whoever runs the agent, and less room for the agent to misread your page.
How content negotiation works
HTTP has always let a client say which formats it prefers, through the Accept request header. A browser sends something like text/html,application/xhtml+xml. An agent that wants Markdown sends text/markdown. The server looks at the header and chooses what to return.
The important rule is that HTML stays the default. Browsers and search crawlers never ask for Markdown, so they keep receiving exactly what they received before. Only requests that explicitly ask get the Markdown version.
The headers to send
A correct Markdown response carries a few headers:
Content-Type: text/markdown; charset=utf-8so the client knows what it received.text/markdownis a registered media type (RFC 7763).Vary: Accepton both versions. This tells caches and CDNs that the response depends on the Accept header, so a cached Markdown copy is never served to a browser, or the other way round.x-markdown-tokens, an informal header some implementations send, with an estimate of how many tokens the Markdown contains. It lets an agent decide whether to read the whole thing. It is a convention, not a standard, and an estimate is fine.
| Piece | What it does |
|---|---|
| Accept: text/markdown | The agent asks for Markdown |
| Content-Type: text/markdown | The server says what it sent |
| Vary: Accept | Caches keep the two versions apart |
| x-markdown-tokens | Tells the agent roughly how big it is |
| rel=alternate link | Points to the Markdown copy from the HTML |
Cloudflare documents this pattern as Markdown for Agents, and agent-readiness checkers test for it by sending Accept: text/markdown to your home page and checking the response type.
Link the Markdown copy from the page
Negotiation works when the agent knows to ask. A link in the page head works even when it does not:
<link rel="alternate" type="text/markdown" href="/services/seo/index.md">An agent that loads the HTML sees the link and can fetch the lighter version for everything it reads next. Put one on every page, pointing at that page's own Markdown copy.
Publishing the copies at predictable URLs also helps on hosts that cannot inspect request headers. The llms.txt proposal suggests offering a Markdown version of a page at the same URL with .md appended. For folder-style URLs, appending index.md is a common variant, and it makes the copies easy to list in an llms.txt file.
How this site does it
You can see a working example on this site. The home page's Markdown copy is at /index.md, and every page links its own copy with rel="alternate". Requesting any page with Accept: text/markdown returns that same file with Content-Type: text/markdown, Vary: Accept and an x-markdown-tokens estimate.
Two decisions made it reliable:
- The Markdown is generated from the finished HTML at build time, not written separately. It cannot drift from the page, because it is the page.
- Page furniture is left out. Navigation, icons, buttons and decorative graphics do not help an agent understand the content, so the converter drops them and keeps headings, text, lists, tables and links.
On our host, the server cannot set response headers from its config file, so a small script answers negotiated requests. The static Markdown files do the rest. Whatever your stack, the outcome to aim for is the same.
What belongs in the Markdown copy
- The page title as a top-level heading, and the meta description as a short summary.
- The canonical URL, so an agent can cite the right page.
- The main content in order: headings, paragraphs, lists, tables and links with absolute URLs.
- FAQ questions as headings with their answers.
Leave out scripts, tracking, cookie banners and anything decorative. If you would not read it aloud to someone, it probably does not belong.
Test it
You can check the basics from a terminal with curl: request a page normally, then again with -H "Accept: text/markdown", and compare the Content-Type and Vary headers.
Or run the free agent-readiness scan, which sends the Markdown request for you and also checks for the rel="alternate" link. For a wider view of how agents consume pages, see how AI agents read web pages.
Common mistakes
- Serving Markdown to everyone. If the server returns Markdown for a plain request, browsers show raw text. Only answer with Markdown when the request asks for it.
- Forgetting Vary: Accept. Without it, a cache in front of your site can store the Markdown response and hand it to the next browser, or the other way round.
- Returning HTML labeled as Markdown. Some setups change the Content-Type but still send the HTML body. Check the body, not just the header.
- Hand-written copies that drift. A Markdown page written once and never updated soon disagrees with the real page. Generate it from the same source.
- Relative links. Links in the Markdown should be absolute, because an agent may read the file far from its original URL.
Next step
Start with the alternate link and generated Markdown copies, since they work on any host. Add negotiation on the same URLs when your server allows it. Then scan your site to confirm both pass. If you want it built into your stack, our web development work covers it.