Serve a Markdown version of every page
Give every documentation page a clean Markdown version that AI tools can fetch, using per-page .md files, content negotiation, and an HTML discovery tag, with examples for Astro, Docusaurus, Mintlify, Vercel, Netlify, and Cloudflare.
AI assistants and crawlers read better when they receive plain Markdown instead of a rendered HTML page wrapped in navigation, scripts, and styling. This guide shows you how to serve a clean Markdown version of each page. That way, any AI tool — a coding assistant, an answer engine, or an on-demand fetcher — gets your content directly.
It works with the llms.txt file but stands on its own: llms.txt is the map of your docs, and the Markdown versions are the clean content that map points to.
Three mechanisms that work together
Section titled “Three mechanisms that work together”You give AI a Markdown copy of a page through three complementary mechanisms. Set up as many as your platform supports — together they cover tools that look for content in different ways.
| Mechanism | What it does | Who uses it |
|---|---|---|
Per-page .md file | Adds .md to any page URL to return raw Markdown (for example, /quickstart.md) | Tools that guess the .md URL, and llms.txt links |
| Content negotiation | Returns Markdown from the same URL when the request sends Accept: text/markdown | Agents that ask for Markdown by header |
| HTML discovery tag | A <link rel="alternate" type="text/markdown"> in the page <head> | Crawlers that read the HTML first, then follow the link |
Generate a .md version of each page
Section titled “Generate a .md version of each page”Most documentation platforms can emit a Markdown file for every page at build time. Pick your platform.
Astro Starlight
Section titled “Astro Starlight”Add the starlight-contextual-menu plugin. It registers a .md route for every page and adds a menu to open or copy the page as Markdown.
npm install starlight-contextual-menuimport starlightContextualMenu from 'starlight-contextual-menu';
// inside starlight({ ... })plugins: [ starlightContextualMenu({ actions: ['copy', 'view', 'chatgpt', 'claude'], }),],Docusaurus
Section titled “Docusaurus”Add docusaurus-plugin-llms, which emits a per-page .md file alongside llms.txt:
npm install docusaurus-plugin-llms --save-devmodule.exports = { plugins: ['docusaurus-plugin-llms'],};Mintlify
Section titled “Mintlify”No setup is required. Mintlify serves a Markdown version of every page automatically — append .md to any page URL.
Another platform
Section titled “Another platform”If your generator has no plugin, configure your build to write the raw .md source of each page to a predictable path (for example, /guide/quickstart.md), and deploy those files alongside your HTML.
Serve Markdown on the same URL (content negotiation)
Section titled “Serve Markdown on the same URL (content negotiation)”Content negotiation returns Markdown from the page’s normal URL when a client sends an Accept: text/markdown header, so an agent does not need to know the .md path. This is configured at your host or CDN, not in your framework.
Vercel
Section titled “Vercel”Add rewrites that map requests with Accept: text/markdown to the .md file:
{ "rewrites": [ { "source": "/:path+/", "has": [{ "type": "header", "key": "accept", "value": "text/markdown" }], "destination": "/:path+.md" } ]}Netlify
Section titled “Netlify”Add an Edge Function that runs on GET requests whose Accept header contains text/markdown, returns the page’s Markdown, and sets Content-Type: text/markdown and Vary: Accept on the response.
Cloudflare
Section titled “Cloudflare”If your site is proxied through Cloudflare, enable the managed Markdown for Agents feature. When a request sends Accept: text/markdown, Cloudflare converts the page to Markdown at the edge — no code required. See Markdown for Agents.
Mintlify
Section titled “Mintlify”Content negotiation is built in. A request to any page URL with Accept: text/markdown returns Markdown automatically.
Advertise the Markdown version in your HTML
Section titled “Advertise the Markdown version in your HTML”Add a discovery tag to each page’s <head> so crawlers that load the HTML first can find the Markdown twin:
<link rel="alternate" type="text/markdown" href="/guide/quickstart.md" />Add it through your platform’s head or custom-head mechanism (for example, a Starlight Head component override, a Docusaurus head injection, or a Mintlify layout setting). Some platforms and plugins add this tag for you.
Verify your setup
Section titled “Verify your setup”-
Request the
.mdfile directly and confirm it returns Markdown:Terminal window curl https://docs.example.com/guide/quickstart.md -
Request the normal URL with the Markdown header and confirm content negotiation works:
Terminal window curl -H "Accept: text/markdown" https://docs.example.com/guide/quickstart -
View the page source and confirm the discovery tag is present:
Terminal window curl -s https://docs.example.com/guide/quickstart | grep 'type="text/markdown"'
Related
Section titled “Related”- What is llms.txt? — the curated index that links to these Markdown versions.
- Control AI crawler access — decide which AI systems may fetch your content.