Skip to content

Serve a Markdown version of every page

Give every documentation page a clean Markdown version that AI tools can fetch, using per-page .md files, content negotiation, and an HTML discovery tag, with examples for Astro, Docusaurus, Mintlify, Vercel, Netlify, and Cloudflare.

AI assistants and crawlers read better when they receive plain Markdown instead of a rendered HTML page wrapped in navigation, scripts, and styling. This guide shows you how to serve a clean Markdown version of each page. That way, any AI tool — a coding assistant, an answer engine, or an on-demand fetcher — gets your content directly.

It works with the llms.txt file but stands on its own: llms.txt is the map of your docs, and the Markdown versions are the clean content that map points to.

You give AI a Markdown copy of a page through three complementary mechanisms. Set up as many as your platform supports — together they cover tools that look for content in different ways.

MechanismWhat it doesWho uses it
Per-page .md fileAdds .md to any page URL to return raw Markdown (for example, /quickstart.md)Tools that guess the .md URL, and llms.txt links
Content negotiationReturns Markdown from the same URL when the request sends Accept: text/markdownAgents that ask for Markdown by header
HTML discovery tagA <link rel="alternate" type="text/markdown"> in the page <head>Crawlers that read the HTML first, then follow the link

Most documentation platforms can emit a Markdown file for every page at build time. Pick your platform.

Add the starlight-contextual-menu plugin. It registers a .md route for every page and adds a menu to open or copy the page as Markdown.

Terminal window
npm install starlight-contextual-menu
astro.config.mjs
import starlightContextualMenu from 'starlight-contextual-menu';
// inside starlight({ ... })
plugins: [
starlightContextualMenu({
actions: ['copy', 'view', 'chatgpt', 'claude'],
}),
],

Add docusaurus-plugin-llms, which emits a per-page .md file alongside llms.txt:

Terminal window
npm install docusaurus-plugin-llms --save-dev
docusaurus.config.js
module.exports = {
plugins: ['docusaurus-plugin-llms'],
};

No setup is required. Mintlify serves a Markdown version of every page automatically — append .md to any page URL.

If your generator has no plugin, configure your build to write the raw .md source of each page to a predictable path (for example, /guide/quickstart.md), and deploy those files alongside your HTML.

Serve Markdown on the same URL (content negotiation)

Section titled “Serve Markdown on the same URL (content negotiation)”

Content negotiation returns Markdown from the page’s normal URL when a client sends an Accept: text/markdown header, so an agent does not need to know the .md path. This is configured at your host or CDN, not in your framework.

Add rewrites that map requests with Accept: text/markdown to the .md file:

vercel.json
{
"rewrites": [
{
"source": "/:path+/",
"has": [{ "type": "header", "key": "accept", "value": "text/markdown" }],
"destination": "/:path+.md"
}
]
}

Add an Edge Function that runs on GET requests whose Accept header contains text/markdown, returns the page’s Markdown, and sets Content-Type: text/markdown and Vary: Accept on the response.

If your site is proxied through Cloudflare, enable the managed Markdown for Agents feature. When a request sends Accept: text/markdown, Cloudflare converts the page to Markdown at the edge — no code required. See Markdown for Agents.

Content negotiation is built in. A request to any page URL with Accept: text/markdown returns Markdown automatically.

Section titled “Advertise the Markdown version in your HTML”

Add a discovery tag to each page’s <head> so crawlers that load the HTML first can find the Markdown twin:

<link rel="alternate" type="text/markdown" href="/guide/quickstart.md" />

Add it through your platform’s head or custom-head mechanism (for example, a Starlight Head component override, a Docusaurus head injection, or a Mintlify layout setting). Some platforms and plugins add this tag for you.

  1. Request the .md file directly and confirm it returns Markdown:

    Terminal window
    curl https://docs.example.com/guide/quickstart.md
  2. Request the normal URL with the Markdown header and confirm content negotiation works:

    Terminal window
    curl -H "Accept: text/markdown" https://docs.example.com/guide/quickstart
  3. View the page source and confirm the discovery tag is present:

    Terminal window
    curl -s https://docs.example.com/guide/quickstart | grep 'type="text/markdown"'