Skip to content

feat: Serve Markdown versions of docs pages for AI agents - #19

Merged
craftzdog merged 1 commit into
mainfrom
feat/markdown-for-agents
Oct 19, 2026
Merged

craftzdog merged 1 commit into
mainfrom
feat/markdown-for-agents

Conversation

@craftzdog

Copy link
Copy Markdown
Contributor

Summary

Makes the docs easy for AI agents to read by serving each page as plain Markdown.

  • <path>.md endpoints: every page is available as Markdown, e.g. /start-guide/how-to-write-markdown-notes.md (/index.md for the home page). src/lib/markdown.ts converts Markdoc tags to standard Markdown: kbd → inline code, callout → blockquote, img/video/carousel-item → images and links, thumb-links → a list built from the navigation. Site-relative links become absolute URLs. Code is left untouched, and the build fails on a Markdoc tag that has no conversion, so content is never dropped silently. Pages are prerendered by a static route handler (/raw-md/...) that the *.md rewrites point to.
  • Content negotiation: src/proxy.ts serves the Markdown at the regular URL when Accept prefers text/markdown. Its matcher only runs for requests mentioning text/markdown, so normal page views and prefetches skip it.
  • /llms.txt is generated from navigation.ts and links to the .md pages. Pages missing from the navigation go under ## Optional. /llms-full.txt joins all pages into one file. Both leave out examples/**, like the search index. This replaces the hand-written public/llms.txt, which had gone stale.
  • Discovery: a "View as Markdown" button with a bot icon above "On this page", plus <link rel="alternate" type="text/markdown"> in the page head.

Notes

  • HTML responses don't get Vary: Accept, because Next overrides the header on pages. On Vercel the proxy runs before the CDN cache and Markdown is cached under its own path, so this only matters for third-party shared caches.
  • The "View as Markdown" button only shows at xl widths and up, like the rest of the table of contents column.

Test plan

  • npm run lint and npm run build pass
  • Every .md link in /llms.txt returns 200 on next start; an unknown .md path returns 404
  • Accept: text/markdown (with or without lower-q text/html) returns Markdown; browser Accept and text/markdown;q=0 return HTML
  • No leftover {% tags in llms-full.txt apart from a documented {% uuid %} example in inline code
  • Check the button and the .md output on the Vercel preview

- Serve every page as plain Markdown at `<path>.md` (`/index.md` for the
  home page), converting Markdoc tags to standard Markdown
- Serve the same Markdown at the regular URL for `Accept: text/markdown`
- Generate `/llms.txt` from the navigation and add `/llms-full.txt`,
  replacing the hand-written `public/llms.txt`
- Add a "View as Markdown" button and a `rel="alternate"` link to each page
@craftzdog
craftzdog merged commit 074b3bf into main Oct 19, 2026
2 checks passed
@craftzdog
craftzdog deleted the feat/markdown-for-agents branch October 19, 2026 02:52

This branch was successfully deployed

1 active deployment
Preview — 1e1aa977 Deployed Oct 19, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant