Skip to content

feat: serve per-page Markdown and a curated llms.txt index - #634

Merged
berry-13 merged 2 commits into
mainfrom
docs/llm-indexing
Jun 20, 2026
Merged

berry-13 merged 2 commits into
mainfrom
docs/llm-indexing

Conversation

@berry-13

Copy link
Copy Markdown
Collaborator

Summary

Makes the documentation easily consumable by LLMs and coding agents by exposing it as Markdown and adding a curated index.

  • Curated /llms.txt index — a hand-picked map of key docs pages (Quick Start, Install, Configuration, librechat.yaml, Agents, MCP, RAG, Auth, env vars, Changelog) with absolute URLs and a per-page Markdown link for each entry.
  • .md per-page URLs — /docs/<path>.md now rewrites to the Markdown route, served alongside the existing legacy .mdx URLs (kept for compatibility). The in-page copy/view buttons now point at .md.
  • Absolute URLs + shared headers — page exports emit absolute https://www.librechat.ai/... URLs, and all llms routes return a shared text/markdown; charset=utf-8 header via MARKDOWN_RESPONSE_HEADERS.

Supporting changes

  • Docs sidebar restructure — grouped into Deploy / Configure / Use / Tools / Contributing sections, surfaced quick_start, and switched the docs index from QuickStartHub to DocsHub.
  • Docs/README refresh — updated the README and the contributor guide for the current Fumadocs + pnpm stack (replacing stale Bun/Nextra references and source links).
  • Playwright — PLAYWRIGHT_BASE_URL lets the suite target an already-running server (skips the managed web server when set).

Testing

  • pnpm typecheck — passes
  • pnpm lint — passes
  • Added e2e/llms-routes.spec.ts covering /llms.txt, /llms-full.txt, /docs.md, per-page .md, and legacy .mdx routes (content-type + body assertions).

Expose the docs as Markdown for LLMs and coding agents:
- Add /llms.txt, a curated index of key pages with absolute URLs
- Support .md per-page URLs (rewritten to the markdown route) alongside
  the legacy .mdx URLs
- Emit absolute page URLs and a shared "text/markdown; charset=utf-8"
  header across the llms routes
- Add e2e coverage for the llms.txt, llms-full.txt, .md and .mdx routes

Supporting changes:
- Restructure the docs sidebar (Deploy/Configure/Use sections, quick_start)
  and switch the docs index to DocsHub
- Refresh the README and docs contributor guide for the Fumadocs/pnpm stack
- Let Playwright target an external base URL via PLAYWRIGHT_BASE_URL
@vercel

vercel Bot commented Jun 20, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
librechat-ai Ready Ready Preview, Comment Jun 20, 2026 10:20pm

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

📦 Next.js Bundle Analysis for librechat.ai

This analysis was generated by the Next.js Bundle Analysis action. 🤖

This PR introduced no changes to the JavaScript bundle! 🙌

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4bc8bb24f1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread proxy.ts Resolved
An Accept: text/markdown request to a .md/.mdx docs URL hit the
content-negotiation branch first, which rewrote the path to the markdown
route with the suffix still attached. The route handler then looked up a
slug ending in .md and returned 404 instead of letting the next.config
rewrite strip the suffix.

Move the raw .md/.mdx pass-through ahead of the Accept-based rewrite so
suffixed URLs always reach the next.config rewrite, and add an e2e test
for a .md request that also sends Accept: text/markdown.
@github-actions

Copy link
Copy Markdown
Contributor

📦 Next.js Bundle Analysis for librechat.ai

This analysis was generated by the Next.js Bundle Analysis action. 🤖

This PR introduced no changes to the JavaScript bundle! 🙌

@berry-13

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: b5e560ddba

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@berry-13
berry-13 merged commit c47c405 into main Jun 20, 2026
5 checks passed
@berry-13
berry-13 deleted the docs/llm-indexing branch June 20, 2026 22:33

This branch was successfully deployed

1 active deployment
Preview — b5e560dd Deployed Jun 20, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant