镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

feat(llms-txt): write each page's Markdown and llms-full.txt - #1127

Merged
avivkeller merged 4 commits into
feat/typedocfrom
feat/llms-full
Oct 10, 2026
Merged

avivkeller merged 4 commits into
feat/typedocfrom
feat/llms-full

Conversation

@ovflowd

@ovflowd ovflowd commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Description

This PR adds two opt-in options to the llms-txt generator:

  • writeMarkdown: writes the Markdown of every page at {path}.md, which is where llms.txt links by default (pageURL: '{baseURL}{path}.md'), so those links resolve without a separate copy step.
  • writeFull: writes llms-full.txt with every page, each preceded by its URL. It's the same format VitePress' llms plugin produces.

Pages are rebuilt from the metadata entries' content with doc-kit's own remark stringify (the MDX one for MDX pages), skipping synthetic pages.

This is part of a proof of concept of migrating rolldown.rs from VitePress to doc-kit. rolldown.rs serves both today, and this keeps parity without a post-build script.

Validation

  • Tests for building the pages' Markdown (entries joined per page, in order; synthetic pages left out).
  • Rolldown's build produces llms-full.txt and a .md file per page.

Related Issues

Part of the Rolldown docs migration PoC: rolldown/rolldown#11072. It's the top of a stack: #1125 (TypeDoc plugin) → #1126 (Graphviz diagrams) → #1127 (llms-full), each targeting the one below. The changes don't depend on each other, the stack just keeps them reviewable one at a time while the Rolldown PoC builds from the top branch.

Check List

  • I have read the Contributing Guidelines and made commit messages that follow the guideline.
  • I have run node --run test and all tests passed.
  • I have check code formatting with node --run format:check & node --run lint.
  • I've covered new added functionality with unit tests if necessary.

@vercel

vercel Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
api-docs-tooling Ready Ready Preview Oct 9, 2026 10:21pm UTC

Request Review

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

🚀 Deploying Preview to Cloudflare 🚀

Preview Deployments by commit

Status Deployment URL Commit Updated (UTC) See this deployment's details
  • Build: Failed ❌

View logs ↗
d2523f5 2026-10-09T22:21:03.084Z View logs ↗
  • Build: Failed ❌

View logs ↗
6261702 2026-10-09T22:16:46.193Z View logs ↗
  • Build: Failed ❌

View logs ↗
fc26ff3 2026-10-04T16:06:23.455Z View logs ↗
  • Build: In progress 🔵

View logs ↗
3b2d6f9 2026-10-04T16:04:04.079Z View logs ↗
  • Build: Failed ❌

View logs ↗
64217c9 2026-10-03T14:24:00.485Z View logs ↗
  • Build: Failed ❌

View logs ↗
762bf78 2026-10-03T14:14:18.955Z View logs ↗
  • Build: Failed ❌

View logs ↗
9566b83 2026-10-03T13:25:49.860Z View logs ↗
  • Build: Failed ❌

View logs ↗
bbfc21c 2026-10-03T13:14:34.515Z View logs ↗
  • Build: Failed ❌

View logs ↗
0f386b2 2026-10-01T14:25:32.775Z View logs ↗
  • Build: Failed ❌

View logs ↗
82490d7 2026-10-01T13:27:00.445Z View logs ↗

@codecov

codecov Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.33333% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.99%. Comparing base (4d01353) to head (d2523f5).

Files with missing lines Patch % Lines
packages/react/src/llms-txt-full/generate.mjs 90.32% 3 Missing ⚠️
...kages/react/src/llms-txt-full/utils/buildPages.mjs 96.66% 1 Missing ⚠️
Additional details and impacted files
@@               Coverage Diff                @@
##           feat/typedoc    #1127      +/-   ##
================================================
+ Coverage         92.96%   92.99%   +0.02%     
================================================
  Files               260      265       +5     
  Lines             25377    25527     +150     
  Branches           2585     2603      +18     
================================================
+ Hits              23592    23738     +146     
- Misses             1763     1767       +4     
  Partials             22       22              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

api-links Generator

Performance estimate (single CI run)

  • Generation time: 4.2% faster (1.43 s → 1.37 s)
  • Peak memory: 17.1% lower (423.48 MB → 351.26 MB)

json Generator

Performance estimate (single CI run)

  • Generation time: 1.2% slower (10.17 s → 10.29 s)
  • Peak memory: 15.3% higher (1.45 GB → 1.67 GB)

legacy-html Generator

Performance estimate (single CI run)

  • Generation time: 53.0% slower (28.34 s → 43.37 s)
  • Peak memory: 0.0% lower (2.45 GB → 2.45 GB)

legacy-json Generator

Performance estimate (single CI run)

  • Generation time: 28.5% slower (6.43 s → 8.26 s)
  • Peak memory: 0.3% lower (1.58 GB → 1.58 GB)

llms-txt Generator

Performance estimate (single CI run)

  • Generation time: 14.3% slower (6.83 s → 7.81 s)
  • Peak memory: 9.0% lower (1.56 GB → 1.42 GB)

orama-db Generator

Performance estimate (single CI run)

  • Generation time: 27.8% slower (6.41 s → 8.19 s)
  • Peak memory: 4.2% higher (1.63 GB → 1.70 GB)

web Generator

Output size: 2 files changed · net +201.00 B

File size details
File Main PR Change
assets/style-Dpi-h_PZ.css — 137.38 KB +137.38 KB
assets/style-CgRaeUUe.css 137.19 KB — -137.19 KB (-100.0%)

Performance estimate (single CI run)

  • Generation time: 12.8% faster (51.81 s → 45.16 s)
  • Peak memory: 23.2% higher (3.25 GB → 4.00 GB)

@ovflowd
ovflowd marked this pull request as ready for review October 1, 2026 14:17
@ovflowd
ovflowd requested a review from a team as a code owner October 1, 2026 14:17
@ovflowd
ovflowd changed the base branch from main to feat/graphviz-diagrams October 1, 2026 14:24
@ovflowd
ovflowd added this pull request to stack #1143 October 1, 2026 14:28
Comment thread packages/react/src/llms-txt/index.mjs Outdated
Comment thread packages/react/src/llms-txt/README.md
Comment thread packages/react/src/llms-txt/generate.mjs Outdated
Comment thread .changeset/llms-full.md
@ovflowd
ovflowd removed this pull request from stack #1143 October 9, 2026 22:15
@ovflowd
ovflowd changed the base branch from feat/graphviz-diagrams to feat/typedoc October 9, 2026 22:15
@ovflowd
ovflowd added this pull request to stack #1160 October 9, 2026 22:16
Two options: `writeMarkdown` writes the Markdown of every page at the `.md` URLs llms.txt links by default, and `writeFull` writes llms-full.txt, holding every page preceded by its URL.

Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
llms-full.txt shares no data with llms.txt, so it gets its own generator,
like legacy-html-all. writeMarkdown is dropped: the pages' Markdown is the
input, which a build can copy as is.

Assisted-by: Claude Opus 5.5 <noreply@anthropic.com>
@avivkeller
avivkeller merged commit c50e75f into main Oct 10, 2026
25 of 30 checks passed
@avivkeller
avivkeller deleted the feat/llms-full branch October 10, 2026 00:12
@github-actions github-actions Bot mentioned this pull request Oct 10, 2026

This branch was successfully deployed

1 active deployment
Preview – api-docs-tooling — d2523f58 Deployed Oct 9, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants