镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

Commit 5ac4a84

Browse files
committed
docs(benchmarks): distinguish full-run and bundled sample aggregation
Keep measured values unchanged and render the bundled sample count from its receipt. Refs #3188, #3171
1 parent 23d74f3 commit 5ac4a84

3 files changed

Lines changed: 9 additions & 9 deletions

File tree

‎README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -484,7 +484,7 @@ dtsx produces **sound** types (correctly widened for mutable containers) while p
484484

485485
Measured on **Apple M3 Pro (`Mac15,6`), 11-core CPU (5 performance + 6 efficiency), 18 GB RAM, macOS 27.0.1 (arm64)** — 2026-11-12.
486486
dtsx 0.11.52 at [`ba9df16`](https://github.com/stacksjs/dtsx/commit/ba9df16) · Bun 1.4.4 · Zig 0.17.1 · oxc-transform 0.158.0 · TypeScript 7.0.2 (native `tsc`) · tsdown 0.23.7.
487-
Each time is the median of 3 full runs; the 1-minute load average ranged 4.5–8.2 during measurement.
487+
API and CLI values are the median of 3 full runs; the 1-minute load average ranged 4.5–8.2 during measurement.
488488

489489
Reproduce with `bun run benchmark:prepare && bun run benchmark:publish`. Raw numbers live in [`benchmark/results.json`](https://github.com/stacksjs/dtsx/blob/main/benchmark/results.json).
490490

@@ -495,7 +495,7 @@ Reproduce with `bun run benchmark:prepare && bun run benchmark:publish`. Raw num
495495
- `tsc` is TypeScript 7's native Go compiler, run directly (no Node launcher) with `--noCheck`, its fastest declaration path. It has no in-process API, so it only appears in the CLI tables.
496496
- oxc has no declaration CLI, so its CLI rows run a minimal Bun-compiled wrapper around `oxc-transform` (`bun build --compile --splitting --bytecode`).
497497
- "No cache" means every dtsx cache is cleared on every iteration. The warm-cache table is reported separately and never used for comparisons.
498-
- Times are means from [mitata](https://github.com/evanwashere/mitata); lower is better. **Bold** marks the fastest tool per column.
498+
- API and CLI per-run means come from [mitata](https://github.com/evanwashere/mitata), aggregated as described above. Bundled builds use the sample means described below. Lower is better; **bold** marks the fastest tool per column.
499499

500500
### In-process API (no cache)
501501

@@ -539,7 +539,7 @@ _The `@dtsx/dtsx` SDK, which powers the bundler and build-tool plugins, called i
539539

540540
#### JS SDK: bundled declarations vs tsdown
541541

542-
_Fresh builds of the same project, checked by strict positive and negative consumer type-checks outside timing._
542+
_Fresh builds of the same project. Each value is the mean of 5 measured samples per tool/project. Strict positive and negative consumer type-checks run outside timing._
543543

544544
| Tool | 2 source files | 11 source files | 101 source files |
545545
|------|------|------|------|

‎benchmark/render.ts‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ const cores = m.performanceCores && m.efficiencyCores
6666
const machineLine = `${m.cpu}${m.model ? ` (\`${m.model}\`)` : ''}, ${cores}, ${m.memoryGB} GB RAM, ${m.os} (${m.arch})`
6767
const loads = (results.loadAverage ?? []).flatMap(l => [l.start, l.end])
6868
const runLine = results.runs && results.runs > 1
69-
? `Each time is the median of ${results.runs} full runs${loads.length ? `; the 1-minute load average ranged ${Math.min(...loads).toFixed(1)}–${Math.max(...loads).toFixed(1)} during measurement` : ''}.`
69+
? `API and CLI values are the median of ${results.runs} full runs${loads.length ? `; the 1-minute load average ranged ${Math.min(...loads).toFixed(1)}–${Math.max(...loads).toFixed(1)} during measurement` : ''}.`
7070
: loads.length ? `The 1-minute load average ranged ${Math.min(...loads).toFixed(1)}–${Math.max(...loads).toFixed(1)} during measurement.` : ''
7171
const sourceLine = m.commit ? `dtsx ${m.versions.dtsx} at [\`${m.commit}\`](https://github.com/stacksjs/dtsx/commit/${m.commit.replace(/-dirty$/, '')})` : `dtsx ${m.versions.dtsx}`
7272
const versionLine = `${sourceLine} · Bun ${m.bun} · Zig ${m.versions.zig} · oxc-transform ${m.versions['oxc-transform']} · TypeScript ${m.versions.typescript} (native \`tsc\`) · tsdown ${m.versions.tsdown}`
@@ -128,7 +128,7 @@ ${h2} How the comparison is kept fair
128128
- \`tsc\` is TypeScript 7's native Go compiler, run directly (no Node launcher) with \`--noCheck\`, its fastest declaration path. It has no in-process API, so it only appears in the CLI tables.
129129
- oxc has no declaration CLI, so its CLI rows run a minimal Bun-compiled wrapper around \`oxc-transform\` (\`bun build --compile --splitting --bytecode\`).
130130
- "No cache" means every dtsx cache is cleared on every iteration. The warm-cache table is reported separately and never used for comparisons.
131-
- Times are means from [mitata](https://github.com/evanwashere/mitata); lower is better. **Bold** marks the fastest tool per column.
131+
- API and CLI per-run means come from [mitata](https://github.com/evanwashere/mitata), aggregated as described above. Bundled builds use the sample means described below. Lower is better; **bold** marks the fastest tool per column.
132132
133133
${h2} In-process API (no cache)
134134
@@ -158,7 +158,7 @@ ${sdkInProcess}
158158
159159
${h3} JS SDK: bundled declarations vs tsdown
160160
161-
_Fresh builds of the same project, checked by strict positive and negative consumer type-checks outside timing._
161+
_Fresh builds of the same project. Each value is the mean of ${results.tsdown ? [...new Set(results.tsdown.results.map(row => row.iterations))].join('/') : 'the recorded'} measured samples per tool/project. Strict positive and negative consumer type-checks run outside timing._
162162
163163
${bundled}
164164

‎docs/benchmarks.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ description: Measured dtsx performance against oxc, tsc 7 and tsdown.
99

1010
Measured on **Apple M3 Pro (`Mac15,6`), 11-core CPU (5 performance + 6 efficiency), 18 GB RAM, macOS 27.0.1 (arm64)** — 2026-11-12.
1111
dtsx 0.11.52 at [`ba9df16`](https://github.com/stacksjs/dtsx/commit/ba9df16) · Bun 1.4.4 · Zig 0.17.1 · oxc-transform 0.158.0 · TypeScript 7.0.2 (native `tsc`) · tsdown 0.23.7.
12-
Each time is the median of 3 full runs; the 1-minute load average ranged 4.5–8.2 during measurement.
12+
API and CLI values are the median of 3 full runs; the 1-minute load average ranged 4.5–8.2 during measurement.
1313

1414
Reproduce with `bun run benchmark:prepare && bun run benchmark:publish`. Raw numbers live in [`benchmark/results.json`](https://github.com/stacksjs/dtsx/blob/main/benchmark/results.json).
1515

@@ -20,7 +20,7 @@ Reproduce with `bun run benchmark:prepare && bun run benchmark:publish`. Raw num
2020
- `tsc` is TypeScript 7's native Go compiler, run directly (no Node launcher) with `--noCheck`, its fastest declaration path. It has no in-process API, so it only appears in the CLI tables.
2121
- oxc has no declaration CLI, so its CLI rows run a minimal Bun-compiled wrapper around `oxc-transform` (`bun build --compile --splitting --bytecode`).
2222
- "No cache" means every dtsx cache is cleared on every iteration. The warm-cache table is reported separately and never used for comparisons.
23-
- Times are means from [mitata](https://github.com/evanwashere/mitata); lower is better. **Bold** marks the fastest tool per column.
23+
- API and CLI per-run means come from [mitata](https://github.com/evanwashere/mitata), aggregated as described above. Bundled builds use the sample means described below. Lower is better; **bold** marks the fastest tool per column.
2424

2525
## In-process API (no cache)
2626

@@ -64,7 +64,7 @@ _The `@dtsx/dtsx` SDK, which powers the bundler and build-tool plugins, called i
6464

6565
### JS SDK: bundled declarations vs tsdown
6666

67-
_Fresh builds of the same project, checked by strict positive and negative consumer type-checks outside timing._
67+
_Fresh builds of the same project. Each value is the mean of 5 measured samples per tool/project. Strict positive and negative consumer type-checks run outside timing._
6868

6969
| Tool | 2 source files | 11 source files | 101 source files |
7070
|------|------|------|------|

0 commit comments

Comments
 (0)