镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

Commit 973dc84

Browse files
hi-ogawaclaude
andauthored
feat(browser): show trace steps timeline on terminal on test failure (#11594)
Co-authored-by: Hiroshi Ogawa <4232207+hi-ogawa@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent e955415 commit 973dc84

6 files changed

Lines changed: 418 additions & 3 deletions

File tree

‎docs/config/browser/traceview.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ outline: deep
99
- **CLI:** `--browser.traceView`
1010
- **Default:** `false`
1111

12-
Enable trace-view collection for browser tests. Vitest captures DOM snapshots for browser interactions and can show them in the browser UI, Vitest UI, or HTML reporter when those surfaces are enabled — no external tools required.
12+
Enable [trace view](/guide/browser/trace-view) collection for browser tests. Vitest captures DOM snapshots for browser interactions and can show them in the browser UI, Vitest UI, or HTML reporter when those surfaces are enabled — no external tools required. When a test fails, the terminal output also includes a [timeline of its recorded steps](/guide/browser/trace-view#failure-output).
1313

1414
```ts
1515
export default defineConfig({
Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
FAIL src/sign-in.test.ts > signs in
2+
Error: expect(element).toHaveTextContent()
3+
4+
Expected element to have text content:
5+
 Welcome
6+
Received:
7+
 Sign in
8+
9+
 ❯ src/sign-in.test.ts:11:51
10+
 9| })
11+
 10| await page.getByRole('button', { name: 'Sign in' }).click()
12+
 11| await expect.element(page.getByRole('heading')).toHaveTextContent('Welcome')
13+
 | ^
14+
 12| })
15+
16+
Caused by: Error: Matcher did not succeed in time.
17+
18+
Trace:
19+
 step locator start duration location
20+
fill form -  +1ms 58ms src/sign-in.test.ts:6:14
21+
├── fill getByLabel('Email')  +2ms 31ms src/sign-in.test.ts:7:40
22+
└── fill getByLabel('Password')  +34ms 24ms src/sign-in.test.ts:8:43
23+
click getByRole('button', { name: 'Sign in' }) +61ms 18ms src/sign-in.test.ts:10:55
24+
 toHaveTextContent [ERROR] getByRole('heading')  +80ms 1003ms src/sign-in.test.ts:11:16

‎docs/guide/browser/trace-view.md‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
`browser.traceView` records browser interactions as DOM snapshots and lets you replay them step by step in Vitest's built-in trace viewer. It is useful when the live browser view is not enough: you can inspect earlier tests, failed retries, screenshots, assertions, and user actions after the browser has already moved on.
44

5-
Trace view is additive to the current browser testing workflow. Enabling it does not force a single debugging mode. You can use it with the normal local browser UI, with a headless browser and Vitest UI, or with the HTML reporter in CI.
5+
Trace view is additive to the current browser testing workflow. Enabling it does not force a single debugging mode. You can use it with the normal local browser UI, with a headless browser and Vitest UI, or with the HTML reporter in CI. When a test fails, the terminal output also includes a text timeline of its recorded steps.
66

77
::: tip Trace view, browser UI, and HTML reports
88

@@ -44,7 +44,7 @@ vitest --browser.traceView
4444

4545
:::
4646

47-
When `browser.traceView` is enabled, tests with recorded traces can be opened in the trace viewer from the [browser UI](/config/browser/ui), [Vitest UI](/guide/ui), and [HTML reporter](/guide/reporters#html-reporter).
47+
When `browser.traceView` is enabled, tests with recorded traces can be opened in the trace viewer from the [browser UI](/config/browser/ui), [Vitest UI](/guide/ui), and [HTML reporter](/guide/reporters#html-reporter). Failed tests also print their recorded steps in the terminal, as described in [Failure Output](#failure-output).
4848

4949
With the HTML reporter enabled, the [Vitest VS Code extension](https://github.com/vitest-dev/vscode#trace-view) adds an **Open Trace View** action to tests in the Testing view and editor gutter.
5050

@@ -81,6 +81,7 @@ TODO: The browser UI / Vitest UI / browser driver combinations are not specific
8181
| Debug locally with a headless browser | `vitest --browser.traceView --browser.headless --ui` | The browser runs headless, while Vitest UI shows recorded trace steps and snapshots. |
8282
| Debug locally with a visible browser window and Vitest UI | `vitest --browser.traceView --browser.headless=false --browser.ui=false --ui` | Vitest UI shows recorded trace steps and snapshots, while tests run in a separate headed browser window. |
8383
| Generate a static report for CI or run mode | `vitest run --browser.traceView --reporter=html` | The HTML report includes the trace viewer for recorded tests. |
84+
| See failure steps in CI logs or from a coding agent | `vitest run --browser.traceView` | The failure summary in the terminal includes a timeline of the steps that led to each failure. |
8485

8586
## Relation to Playwright Traces
8687

@@ -107,6 +108,14 @@ Each entry captures the DOM state at that point, along with timing information,
107108

108109
In Vitest UI, trace entries are streamed as the test runs, so you can inspect recorded steps before the test finishes. Long-running actions, `expect.element(...)` assertions, and callback `page.mark()` entries appear as in-progress steps first, then update with their final status and duration.
109110

111+
## Failure Output
112+
113+
When a test fails, the error in the terminal is followed by a timeline of its recorded steps:
114+
115+
<<< ./snippets/trace-failure.ansi [Error report]
116+
117+
These are the same steps the trace viewer lists, without DOM snapshots. Only the last 10 steps are printed, so open the trace viewer to see the full trace.
118+
110119
## Custom Trace Entries
111120

112121
You can insert your own named entries with `page.mark()` and `locator.mark()`:

‎packages/vitest/src/node/reporters/base.ts‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ import {
4444
renderBenchmarkRow,
4545
} from './renderers/benchmark-table'
4646
import { F_CHECK, F_DOWN_RIGHT, F_POINTER } from './renderers/figures'
47+
import { formatTraceTimeline } from './renderers/trace-timeline'
4748
import {
4849
countTestErrors,
4950
divider,
@@ -1529,6 +1530,16 @@ export abstract class BaseReporter implements Reporter {
15291530
screenshotPaths,
15301531
})
15311532

1533+
// TODO: print the trace of the attempt that produced this error instead of the last attempt,
1534+
// using `task.result.attempts` (https://github.com/vitest-dev/vitest/pull/11232)
1535+
for (const task of tasks) {
1536+
const timeline =
1537+
task.type === 'test' && formatTraceTimeline(task.artifacts, this.ctx.config.root)
1538+
if (timeline) {
1539+
this.error(`${timeline}\n`)
1540+
}
1541+
}
1542+
15321543
if (tasks[0].type === 'test' && tasks[0].annotations.length) {
15331544
const test = this.ctx.state.getReportedEntity(tasks[0]) as TestCase
15341545
this.printAnnotations(test, 'error', 1)
Lines changed: 207 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,207 @@
1+
import type { TestArtifact } from '../../../runtime/runner/types'
2+
import { relative } from 'pathe'
3+
import c from 'tinyrainbow'
4+
import { F_TREE_NODE_END, F_TREE_NODE_MIDDLE } from './figures'
5+
6+
interface TimelineRow {
7+
cells: [name: string, locator: string, start: string, duration: string, location: string]
8+
status?: 'pass' | 'fail'
9+
}
10+
11+
const MAX_STEPS = 10
12+
13+
export function formatTraceTimeline(artifacts: TestArtifact[], root: string): string | undefined {
14+
const attempt = [...getTraceAttemptMap(artifacts).values()].at(-1)
15+
if (!attempt) {
16+
return
17+
}
18+
19+
// the test end is already shown by the error location
20+
const visible = attempt.entries.filter((step) => step.name !== 'vitest:onAfterRetryTask')
21+
if (!visible.length) {
22+
return
23+
}
24+
const shown = visible.slice(-MAX_STEPS)
25+
const prefixes = getTreePrefixes(shown)
26+
const headerRow: TimelineRow = { cells: ['step', 'locator', 'start', 'duration', 'location'] }
27+
const rows = [headerRow, ...shown.map((step, i) => formatStep(step, prefixes[i], root))]
28+
29+
const widths = headerRow.cells.map((_, i) => Math.max(...rows.map((row) => row.cells[i].length)))
30+
const lines = rows.map((row) => {
31+
// the last column is left unpadded so rows have no trailing spaces inside its color codes
32+
const [name, locator, start, duration, location] = row.cells.map((cell, i) =>
33+
i === 4 ? cell : i === 2 || i === 3 ? cell.padStart(widths[i]) : cell.padEnd(widths[i]),
34+
)
35+
if (row === headerRow) {
36+
return c.dim(` ${name} ${locator} ${start} ${duration} ${location}`)
37+
}
38+
const text = ` ${name} ${colorCell(locator, c.cyan)} ${start} ${colorCell(duration)} ${colorCell(location, c.dim)}`
39+
return row.status === 'fail' ? c.red(text) : text
40+
})
41+
const omitted = visible.length - shown.length
42+
if (omitted) {
43+
lines.splice(1, 0, c.dim(` ... ${omitted} earlier ${omitted === 1 ? 'step' : 'steps'}`))
44+
}
45+
46+
const header: string[] = []
47+
if (attempt.retry) {
48+
header.push(`retry ${attempt.retry}`)
49+
}
50+
if (attempt.repeats) {
51+
header.push(`repeat ${attempt.repeats}`)
52+
}
53+
const title = header.length ? `Trace (${header.join(', ')}):` : 'Trace:'
54+
return [`\n${title}`, ...lines].join('\n')
55+
}
56+
57+
function getTreePrefixes(steps: NormalizedBrowserTraceEntry[]): string[] {
58+
const hasNextSibling = steps.map((step, i) => {
59+
const next = steps.slice(i + 1).find((other) => other.depth <= step.depth)
60+
return next?.depth === step.depth
61+
})
62+
const lastAtDepth: number[] = []
63+
return steps.map((step, i) => {
64+
lastAtDepth[step.depth] = i
65+
if (step.depth === 0) {
66+
return ''
67+
}
68+
let prefix = ''
69+
for (let level = 1; level < step.depth; level++) {
70+
prefix += hasNextSibling[lastAtDepth[level]] ? '│ ' : ' '
71+
}
72+
return `${prefix}${hasNextSibling[i] ? F_TREE_NODE_MIDDLE : F_TREE_NODE_END} `
73+
})
74+
}
75+
76+
function formatStep(step: NormalizedBrowserTraceEntry, prefix: string, root: string): TimelineRow {
77+
const location =
78+
step.location &&
79+
`${relative(root, step.location.file)}:${step.location.line}:${step.location.column}`
80+
return {
81+
cells: [
82+
`${prefix}${formatStepName(step.name)}`,
83+
step.element?.locator || '-',
84+
`+${formatMs(step.startTime)}`,
85+
step.duration == null ? '-' : formatMs(step.duration),
86+
location || '-',
87+
],
88+
status: step.status,
89+
}
90+
}
91+
92+
function colorCell(cell: string, format = (text: string) => text): string {
93+
return cell.trim() === '-' ? c.dim(cell) : format(cell)
94+
}
95+
96+
function formatMs(ms: number): string {
97+
return `${Math.round(ms)}ms`
98+
}
99+
100+
function formatStepName(name: string): string {
101+
return name.startsWith('vitest:') ? name.slice('vitest:'.length) : name
102+
}
103+
104+
// TODO: everything below is copied from `BrowserTraceData`/`BrowserTraceEntry` in
105+
// packages/browser/src/client/tester/trace.ts (only the fields used here) and
106+
// `getTraceAttemptMap` in packages/ui/client/composables/trace-view.ts.
107+
// share them instead (https://github.com/vitest-dev/vitest/pull/11604)
108+
109+
interface BrowserTraceData {
110+
retry: number
111+
repeats: number
112+
entries: BrowserTraceEntry[]
113+
}
114+
115+
interface BrowserTraceEntry {
116+
name: string
117+
range?: { id: string; phase: 'start' | 'end' }
118+
status?: 'pass' | 'fail'
119+
startTime: number
120+
duration?: number
121+
location?: { file: string; line: number; column: number }
122+
element?: { locator: string }
123+
}
124+
125+
interface NormalizedBrowserTraceData extends BrowserTraceData {
126+
entries: NormalizedBrowserTraceEntry[]
127+
}
128+
129+
interface NormalizedBrowserTraceEntry extends BrowserTraceEntry {
130+
depth: number
131+
}
132+
133+
function getTraceAttemptKey(trace: BrowserTraceData): string {
134+
return `${trace.repeats}:${trace.retry}`
135+
}
136+
137+
function normalizeTraceEntries(entries: BrowserTraceEntry[]): NormalizedBrowserTraceEntry[] {
138+
const merged: NormalizedBrowserTraceEntry[] = []
139+
const startMap = new Map<string, number>()
140+
const parents: string[] = []
141+
142+
for (const entry of entries) {
143+
const range = entry.range
144+
if (!range) {
145+
merged.push({
146+
...entry,
147+
depth: parents.length,
148+
})
149+
continue
150+
}
151+
152+
if (range.phase === 'start') {
153+
startMap.set(range.id, merged.length)
154+
merged.push({
155+
...entry,
156+
depth: parents.length,
157+
})
158+
parents.push(range.id)
159+
continue
160+
}
161+
162+
// when range.phase === 'end'
163+
const index = startMap.get(range.id)
164+
if (index == null) {
165+
// unpaired range shouldn't happen but just leave it there
166+
merged.push({ ...entry, depth: 0 })
167+
continue
168+
}
169+
170+
// Keep the start timestamp for positioning, but derive display duration
171+
// from the end event timestamp.
172+
const start = merged[index]
173+
merged[index] = {
174+
...start,
175+
...entry,
176+
startTime: start.startTime,
177+
duration: entry.startTime - start.startTime,
178+
}
179+
parents.pop()
180+
}
181+
182+
return merged
183+
}
184+
185+
function getTraceAttemptMap(artifacts: TestArtifact[]): Map<string, NormalizedBrowserTraceData> {
186+
const grouped: Record<string, BrowserTraceData[]> = {}
187+
for (const artifact of artifacts) {
188+
if (artifact.type !== 'internal:browserTrace') {
189+
continue
190+
}
191+
const trace = artifact.data as BrowserTraceData
192+
const key = getTraceAttemptKey(trace)
193+
grouped[key] ??= []
194+
grouped[key].push(trace)
195+
}
196+
197+
const merged = new Map<string, NormalizedBrowserTraceData>()
198+
for (const [key, traces] of Object.entries(grouped)) {
199+
const trace = traces[0]
200+
const entries = traces.flatMap((trace) => trace.entries)
201+
merged.set(key, {
202+
...trace,
203+
entries: normalizeTraceEntries(entries),
204+
})
205+
}
206+
return merged
207+
}

0 commit comments

Comments
 (0)