Repository navigation
Closed
Description
Parents: #460, #493
Related: #734, #750, #751, #763, #764
Problem
The required benchmark-comparison-harness CI leg runs zig build benchmark-comparison-test under the gate matrix's 60-minute budget. On CI run 32890815195, 52 other jobs completed successfully and the nightly-only job was intentionally skipped, but this leg was automatically cancelled at exactly 60 minutes. Its last visible completion was the instrumentation-overhead self-test at 20:00:34 UTC; GitHub cancelled the still-running operation at 20:51:48 UTC. The preceding run was also cancelled while this same leg was active.
This prevents a truthful full-integration result even when every engine, sanitizer, corpus, docs, ABI, and threading gate has completed successfully.
Scope
- Attribute the silent tail to the exact
benchmark-comparison-testdependency, build, or child process and distinguish useful cold work from a hang. - Preserve a machine-checkable inventory of every benchmark harness dependency and its terminal result.
- Restructure or bound the expensive work so cold Linux CI completes within the existing required-check budget with useful margin.
- Preserve pinned Home execution, all fail-closed benchmark/evidence validators, independent-suite adapter coverage, and the rule that this harness validates contracts without running scored benchmarks.
- Keep local and CI entry points equivalent and make a future timeout name the unfinished subcheck directly.
No-workaround rules
- No timeout increase as the sole fix, removed/skipped validator,
continue-on-error, hidden background process, warm-cache dependency, reduced fixture, swallowed child failure, or replacement of execution with source-text matching. - No stack, queue, job, Promise, Worker, or Map/Set iterator implementation changes.
Acceptance
- Every
benchmark-comparison-testdependency is inventoried and produces an attributable terminal result. - A cold Linux CI leg completes under 60 minutes with margin and with no validator omitted or downgraded.
- Local focused tests preserve all current benchmark, publication, attribution, quickening, and independent-suite contracts.
- A complete main CI run has every required job green, allowing Math.sumPrecise: capture the iterator next method once #750 and Math.sumPrecise: enforce the iterator element-count bound #751 to close from real full-integration evidence.
Metadata
Metadata
Assignees
Labels
No labels
Activity
chrisbbreuer commentedon Aug 25, 2026
Causal implementation progress:
benchmark-comparison-testlaunched 18 overlapping Home self-test module graphs as separate system commands. A focusedalgorithmic-growth.ts --self-testsample was still inside Home's TypeScript checker before JavaScript execution, matching the prior CI job's long no-output tail.benchmark-harness-self-tests.tsmodule graph, then runs the exact same 18 exported self-tests sequentially with namedSTART,PASS, andFAILrecords. Failures are accumulated so one failed validator does not hide later terminal results.benchmark-comparison-testdependencies./usr/bin/envcommand nodes instead of 56, 69 TypeScript tools, 63 classified repository tools, and every changed caller edge.zig build dependency-auditpasses.This is not closure evidence yet. The full
zig build benchmark-comparison-testgraph and a cold Linux main CI leg under the existing 60-minute limit still need to complete. I am holding the local full graph while unrelated Home compiler/corpus work is active so its result is not distorted.ci(bench): consolidate harness self-tests (#765)
chrisbbreuer commentedon Aug 25, 2026
Landed as 418687fd. The complete local
zig build benchmark-comparison-testgraph passed 15/15 steps and 4/4 native tests in 1,125.70 seconds (18m 45.70s): the named Home inventory passed 18/18, the ReleaseFast zig-js adapter compiled in parallel, both zig-js and system-JSC adapter self-tests passed, the independent-suite audit passed, and both VM quickening checks passed. This flips 0 Test262 cases because the change is CI/tooling-only. The cold Linux main run remains the closure gate.chrisbbreuer commentedon Aug 25, 2026
Main CI run 32905159766 completed with 52 required jobs green and the nightly-only job skipped, but benchmark-comparison-harness was still auto-cancelled at the fixed 60-minute limit. The new attribution changes the diagnosis: the Home module graph compiled and began execution in 7m46s (command start 22:16:23 UTC; first START at 22:24:09), and nine validators passed through exact-parent-regression by 22:26:46.
algorithmic-growththen remained active for 47m44s until cancellation at 23:14:31. Therefore the compile consolidation is retained as useful progress, but it is insufficient for closure. The next fix targets the real zig-js-backed runtime hotspot inside the complete algorithmic-growth self-test; no timeout increase, skipped/reduced validator, engine switch, or swallowed failure.test(bench): attribute algorithmic-growth phases (#765)
chrisbbreuer commentedon Aug 26, 2026
Two linked commits are now on main:
Causal evidence: the pre-fix exact pinned reproduction did not finish its first artifact hash in 368.86 s, and main CI previously spent 47m44s in algorithmic-growth before cancellation. The complete unchanged pinned zig-js-backed self-test now passes all phases in 333.72 s total. The focused 16,384-unit sequential String scan improved from 11.079 s at the index-only intermediate state to 268 ms after the repeated length scan was removed. Focused normal and TSan tests pass 5/5 each; Test262 String remains 1,223/1,223 (0 flips); dependency audit is green with 16 classified edges, 0 migration targets, and 63 classified tools.
The decisive cold Linux closure run is 32923883777. #765 remains open until every required job, especially benchmark-comparison-harness, finishes green under the existing 60-minute budget.
chrisbbreuer commentedon Aug 26, 2026
Completed by 418687fd, 00bfb3e3, and 1ab4ba5c.
The decisive cold Linux main CI run 32923883777 completed successfully: 53 required jobs passed, the one nightly-only job was skipped as configured, and 0 jobs failed or were cancelled. The benchmark-comparison-harness job ran from 02:58:38 to 03:16:55 UTC (18m17s), leaving 41m43s of margin inside the unchanged 60-minute budget. Its inventory reported 18/18 validators passed. The formerly timing-out algorithmic-growth validator ran from 03:12:23.978 to 03:16:48.891 (4m24.913s) and named every fixture hash, accepted report, zero-allocation replay, and fail-closed mutation as PASS.
No timeout was raised, no fixture or validator was reduced/skipped, and no failure was swallowed.