Repository navigation
Closed
Labels
enhancementNew feature or requestNew feature or request
Description
Goal
Ship production-quality JavaScript multithreading in zig-js across both supported isolation models:
- isolated agents/Workers exchanging bounded structured-clone values and retained
SharedArrayBufferstorage; - shared-realm
Threadexecution preserving JavaScript object identity on real OS threads.
Pre-stabilization backward compatibility was intentionally not a constraint; the public contract reflects the cleaner final design.
Public contract
// True-parallel shared-realm JavaScript by default.
const parallel = try js.Context.createWith(gpa, .{ .enable_threads = true });
// Supported serialized fallback.
const serialized = try js.Context.createWith(gpa, .{
.enable_threads = true,
.gil = true,
});The C API exposes the same choice through ZJSGlobalContextCreateThreaded(gil). parallel_js, parallel_midscript_gc, and profiling/fuzz controls remain internal and test-only.
Delivered
- Real OS-thread agents and Workers; versioned structured clone; ArrayBuffer transfer/detach; retained shared storage; typed-array wait/notify/waitAsync.
- Fallible, nonblocking, resource-bounded Worker channels with checked per-message, queued-byte, and message-count accounting; complete-frame limits; exact SAB ownership; observable Zig/JS/C rejection.
- Shared-realm
Thread,Lock,Condition,ThreadLocal,ConcurrentAccessError, property Atomics, and proposal-alignedAtomics.Mutex/Atomics.Condition. - No-GIL execution by default with synchronized shapes, properties/elements, environments, promises/microtasks, waiters, tasks, handles, GC roots, and lifecycle state.
- GC-managed thread-safe cells, precise VM roots, safe native-stack publication, weak/finalization processing, a non-moving one-cycle nursery, and abort-safe internal mid-script parallel collection.
- Heap caps and catchable OOM where supported; bounded live-peer emergency recovery for GC cells and ArrayBuffer bytes; deliberate fail-closed policy for trace-sensitive, deferred, arena-backed, or otherwise unprovable sites.
- A documented JavaScript memory model and suppression-free TSan boundary.
- Profile-guided contention and GC/lifecycle guidance, including the supported
.gil = truefallback for coordination-heavy workloads. - Security hardening for path confinement, shared-buffer/handle counts, context affinity, buffer/view races, timeout conversion, clone framing/count/depth validation, and nondeterministic random seeding.
Verification
- Unit, C-API, test262-parallel, no-GIL witness, GIL corpus, four-way no-GIL TSan corpus, suppression-free TSan, deterministic fuzz, lifecycle fuzz, and mid-GC fuzz gates.
- WebKit PR-249 audit: 236/259 executable files promoted; all 23 reference-only executables and 5 helpers classified with zero missing or uncategorized entries. Remaining cases require nonportable JSC/JIT/heap/WASM/detached-source premises.
- Focused
threads-profile,midgc-profile, andgc-profileevidence for policy and optimization changes. - Final combined gate: https://github.com/zig-utils/zig-js/actions/runs/29236051962 — 31 success, 1 expected nightly-only skip, 0 failures.
Completed trackers
- Threads: promote remaining WebKit PR-249 reference cases #11 reference promotion/audit
- Threads: maintain the shared-realm JavaScript memory model #12 shared-realm memory model
- Threads: expand concurrent lifecycle and semantic fuzzing #13 lifecycle and semantic fuzzing
- Threads: mature mid-script parallel GC #14 mid-script parallel GC policy
- Threads: profile and reduce no-GIL contention #15 measured no-GIL contention
- Threads: nursery/generational GC and context lifecycle performance #16 nursery and Context lifecycle performance
- Runtime: extend heap-cap emergency recovery beyond current safe allocation classes #30 heap-cap emergency recovery
- Workers: make channel delivery fallible and resource-bounded #40 fallible resource-bounded Worker delivery
Completion criteria
- Final Zig and C semantics are documented and free of accidental compatibility shims.
- No known engine-state race remains in supported paths.
- GC/lifecycle and no-GIL scaling have measured production guidance.
- Mid-script collection has an explicit abort-safe internal support policy and is not exposed as stable API.
- Fuzzer and corpus gates cover the supported lifecycle surface at sustained depth.
- Every executable PR-249 file is promoted or has a current auditable nonportable/feature blocker.
- Worker delivery has explicit ownership, failure, and resource-control contracts.
Authoritative docs
docs/threads/index.mddocs/threads/api.mddocs/threads/memory-model.mddocs/threads/testing.mddocs/threads/production-readiness.mddocs/threads/limits.mddocs/threads/bindings.md
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request
Activity
chrisbbreuer commentedon Jun 10, 2026
Also should keep an eye on this one oven-sh/WebKit#249 - maybe we can reuse quite a bit of it
feat: Context thread-affinity guardrails + C API thread rules (41,898…
docs: vendor WebKit PR-249 threads corpus + specs as reference (41,89…
docs: global-state bindings audit with per-thread/locked/refused ruli…
docs: Phase 2 design note — real agents, AgentGroup, waiter table (42…
feat: refcounted SharedBufferStorage + genuinely atomic Atomics ops (…
feat: Atomics subtree runs one test per worker process (42,097/47,928…
feat: real OS-thread agents + blocking Atomics.wait/notify (42,137/47…
chrisbbreuer commentedon Jun 10, 2026
Progress 2026-06-10: Phases 0–2 are implemented and on main (through
d85c080).docs/threads/bindings.mdaudit (10 globals ruled; found two the plan missed:Math.random's process-global PRNG and the global symbol counter — both fixed in P2), PR-249 corpus vendored underreference/webkit-249/(240 test files + fulldocs/threadsspecs +THREAD.md, head25375a99).src/shared_buffer.zig— refcounted process-wide SAB storage, in-place CAS grow,RetainListper realm;ArrayBufferDatareworked behindbytes()(compiler-forced audit of all ~56 byte-access sites); Atomics RMW/CAS/load/store are single SeqCst hardware ops (raw-bits path also fixed BigInt64 precision); 8-byte-aligned buffers.src/agent.zig—$262.agenton real OS threads with the INTERPRETING.md broadcast rendezvous;Atomics.waitgenuinely blocks (FIFO waiter table, per-ticket condition + timeout,[[CanBlock]]);notifyreally wakes; realsleep/monotonicNow; the cooperative model is deleted. Atomics-subtree workers run one test per process so a deadlock costs one 30s timeout.Score: 41,898 → 42,137/47,928 (87.9%);
built-ins/Atomics308 → 349/388 (89.9%), zero host-fails (no hangs in two full runs). End-to-end concurrency is unit-tested (agent parks → broadcast → blocked wait → cross-thread notify → report).Remaining Atomics failures are mostly
Atomics.waitAsync(101 test files) → that's Phase 3, designed indocs/threads/P2-agents.md(async tickets on the same waiter table + drain-loop integration inContext.evaluate). Also still open from P2: theCanBlockIsFalserunner mode (2 tests).feat: Atomics.waitAsync settles via async waiter tickets (42,190/47,9…
feat: run CanBlockIsFalse tests with main-agent [[CanBlock]] false (4…
chrisbbreuer commentedon Jun 10, 2026
Phase 3 landed (
9999301,28baab2):Atomics.waitAsyncsettles for real — async tickets share the blocking waiters' FIFO lists (spec notify ordering across wait/waitAsync), and the realm's drain tail resolves the promises on notify/deadline/teardown, abandoning unsettleable infinite waits instead of hanging. TheCanBlockIsFalserunner mode closes the last Phase 2 box (both tests now scored and passing).42,230/47,930 (88.1%);
built-ins/Atomics356/390 (91.3%), still zero host-fails. ~34 Atomics failures remain (under diagnosis). Next big block: Phase 4 structured clone.feat: structuredClone + structured serialize/deserialize IR (42,230/4…
chrisbbreuer commentedon Jun 10, 2026
Phase 4 landed (
ba41807): structured clone.src/structured_clone.zigserializes value graphs into a self-contained byte stream (deliberately so — it's the Phase 5postMessagewire format) and deserializes into any realm's arena. Full type coverage incl. cycles/identity, array holes, SAB storage sharing, and{ transfer }(move + detach); DataCloneError for the unclonables. ThestructuredCloneglobal is installed. test262 unchanged at 42,230/47,930 (88.1%) — additive feature, zero regressions, gate run confirms.Still open from Phase 4:
ArrayBuffer.prototype.transferpredates this work (already done). Next: Phase 5 — embedder Worker API (spawn = Context-per-thread + this wire format + the agent group's channel machinery), then Phase 6 (GILThreadAPI, port the vendored PR-249 corpus).485 remaining items
chrisbbreuer commentedon Jul 13, 2026
Closed #13 as complete after
bdb369e7removed the last machine-speed-dependent released-waiter oracle. The deterministic/generated lifecycle, mid-GC, TSan, replay, watchdog, and cleanup coverage meets the tracker acceptance criteria; ongoing depth tuning remains normal CI maintenance. Green gate: https://github.com/zig-utils/zig-js/actions/runs/29232877388chrisbbreuer commentedon Jul 13, 2026
Closed #16 as complete. The non-moving one-cycle nursery, barriers, fallback, telemetry, scratch trimming, regression gates, and pooling guidance meet its acceptance criteria. Based on the focused profile evidence, multi-age/moving/parallel-minor GC does not currently earn its complexity; any future measured bottleneck should get a narrow issue. Green gate: https://github.com/zig-utils/zig-js/actions/runs/29232877388
chrisbbreuer commentedon Jul 13, 2026
Closed #11 as complete. The PR-249 audit is terminal at 236/259 promoted executable cases with all 23 reference-only executables and 5 helpers classified, zero missing/uncategorized entries, and fresh blocker probes confirming the remaining JSC/JIT/heap/WASM/detached-source premises are nonportable rather than hidden passing cases. Focused probe gate passed; full audit gate: https://github.com/zig-utils/zig-js/actions/runs/29232877388
chrisbbreuer commentedon Jul 13, 2026
Closed #30 as complete under the proven per-allocation-class heap-cap policy. GC cells, ArrayBuffer bytes, and safe budgeted side stores recover; trace-sensitive lock sites, deferred generator/iterator stores, allocator-internal locks, arena caps, and unsafe peer-publication states deliberately fail closed with focused witnesses. Universal retry would violate correctness. Green gate: https://github.com/zig-utils/zig-js/actions/runs/29232877388
chrisbbreuer commentedon Jul 13, 2026
Closed #15 as complete. The contention profiler and focused rows now attribute the planned engine/queue/Worker/GC/allocator sources; measured hot paths received regression-gated optimizations; lock ordering and
.gil = truefallback guidance are documented. Additional host sampling is ongoing benchmarking, not missing implementation. Green gate: https://github.com/zig-utils/zig-js/actions/runs/29232877388chrisbbreuer commentedon Jul 13, 2026
Closed #14 with an explicit support decision: the abort-safe mid-script parallel collector is a deeply tested internal optimization, not stable embedder API. Publication/round/deferred abort limits and quiescent/full fallback remain load-bearing correctness policy; the TSan mid-GC/lifecycle matrix is green at https://github.com/zig-utils/zig-js/actions/runs/29232877388.
chrisbbreuer commentedon Jul 13, 2026
Correction: reopened #30 after final #40 CI exposed a real schedule-sensitive failure in the positive live-peer ArrayBuffer heap-cap recovery witness (
tsan-unitshard 1): https://github.com/zig-utils/zig-js/actions/runs/29235127135. The Worker changes are unrelated; #30 remains open until live-peer recovery has a bounded convergence contract and green gate.chrisbbreuer commentedon Jul 13, 2026
Closing the canonical multithreading umbrella as complete. Every child tracker is closed on either implemented behavior or an explicit conservative support decision; the final body records the stable public contract, resource limits, GC/recovery boundaries, nonportable PR-249 tail, verification matrix, and authoritative docs.\n\nFinal combined acceptance gate: https://github.com/zig-utils/zig-js/actions/runs/29236051962 — 31 success, 1 expected nightly-only skip, 0 failures.\n\nFuture bugs, performance evidence, new WebAssembly/JIT scope, or attempts to promote internal mid-script GC policy should open narrow issues with focused witnesses rather than reopening this historical umbrella.
chrisbbreuer commentedon Jul 13, 2026
The preserved pre-rewrite marker experiment was not applicable wholesale, but its allocator concern was valid on current main. Tracked as #54 and fixed cleanly in 35dff22; the obsolete GC layout and deleted worktree dependency were not pushed.
test(gc): exercise full concurrent driver (#1)
chrisbbreuer commentedon Jul 16, 2026
Landed 7a061ce (test(gc): exercise full concurrent driver (#1)). The M3 concurrent-marker test now disables the independently enabled nursery before its workload, so old-byte growth deterministically exercises and closes multiple full concurrent cycles. This fixes a stale test setup that reproduced on the pre-change main baseline; the focused gate passes 2/2.
chrisbbreuer commentedon Aug 2, 2026
048a4b7ecloses a no-GIL race in inherited property cache observation found by the sharded TSan corpus.The canonical property read already held
property_lock, but the advisory cache record reread the prototypeshapeand receivershapeafterward as plain fields.gc-stress/watchpoint-storm.jstherefore raced a main-thread shape transition against a worker VM read. The engine now returns value + immutable holder shape/slot from one locked snapshot and snapshots the receiver shape through the same publication lock.Verification: exact storm case under TSan passes 1/1 with 7 optimizer publications and 4 invalidations; the complete unit suite passes 1,477/1,477 across 10 shards with zero failures, skips, or leaks. Test262 flips: 0.
chrisbbreuer commentedon Aug 2, 2026
8b1ae613fixes a process-wide parallel-sync activation race found by sustained lifecycle seed 35.Object.lockElements()previously reread the enable-only global gate independently inunlockElements(). During concurrent Context bootstrap, a call could bypass while the gate was false, then release after another host thread enabled it, aborting on trace-depth underflow and attempting an unmatched mutex unlock. Acquisition now returns a token consumed by every release site across the interpreter, VM, GC, structured clone, C API, and object helpers.Verification: focused transition witness passes in Debug and TSan; five sustained seed-35 lifecycle runs cover 280 programs with zero failures; the full suite passes 1,478/1,478 across 10 shards with zero failures, skips, or leaks. Test262 flips: 0.
chrisbbreuer commentedon Aug 3, 2026
Landed
235f6124for the unshared WebAssembly.Memory relocation hole exposed by the ReleaseSafe no-GIL gate. Peer mutators now stop only at complete operation boundaries while relocation detaches the historical ArrayBuffer, with the stop and shared-heap GC publication mutually arbitrated. No hidden GIL, suppression, retry, or baseline exemption was added. Evidence: ReleaseSafe 30/30 amplified plus final 1/1, suppression-free TSan 1/1, full units 1,479/1,479.