镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

Threads: make completion rooting and join settlement OOM-atomic #887

Closed

Description

@chrisbbreuer

Parent: #482
Related: #474, #480, #493, #885, #886

Problem

A completed Thread’s return value is not published into ThreadRecord.result until after several GC-capable operations. Because a running peer’s native stack is not conservatively scanned by the parallel collector, threadMain tries to bridge that interval with:

machine.gc_temp_roots.append(machine.arena, out) catch {};

Its own comment identifies this as the return value’s only precise root. If the append fails, microtask drains, waiter settlement, timers, or task pumping can advance collection and reclaim/relocate the value while it is still used.

After publishThreadCompletion, rec.result and rec.settling_joins are already traced under join_mutex; the later ignored temp-root appends are therefore redundant, not the only owners. However, the corresponding promise.resolve/reject calls and final drains also use catch {}, and finishThreadJoinSettlement publishes joins_settled = true even if a pending asyncJoin Promise was never settled or its required reaction publication failed.

#885 owns general Promise state-to-microtask publication. #886 owns already-existing queue transfer/restoration. This issue owns Thread completion’s pre-publication result root and its completion-state transaction.

Required implementation

  • Publish the Thread result into a traced, join-mutex-protected ThreadRecord slot before any post-call GC-capable drain, without making done observable early.
  • Keep rec.result and rec.settling_joins as the authoritative post-publication roots; remove redundant best-effort temp-root appends or turn them into proved non-fallible optimization-only storage.
  • Publish done and joins_settled only at states that accurately describe the result, pending join capability settlement, and rerouted reaction ownership.
  • Propagate or materialize OOM through each asyncJoin result exactly once; never silently leave it pending.
  • Compose with Thread termination, nested asyncJoin, thenable assimilation, local-to-realm queue closure, concurrent marking/moving GC, and teardown.
  • Preserve the current synchronous join rule: it waits for done && joins_settled && exited.

No-workaround rules

  • No swallowed required root/settlement error, reliance on an unscanned running stack, disabling parallel GC, global GIL, leaking Thread records, capping valid pending joins, retrying user callbacks, or marking completion early.
  • A barrier is not a substitute for durable rooting across later collection progress.
  • Do not retain redundant fallible root copies as a false safety claim when the ThreadRecord already owns the post-publication roots.

Acceptance

  • A deterministic allocator-fault witness proves the pre-publication result remains rooted when the temp-root allocator is exhausted.
  • Forced concurrent GC between function return, each drain/waiter step, completion publication, join settlement, and Thread exit cannot reclaim or relocate an unpublished result or Promise.
  • Every pre-existing asyncJoin Promise fulfills/rejects exactly once, or receives the exact engine OOM completion, before joins_settled becomes observable.
  • Root tracing/relocation tests prove rec.result, pending_joins, and settling_joins ownership at each state transition.
  • Nested joins, thenables, throwing Threads, termination, waitAsync/property-async rerouting, and late local-queue closure preserve order and liveness.
  • Bounded heap, moving/parallel GC, lifecycle/mid-GC/threadfuzz, full units, and suppression-free TSan pass without leaks or suppressions.
  • Exact Promise/async/Thread Test262 and PR-249 coverage has zero regressions; documentation makes no unsupported recovery guarantee.

Security boundary

Thread return values and pending join counts are script-controlled, and allocation pressure can coincide with concurrent marking. Required precise roots must be fail-closed engine invariants, not best-effort bookkeeping.

Activity

  1. chrisbbreuer commented on Sep 5, 2026

    @chrisbbreuer
    MemberAuthor

    Source-level ownership proof for the implementation:

    • Binding.traceRoots marks rec.result, rec.pending_joins, and rec.settling_joins unconditionally while holding join_mutex; it does not require done.
    • Therefore the successful-call path can replace its fallible gc_temp_roots.append(out) catch {} with an allocation-free publication of out to rec.result under join_mutex, preceded by the ordinary insertion barrier, while leaving done = false. Neither synchronous join() nor asyncJoin() exposes the provisional slot because both branch on the locked done state.
    • The thrown path already retains its exception in the active Interpreter until completion publication; the reserved OOM value is a Context root. It needs the same explicit state proof but not a speculative allocation.
    • After publishThreadCompletion, the local pending_joins slice aliases the same backing stored in rec.settling_joins, so the ignored result/join temp-root appends should be removed rather than “fixed”; they add failure points without adding ownership.
    • finishThreadJoinSettlement must remain gated on successful settlement of the complete frozen snapshot. Making that failure path exact depends on Promises: make reaction scheduling OOM-atomic #885’s Promise-to-microtask transaction; queue rerouting/restore remains Microtasks: make queue transfer and restoration OOM-atomic #886.

    No code is being mixed into the currently running #875 revision.

  2. chrisbbreuer commented on Sep 8, 2026

    @chrisbbreuer
    MemberAuthor

    Partial implementation landed on origin/main: fix(threads): preserve completion roots and queue handoff (#886, #887) (paired with #886).

    ThreadRecord.result now receives successful values and the exact body exception/OOM completion under join_mutex before any post-call drain, waiter settlement, timer, or task pump. done remains false during those checkpoints. This also fixes a separate observable bug: a throwing microtask previously overwrote the body exception before it was copied to the completion record.

    The record and settling_joins remain the authoritative post-publication roots; redundant best-effort temp-root copies are removed. Settlement iterations and synchronous join reload the result after GC-capable work. The new allocator-fault witness exhausts temp-root storage, then proves result/child retention with native stack scanning disabled before done, tests relocation, and proves collection after ownership is removed.

    Validation on the committed source:

    • Full Debug unit suite: 2,326 passed, 1 skipped, 0 failed, 0 leaked (2,327 linked), across 10 shards.
    • 19 focused tests pass in Debug, ReleaseSafe, and suppression-free TSan (TSAN_OPTIONS=halt_on_error=1).
    • Seed-1 no-GIL threadfuzz: 400/400 default, 300/300 exact-result verify.
    • Exact-parent Test262 diagnostics are identical: Promise 677 → 677, async-function statements 74 → 74, expressions 93 → 93, async-generator statements 301 → 301, expressions 623 → 623, async arrows 60 → 60. 1,828 → 1,828 passing, flips 0, regressions 0 in those measured subtrees.
    • Formatting and diff whitespace checks pass. Local transcripts, before/after diagnostic logs, source hashes, and the full shard plan/logs are retained at /tmp/zig-js-thread-transfer-evidence/ (local evidence, not a portable checked-in report).

    Still outstanding: asyncJoin resolve/reject errors are still swallowed, and early worker setup failure still lacks a complete pending-join settlement transaction. Those paths must deliver an exact failure or commit thenable resolution/job ownership before joins_settled is published. The broader forced-GC, startup/termination, PR-249, lifecycle/mid-GC, and full TSan acceptance checks also remain. This issue stays open; the landed root fix is not a claim that those OOM guarantees are complete.

  3. chrisbbreuer commented on Sep 8, 2026

    @chrisbbreuer
    MemberAuthor

    The remaining completion transaction is now implemented locally and in validation; it is not yet committed.

    • Each asyncJoin capability reserves a settlement descriptor in its final realm/host queue before publication. Failed resolution retains ownership of that slot. The caller reloads the authoritative Promise root and commits the exact engine OOM rejection without allocating or retrying a then getter.
    • Promise settlement now has a shared allocation-free locked commit. Ordinary settlement still performs its fallible reservation before changing state; prepared completion invokes the same commit with already-owned capacity.
    • Both worker startup allocation failures are removed by preparing the local queue and reserving an active-interpreter registry slot before OS-thread spawn. Other registrations respect outstanding reservations. The worker initializes its interpreter after the GC registration gate, then activates its reserved slot without allocation.
    • The entry callable and argument values are now ThreadRecord roots from before spawn through the body call, including tracing and relocation. A worker waiting to register cannot rely on an unscanned native argument stack.
    • Final settlement no longer swallows resolve/reject failures, and joins_settled asserts every prepared token has been consumed before releasing the snapshot roots.

    The initial Debug witnesses pass: competing registry growth under exhausted allocation; entry-root retention/relocation with native scanning disabled; 17 pending joins under first-root, second-root, and GC-capable then-getter OOM; allocation-free thrown-result rejection; delayed thenable adoption; and a throwing getter that reentrantly calls asyncJoin. The JS cases run in both GIL modes. Broader TSan/ReleaseSafe, full-unit, conformance, PR-249, and fuzz checks are underway sequentially.

    One acceptance clarification: joins_settled must mean that the snapshot's resolution/rejection and required job publication have committed. A Promise adopting a thenable may legitimately remain pending after Thread exit; synchronous join must not wait for an arbitrary future thenable result. The new test explicitly proves that join returns while adoption is pending and that all 18 early/late capabilities later deliver the exact result once.

  4. chrisbbreuer commented on Sep 8, 2026

    @chrisbbreuer
    MemberAuthor

    Landed on origin/main: fix(threads): commit prepared asyncJoin completion (#887).

    The completion transaction now reserves required publication before an asyncJoin capability escapes. Fallible root registration and then lookup retain that reservation; an engine OOM becomes the exact rooted OOM rejection with no Promise/queue growth and no replay of user code. The shared locked settlement commit preserves ordinary Promise scheduling as well.

    Pending joins reserve the final realm/host destination. An already-completed Thread preserves the caller's current queue, including worker-local thenable jobs. joins_settled is published after every token is consumed and the final worker queue is handed off; an adopted thenable can legitimately remain pending afterward.

    Both startup OOM paths that discarded pending joins have been removed: queue storage and an active-interpreter registry slot are prepared before OS-thread spawn. Ordinary registrations respect those reserved slots. The worker initializes its interpreter after the GC gate, then registers without allocation. The entry callable and arguments remain traced/relocated ThreadRecord roots until the body result is published.

    Six new regression tests cover 17-capability OOM completion at both root allocations and a GC-capable then getter, exact thrown-result rejection under exhaustion, deferred adoption, reentrant throwing getters, late-join worker affinity, registry reservation pressure/cancellation, and startup root retention/relocation. GC tests disable native-stack fallback where relevant.

    Validation on the committed executable source:

    • Full Debug units: 2,332 passed, 1 skipped, 0 failed, 0 leaked (2,333 linked).
    • 35 distinct focused tests pass in Debug, ReleaseSafe, and suppression-free TSan (TSAN_OPTIONS=halt_on_error=1).
    • PR-249 lifecycle/async-join.js and lifecycle/nested-threads.js: 2/2 in GIL and 2/2 in no-GIL.
    • Seed-1 threadfuzz: 400 default, 300 exact-result verify, 80 broad, 900 mid-GC subcases across 20 seeds, 3,360 lifecycle subcases across 60 seeds. 5,040 generated programs, zero failures.
    • Exact-parent Test262 diagnostics are identical: Promise 677 → 677, async-function statements 74 → 74, expressions 93 → 93, async-generator statements 301 → 301, expressions 623 → 623, async arrows 60 → 60. 1,828 → 1,828 passing, flips 0, regressions 0 in these measured subtrees.
    • Formatting/whitespace checks pass. Local logs, source hashes, clean tracked dependency commits, validation driver, and full shard plan/logs are retained in /tmp/zig-js-887-evidence/ (local evidence, not a portable checked-in report).

    Keeping #887 open for remaining acceptance work, including full-suite TSan and explicit moving-GC fault injection across the completion/checkpoint boundaries. General Promise job-execution OOM after successful job publication remains tracked in #885. A separate source-review finding about capped admission's completion-state lock is now #895; that test-only path is not covered by the default-cap test claim above.

  5. chrisbbreuer commented on Sep 22, 2026

    @chrisbbreuer
    MemberAuthor

    Closing as implemented and accepted.

    The production transaction landed in a8f1b3cc and 9fb57fa0; the final real-moving-GC witness landed in 69400b3f.

    Final acceptance evidence:

    • Temp-root exhaustion publishes the return graph into ThreadRecord.result while done remains false; normal and actual moving collections retain it, rewrite both parent and child pointers, preserve the exact value, and reclaim it after record ownership clears.
    • The new production-compactor witness passes in Debug and TSan. The focused completion set is 4/4, including a mid-script parallel-GC thrown completion and a body exception surviving a throwing microtask drain.
    • The 17-capability settlement matrix covers first-root, later-root, then-getter, and thrown-result failure phases; every capability receives one exact completion before joins_settled, with no outstanding reservations.
    • Startup callable and argument roots are retained and relocated before worker registration; completion, pending-join, settling-join, GIL-task, and property-waiter relocation mirrors their traced roots.
    • PR-249 lifecycle cases pass in GIL and no-GIL modes. The landed validation covered 5,040 deterministic default/verify/broad/mid-GC/lifecycle fuzz programs with zero failures.
    • Linux integration run 34287554681, which contains the production Threads: make completion rooting and join settlement OOM-atomic #887 commits, passed full Debug and TSan unit shards, every Debug/ReleaseSafe/TSan no-GIL corpus shard, all configured threadfuzz and TSan-fuzz profiles, and private ABI. Its sole unrelated failure was the Errors: Proxy and detached-buffer messages are V8-derived, not JavaScriptCore's #903 detached-buffer reference expectation.
    • Fresh exact-parent Promise/async matrix after the final test: Promise 677/677, async-function statements 74/74, expressions 93/93, async-generator statements 301/301, expressions 623/623, async arrows 60/60 on both sides: 1,828/1,828 -> 1,828/1,828, 0 flips.

    The completion state now has durable root ownership and prepared settlement ownership at every fallible boundary, without a global GIL, pending-join cap, callback replay, or leaked Thread record.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions