镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

GC: diagnose incomplete parallel sweep in finalization asyncJoin mid-GC witness #907

Closed

Description

@chrisbbreuer

Parent: #476. Related: #491, #502, #812, #896, #905.

Measured failure

On macOS arm64, Zig 0.17.0-dev.1441, the unchanged Debug command zig-out/bin/threadfuzz midgc 20 1 reported:

seed 2: finalization asyncJoin did not finish a parallel collection
threadfuzz midgc: 900 programs from seed 1, 1 failures, wall-ms 646543

The tested local integration was 1d4f9270, containing the checkpoint-error ownership and Context Thread-identity candidates above origin/main 65dbb404. Those candidates are not yet attributed as the cause. The witness attempted parallel collection but observed no completed parallel sweep. This is a failed collection-progress assertion, not a measured race, UAF, or failed JavaScript result assertion.

Three alternating, unchanged midgc 1 2 replays on the origin/main baseline and three on the candidate each passed 45/45 programs. Baseline wall-ms: 35373, 32827, 32486. Candidate wall-ms: 33071, 36007, 32773. These replays do not erase the original 1/900 failure or establish introducing revision/root cause. #812 involved different cases and has its own later Linux passing evidence; this report does not reopen that distinct issue.

Required work

  • Capture attempted/completed/aborted cycles, publication timeouts, round/deferred limits, born growth, and finish retries for this exact finalization/asyncJoin failure. Keep the diagnostic in the runner so another occurrence is actionable.
  • Establish a repeatable source/ownership or progress cause, comparing exact-parent and candidate behavior without running other corpus jobs or units concurrently.
  • Correct the responsible engine/collector boundary and preserve exact join results, unregister suppression, cleanup counts/sums, and worker/waiter roots.
  • Keep the existing workload, assertion, denominator, and production budgets. Do not add sleeps, increase allocation pressure to conceal failure, force an extra collection/drain, suppress GC, relax the assertion, or retry until green.
  • Verify full normal and suppression-free TSan mid-GC profiles, lifecycle/no-GIL integration, and full units; retain failed evidence alongside subsequent runs.

Activity

  1. chrisbbreuer commented on Sep 8, 2026

    @chrisbbreuer
    MemberAuthor

    Diagnostic support landed in e5fe2513 — tools(threadfuzz): report finalization collection progress counters (#907). The existing failed completion branch now emits the parallel-GC counter snapshot; workload, assertion, denominator and budgets are unchanged.

    The tested source tree is identical to the earlier local e1a044ab tree; commit descriptions were finalized with the measured results before push. Original local 1d4f9270 likewise corresponds to pushed 4b6b5ec2. The original failure remains valid and unattributed. No closure: a repeatable cause and passing full normal/TSan collector acceptance are still required.

  2. chrisbbreuer commented on Sep 9, 2026

    @chrisbbreuer
    MemberAuthor

    Additional Linux evidence on 09279120, with the same engine source as the landed diagnostic change:

    These passing runs add cross-platform evidence but do not explain or erase the original local incomplete-sweep failure. Keeping this issue open until the cause is identified; no assertion, workload, or timeout was relaxed.

  3. chrisbbreuer commented on Sep 9, 2026

    @chrisbbreuer
    MemberAuthor

    Additional measured evidence on 8d7a3b7a — fix(finalization): retain cleanup roots across reentrant checkpoints (#906): the unchanged full seed-1 mid-GC workload completes 900 programs, 0 failures, wall-ms 604422. The new finalization ownership witnesses also pass actual nursery movement and suppression-free TSan.

    This is another passing run, not an explanation of the original incomplete-sweep failure. #907 remains open; no collection-completion assertion, workload, or timeout was relaxed.

  4. chrisbbreuer commented on Sep 21, 2026

    @chrisbbreuer
    MemberAuthor

    Resolved by 50659f98 — fix(gc): bound parallel retry cooldown (#907).

    Root cause was collector progress policy, not a missing root or unsafe sweep. Exact seed 2 reproduced on the historical diagnostic revision at repetition 29:

    • attempts 1, collections 0, aborts 1
    • publication-timeout aborts 0, deferred aborts 0
    • round-limit aborts 1
    • generations 64, peer publications 191
    • born-growth rounds 62, finish retries 2

    Every peer published and the cycle remained abort-safe, but continuous born allocation consumed the 64-round budget. The abort then armed a one-millisecond wall-clock cooldown. The short workload spent its remaining eligible safepoints inside that cooldown and returned without another ordinary pressure-driven attempt.

    The fix keeps the existing 64-round and 25 ms publication budgets. It bounds the cooldown by one otherwise-eligible safepoint and bypasses it earlier when the running interpreter population shrinks because a peer exits or parks. Read-only failure telemetry now includes publication wait totals/maxima, running/parked peer observations, finish maxima, round extensions, pause time, backoff skips, and both bypass paths. threadfuzz midgcfinal reproduces the exact finalization/asyncJoin case without changing its workload or oracle.

    A first full post-change run retained a distinct Thread.restrict progress failure (900 programs, 1 failure). Its new snapshot showed the same policy defect: one 397.6 ms attempt, 64 generations, all five peers publishing, 57 born-growth rounds, 7 finish races, no publication/deferred abort, then two backoff skips and no remaining attempt. Capping the cooldown at one eligible safepoint fixes both witnesses.

    Measured final evidence:

    • exact finalization seed 2: 100/100 repetitions pass (historical policy failed at 29)
    • exact Thread.restrict seed 13: 100/100 repetitions pass (topology-only candidate failed at 33)
    • normal mid-GC: 900 programs, 0 failures, wall-ms 585490
    • suppression-free TSan mid-GC: 90 programs, 0 failures, wall-ms 391516
    • suppression-free TSan lifecycle/no-GIL: 112 programs, 0 failures, wall-ms 83703
    • focused M3 units: 6/6, no leaks
    • full units: 2436 passed, 1 platform skip, 0 failed, 0 leaked, 487.9 s across 10 shards

    Five ReleaseFast profile samples kept performance in range. Median wall time was about 534 ms after versus 553 ms before; attempts changed from 10–13 before to 6–10 after. No collection workload, assertion, denominator, allocation pressure, sleep, round budget, publication budget, forced collection, or retry-until-green loop was added. The internal-only path flips 0 test262 cases.

  5. chrisbbreuer commented on Oct 8, 2026

    @chrisbbreuer
    MemberAuthor

    A residual explicit-request case was found and fixed in #1038 / bc2f22e. The #907 one-checkpoint bound remains for opportunistic retries; pending shell/host GC requests now bypass that cooldown and heap threshold, then clear only after a finishing sweep.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions