Repository navigation
Description
Parents: #478
Related: #531, #532, #480, #105, #310
Problem
Current-main diagnostics for representative_shape_transition_fanout still scale through four shared-realm lanes and fall at eight even though Shape publication records zero writer yields and #532 reduced named-property contention to near zero.
One-job diagnostics on 6dbe188f (1,000 scored publications per lane, exact frozen checksums) measured approximately 282k / 391k / 487k / 377k aggregate edges/s at 1/2/4/8 lanes. An amplified eight-lane run of 500,000 jobs per lane completed 4,000,000 scored edges in 16.538 seconds with checksum 4,500,013,000,000.
A three-second native sample of that exact amplified invocation recorded these collapsed leaves:
GcCellBacking.allocFn: 2,649 samples;swtch_pri: 1,085 samples;GcCellBacking.freeFn: 511 samples;GcCellBacking.ownedChunkIndexLocked: 91 samples.
The process reached about 4.1 GiB physical footprint because the row intentionally publishes unique retained Shape edges. The hot cell-slab size class still has one shared allocation lock. #105 amortizes explicit same-kind allocation batches, but this workload publishes one observable edge per operation and therefore enters the single-cell backing path. Repeating the rejected pressure-counter sharding from #310 would not address this lock.
Scope
- Add opt-in exact attribution for cell-backing allocation/free acquisitions, contention/spins or wait, size class, fresh/reused slots, chunk growth, and shard/owner selection.
- Freeze a benchmark-first allocation-pressure row or reuse the existing base/variant/contended Shape rows only after proving that the timed work and counters isolate this backing path.
- Implement bounded Context-owned allocation sharding or another measured design that lets independent mutators reserve/reuse same-size cell storage without one hot size-class lock.
- Preserve batch allocation/publication, realm ownership, published-bit ordering, address classification, stable identity epochs, moving/parallel GC, compaction, sweep batching, and exact teardown.
- Keep retained unused capacity bounded and reclaimable. Any mutator-local reserve needs an explicit publication, revocation, thread-exit, Context-destroy, and OOM rollback proof.
No-workaround rules
- No GIL, hidden serialization, global mutable TLS cache, unbounded per-thread slabs, leaking unpublished reservations, reduced scored work, prepublishing benchmark keys, predictable attacker-controlled shard selection, disabled compaction, or delayed safepoints.
- Do not retry Perf: shard cooperative GC allocation-pressure accounting #310's rejected pressure-counter design without new causal evidence.
- Do not touch stacks, queues, jobs, Promise/microtask, or Worker lifecycle code.
Acceptance
- Benchmark/counter source, jobs, checksums, warmup, timed boundary, and 1/2/4/8 lane inventory land before the runtime change.
- Exact attribution accounts for every eligible cell-backing allocation/free acquisition and identifies the affected size classes and ownership path.
- Independent shared publication throughput is non-decreasing across 1/2/4/8 lanes on stable repeated samples; same-parent serialization remains visible and bounded.
- Direct/independent allocation, short batches, mixed size classes, reused/fresh slots, and retained RSS do not regress materially.
- Allocation failure and partial-prefix ordering remain exact; bounded heap, thread exit, Context teardown, moving/parallel GC, compaction, stable identities, realm routing, and sweep reuse pass.
- Suppression-free TSan, threadfuzz profiles, full units, and exact Object/Array/Proxy test262 pass without regressions.
- Seven-pair exact-parent attribution and final scaling evidence publish wall, CPU, RSS, allocation/GC counters, instructions/cycles/energy availability, thermal state, and complete dispersion with no discarded samples.
Security boundary
Allocation width, size class, property names, mutator schedules, collection timing, and OOM pressure are attacker-controlled. The design must remove engine-state contention without exposing half-published cells, cross-realm ownership confusion, reusable-address identity bugs, unbounded retention, or allocator races.
Activity
bench(gc): attribute cell-slab lock paths (#559)
[-]GC: shard hot cell-slab allocation by mutator[/-][+]GC: amortize hot cell-slab lock traffic in shared realms[/+]chrisbbreuer commentedon Aug 14, 2026
bench(gc): attribute cell-slab lock paths (#559) landed the benchmark-first schema-v11 foundation on
main.What is now measured exactly:
The unprofiled lock loop is unchanged. Only opt-in attribution contexts increment these counters, and uncontended acquisition counters are recorded before the bucket critical section.
An exact-final diagnostic used
representative_shape_transition_fanout, 100,000 jobs per lane, eight shared-realm lanes, one fresh profiled process, and the existing configuration/warmup/invocation boundaries. The invocation delta preserved checksum180002600000and recorded:This rejects a conventional contended-lock/spin-wait diagnosis. The runtime candidate now needs to reduce or amortize acquisition/coherence traffic itself. Per-mutator ownership remains one possible mechanism, but it should not land unless the exact-parent A/B earns its complexity and retained-memory cost.
Validation:
zig build benchmark-comparison-testpassed;The diagnostic is causal instrumentation, not publication evidence. No throughput or regression claim is being made from this one run.
chrisbbreuer commentedon Aug 14, 2026
Rejected candidate: collector-exclusive cell-slab lock elision.
The candidate used the existing exact cooperative rendezvous and the
hasRunningJsThreads() == falseevaluation boundary to bypass bucket atomics only while one collector owned the heap. Concurrent allocation, publication, classification, and abort-safe concurrent marking retained their locks. A focused moving-nursery witness passed suppression-free TSan.The schema-v11 counters confirmed that the mechanism did what it claimed on the eight-lane, 100,000-jobs-per-lane
representative_shape_transition_fanoutdiagnostic:180002600000.It did not earn the runtime complexity. Four alternated samples per side measured:
This was a diagnostic screen, not publication evidence, but it is decisively non-promising: the entire candidate was discarded and no tracked change remains. The result also narrows the next experiment: merely removing collector-only uncontended atomics does not address the scaling loss. A future candidate must target the high-frequency live-mutator ownership/allocation/publication path and still prove bounded memory, exact publication, and a same-parent performance win.
perf(gc): route exact object stores through managed barrier (#559)
chrisbbreuer commentedon Aug 14, 2026
Landed the first runtime optimization slice in 6b81b734: perf(gc): route exact object stores through managed barrier (#559).
What changed:
Diagnostic shared-realm attribution (8 lanes x 100000 jobs, exact checksum 180002600000):
The timing and counters are diagnostic, not publication evidence. Unlike the rejected quiescent collector-lock elision above, this slice did not regress the measured median.
Validation:
#559 remains open. The measured invocation still carries about 4.0M tolerant ownership acquisitions plus allocation/publication/relocation traffic; the next slice should identify the remaining exact managed store families before changing allocator synchronization.
bench(gc): split slab ownership attribution (#559)
chrisbbreuer commentedon Aug 14, 2026
Landed schema-12 ownership-path attribution in 1b4727e9. The seven ownership subpaths must now sum exactly to the ownership total. After 6b81b73, the 8-lane x 100000-job representative_shape_transition_fanout invocation retained checksum 180002600000 and attributed all 3,999,063 residual ownership acquisitions to exact-allocation validation; stable identity, conservative interior classification, realm lookup/scan, and allocator resize/remap were all zero. This rules out conservative scanning and allocator callbacks for this workload and narrows the next slice to audited interpreter/VM managed-cell stores. Arbitrary-pointer boundaries remain tolerant. Validation: all benchmark meta/self-tests, focused profiler and real-backing tests, and full units (1,634 passed, 1 skipped, 0 failed, 0 leaked). Diagnostic only, not publication evidence.
perf(gc): use exact barriers for managed environments (#559)
chrisbbreuer commentedon Aug 14, 2026
Landed the managed-environment runtime slice in db127f4b. Environment binding and disposal stores now use the exact-managed barrier only when the immutable gc_managed bit proves the owner is a heap cell; embedded root, arena, test, and arbitrary-pointer owners remain tolerant. Against exact pushed parent 1b4727e at 8 lanes x 100000 jobs, invocation ownership validation fell 3,999,063 -> 23 and total slab-lock acquisitions fell 13,664,454 -> 9,665,416, checksum 180002600000 unchanged. Four alternating diagnostic samples moved median wall time 2,699,201,042 ns -> 2,574,367,354 ns (-4.6%). Validation: focused generational/incremental/shared-moving GC; focused TSan old-owner/shared-moving; full units 1,634 passed, 1 skipped, 0 failed, 0 leaked. Diagnostic only, not publication evidence. #559 stays open: the former ownership hotspot is now effectively exhausted for this workload, so the next step should reassess the remaining allocation/publication/relocation mix rather than broadening strict ownership assumptions.
bench(gc): isolate shared fixture preparation (#559)
chrisbbreuer commentedon Aug 14, 2026
Landed the benchmark-boundary correction in b3d41abe:
bench(gc): isolate shared fixture preparation (#559).The causal finding is that
__benchmarkPreparestill ran inside__benchmarkRunShared. Forrepresentative_shape_transition_fanout, that meant each scored call constructed the unique managed StringCell key fixture and armed cooperative GC; because requested collection is serviced at the next evaluation boundary, fixture collection/relocation also appeared in the worker invocation delta. The prior 64-byte allocation/publication/relocation inventory therefore did not describe concurrent shape publication.The runner now builds the exact same fixture before the timer, completes preparation-triggered quiescent collection before the GC snapshot, and then times only worker creation/join plus the selected workload. Workloads without a prepare hook do not receive an artificial collection.
Exact 8-lane x 100000-job attribution, checksum
180002600000unchanged:Corrected three-sample medians for 100000 jobs per lane:
The workload still falls from four to eight lanes, but the scored path now performs only 82 slab acquisitions across 800,000 edges. Cell-slab sharding cannot logically explain or repair that residual. The remaining shape/publication scaling investigation belongs to #531 (and the broader shared allocation/object program remains #478).
Validation: benchmark comparison meta/self-tests, frozen representative-matrix self-test, actual profiled 8-lane invocation, actual 1/2/4/8 lane sweep, and a no-prepare shared arithmetic telemetry control. Test262 flips: 0; this changes benchmark timing/counter boundaries only.
This resolves #559 as a benchmark-causality correction rather than landing an unjustified allocator-sharding mechanism. The earlier exact barrier improvements remain valid general reductions in redundant ownership validation.