镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

Runtime: optimize owned string, RegExp, and JSON pipelines #473

Open

Description

@chrisbbreuer

Parent: #459
Depends on: #460, #465, #469

Goal

Build owned high-throughput string, RegExp, and JSON pipelines that are competitive with JSC across ASCII, UTF-8, UTF-16, Unicode-heavy, and adversarial inputs.

Scope

  • Profile and optimize string representation transitions, flat/slice/rope/concat forms, interning/atomization, hashing, equality, search, replace, split, case conversion, normalization, iteration, and property-key conversion.
  • Avoid repeated UTF-8/UTF-16 transcoding while preserving code-unit/code-point semantics, lone surrogates, normalization, locale behavior, and stable embedding views.
  • Extend the owned zig-regex integration with reusable compiled plans, literal admission, capture storage reuse, one-pass/DFA/backtrack selection, replacement expansion, Unicode sets/properties, and bounded failure behavior.
  • Add a streaming or iterative owned JSON parser/stringifier fast path with exact reviver/replacer/toJSON/property-order/cycle/BigInt/error semantics and bounded nesting.
  • Expose tier/runtime/allocator counters and memory profiles for these pipelines.

No-workaround rules

  • No system regex/JSON library, ICU, JSC/V8 code, third-party parser, invalid Unicode shortcut, omitted capture/replacer behavior, or input-specific precomputation.
  • Cache keys and lifetimes must be bounded, content-correct, realm-safe, and no-GIL synchronized.

Acceptance

  • Bench: define the representative dependency-free performance matrix #460 includes small/large, ASCII/Unicode, flat/rope, match/no-match/pathological, parse/stringify, and callback-observable rows.
  • test262, Unicode/CLDR generators, regex dependency tests, fuzzing, OOM, deep input, GC, no-GIL, and TSan pass.
  • Exact-parent profiles prove lower conversion/allocation/backtracking/hash/dispatch cost with bounded memory.
  • Every accepted direct JSC comparison row wins on the declared host with exact output/checksum equality.
  • build.zig.zon retains only the owned local zig-regex dependency for regex semantics.

Foundation

Current canonical semantics live in src/value.zig, src/jsstring.zig, src/interpreter.zig, and the sibling ../zig-regex package.

Activity

  1. added
    enhancementNew feature or request
    runtimeJavaScript runtime and builtin implementation
    performanceMeasured runtime, memory, scaling, or build performance
    on Aug 2, 2026
  2. chrisbbreuer commented on Aug 5, 2026

    @chrisbbreuer
    MemberAuthor

    Landed the reusable RegExp execution slice in two main-branch commits:

    • zig-regex 0da6eb2 adds absolute-offset Matcher.findFrom while preserving original-input anchor semantics and reusing Thompson VM scratch.
    • zig-js 549ff01f keeps one matcher per interpreter/compiled RegExp, routes exec/test/split/replace through it, and releases every owned Match metadata result.

    The representative variant /id=(\d+);\s+state=([a-z]+)/ was rebuilding a Thompson VM for every global exec call because character-class sets do not take the one-pass path. In shared-realm mode those temporary allocations serialized on the shared arena.

    Measured diagnostic impact with the frozen checksum unchanged:

    • quick 4-lane / 40-job variant: about 12 s -> 0.96 s; arena contention 887,532 -> 1,127; spins about 1.50 billion -> 0.99 million; peak RSS about 1.59 GB -> 455 MB;
    • full 4-lane / 800-job variant now completes in about 40 s with 1.48 GB peak and 76 MB retained RSS; the previous implementation produced no 4-lane checkpoint after about 30 minutes;
    • full 8-lane / 800-job variant completes with checksum 1,643,904,000, 1.12 GB peak RSS, and 123 MB retained RSS.

    Validation: zig-regex 49/49 build steps; zig-js 1,503 unit tests with 0 skipped/failed/leaked; benchmark-comparison-test; exact full 4- and 8-lane V15 rows.

    This advances the reusable-plan, capture-lifetime, bounded-memory, and shared-realm portions of #473. The issue remains open: the 8-lane row still reports material shared-arena contention, and the broader string/JSON, Unicode/adversarial, fuzz/OOM/TSan, exact-parent, and direct-JSC acceptance work is not yet complete.

  3. chrisbbreuer commented on Aug 5, 2026

    @chrisbbreuer
    MemberAuthor

    Follow-up publication evidence for the reusable RegExp matcher work is now on main in 96220688: report, raw artifact.

    The clean full-work V15 run completed all 510 phase snapshots in one exact-identity segment. Every shared RegExp base/variant row passed at 1/2/4/8 lanes with frozen checksum, tier, environment, worker, allocation, GC, CPU, and RSS validation. The eight-lane variant records checksum 1,643,904,000 and retained RSS 1,018,609,664 bytes.

    The same row still records 740,010 invocation arena contentions and 47.69 billion spins in the standalone diagnostic probe, so this is not being presented as completion of #473. It closes the specific per-call VM-construction and Match-lifetime defect with full-matrix evidence; shared allocator pressure plus the remaining string/JSON, Unicode/adversarial, fuzz/OOM/TSan, exact-parent, and direct-JSC acceptance work stays open.

  4. chrisbbreuer commented on Aug 5, 2026

    @chrisbbreuer
    MemberAuthor

    Implemented the next RegExp parser boundary in zig-regex 7196733, with downstream coverage in zig-js e8432679. Ordinary legacy and /u character classes now accept an unescaped opening bracket as ClassAtomNoDash; /v continues to reserve it for class-set syntax. Validation: zig-regex 49/49 build steps; focused zig-js boundary test passed with zero leaks. The exact unmodified Octane RegExp row now parses and begins running; it next exposes a separate runtime call failure tracked under #504, so this is a staged conformance gain rather than closure evidence.

  5. chrisbbreuer commented on Aug 5, 2026

    @chrisbbreuer
    MemberAuthor

    The Octane follow-through exposed and fixed a second general RegExp invariant in zig-js 72745298. RegExp literals and internal RegExpCreate operations no longer inherit an enclosing constructor new.target; they allocate from the current realm intrinsic, while actual RegExp construction retains its explicit new.target-aware path. Before the fix, a literal such as /^ba/ inside a constructor received that constructor prototype and lost .exec. Coverage exercises both a compiled function constructor and a class-constructor/tree-walker shape. Validation: 18 RegExp-focused unit tests passed with zero leaks; test/language/statementList remained 80/80. Together with zig-regex 7196733, the exact unchanged Octane RegExp row now completes and validates its expected output in both isolated score and separate attribution modes.

  6. chrisbbreuer commented on Aug 10, 2026

    @chrisbbreuer
    MemberAuthor

    #515 is complete on main in e7d430f. JSON.parse, JSON.rawJSON, and reviver traversal now fail hostile recursive nesting through the existing native/logical stack contract instead of overflowing the process stack. The 50,000-array reproduction changed from SIGSEGV/139 to exact RangeError, JSON.parse remains 77/77, and Debug/ReleaseSafe/TSan plus the full 1,545-test unit inventory are green.

  7. chrisbbreuer commented on Aug 10, 2026

    @chrisbbreuer
    MemberAuthor

    Completed the next owned-string/JSON slice in #516. 88b5b169 makes validated unescaped JSON values and keys borrow their source slice until the runtime establishes final ownership, removing one scratch allocation and byte copy per plain string. b0bcc993 records two exact-parent rows at 0.9787x / 0.9786x retired instructions; wall time remains diagnostic-only and peak RSS is mixed. JSON test262 stays 165/165.

  8. chrisbbreuer commented on Aug 10, 2026

    @chrisbbreuer
    MemberAuthor

    Closed #517 with a spec-shaped JSON Parse Record tree in 535d5cda, replacing the global reviver-source scan while keeping the no-reviver path allocation-free. Exact-parent evidence records 0.188x wall time, 0.1248x retired instructions, and 37.2 MB vs 46.3 MB median peak RSS on the callback-observable width row. JSON.parse test262 remains 77/77.

  9. chrisbbreuer commented on Aug 10, 2026

    @chrisbbreuer
    MemberAuthor

    The #518 escaped-value ownership experiment was measured and rejected, not retained. The managed-backing handoff regressed the seven-pair decision row to 1.021x wall, 1.0017x instructions, and higher peak RSS, so 5a86e04 restores the prior parser/GC path exactly. The workload remains in 9961a43 and the rejected raw evidence is recorded by 500e67a for a future owned string-builder design.

  10. chrisbbreuer commented on Aug 10, 2026

    @chrisbbreuer
    MemberAuthor

    Completed another runtime-pipeline item in #519. JSON.stringify replacer PropertyList construction now preserves specification ordering while using an invocation-local hash index instead of a quadratic duplicate scan. The isolated workload improved to 0.038x wall time and 0.0336x instructions; complete-pipeline instructions improved to 0.8661x. Exact follow-up evidence also drove a bucket-lifetime correction, reducing median peak RSS from 45.5 MB to 34.3 MB. Commits: c823cfba, 2e48c838; evidence: 11833827.

  11. chrisbbreuer commented on Aug 12, 2026

    @chrisbbreuer
    MemberAuthor

    #537 completes another owned-string/frontend boundary. Template cooked values now avoid per-byte arena growth and redundant copies for plain quasis, while escaped and normalized text retains exact owned semantics; tagged raw strings and invalid-escape undefined are preserved. Exact-parent ordinary instruction ratios are 0.816x/0.803x/0.800x at 1K/2K/4K, with escaped/tagged controls at 0.887x/0.924x and identical checksums. Validation and commit links are recorded in #537; #473 remains open for its broader runtime string/RegExp/JSON acceptance work.

  12. chrisbbreuer commented on Aug 14, 2026

    @chrisbbreuer
    MemberAuthor

    Child #556 is complete. JSON.stringify active-cycle membership now uses a shallow-lazy, relocation-stable exact identity index with full OOM/GC/TSan/test262 coverage. Exact-parent evidence: ac9dc52b . The real depth-4096 cycle retired 0.3688x instructions; the separately exposed nested-output recopy cost is tracked in #558.

  13. chrisbbreuer commented on Aug 14, 2026

    @chrisbbreuer
    MemberAuthor

    Child #558 is complete. Runtime 3ec9847f removes recursive nested-output copying; evidence 61652299 records the exact-parent result. At JSON.stringify depth 4,096, retired instructions fell to 0.2341x and median peak RSS to 0.0311x, with Test262 remaining 66/66. Next adjacent JSON work is tracked in #557.

  14. 119 remaining items

  15. glennmichael123 commented on Sep 29, 2026

    @glennmichael123
    Contributor

    #1011 is complete in 3abc2fc.

    A coherent ever-issued slab envelope now rejects impossible conservative roots before any size-class lock. Exact-source attribution on representative_json cuts conservative-interior locks 2,216,853 → 16,863 (-99.24%) and total slab locks 4,615,990 → 2,416,000 (-47.66%) with identical checksums, cell counts, and collection counts.

    Seven interleaved pairs keep every sustained JSON row at 0.98940x–1.00034x retired instructions with sub-0.7% RSD. JSON test262 stays 165/165, required VM agreement 165/165, and Debug/TSan/no-GIL/deep-input gates are green. Full evidence: #1011.

  16. glennmichael123 commented on Sep 29, 2026

    @glennmichael123
    Contributor

    Measured #1012 (coalescing stable-identity epoch invalidation across sweep) in two formulations and closed it as a no-go with no commit. Seven-pair instructions were mixed: parse rows improved only 0.08–0.21%, depth/shallow stringify regressed 0.11–0.12%, and the cold Unicode control regressed 0.96%. The tree is restored to pushed 3abc2fc; evidence and rationale are on #1012.

  17. glennmichael123 commented on Sep 30, 2026

    @glennmichael123
    Contributor

    #1013 landed on main as b0f4b41. Repeated private small-object parses now reuse a byte-verified prepared shape chain while duplicate, indexed, reordered, wide, and encoded-key cases retain the generic path.

    Exact attribution cuts shape requests 1,984,837 -> 4,892 (-99.75%) on both frozen ordinary rows with identical checksums, cell allocations, backing allocations, and collection counts. Seven exact-final pairs improve retired instructions 2.45%/2.69%, CPU 3.99%/5.11%, and wall time 3.97%/5.28%; controls are instruction-neutral. JSON remains 165/165 with 165/165 required-VM agreement, and OOM, moving-GC, no-GIL TSan, and 50k-depth gates are green. Full evidence: #1013.

  18. glennmichael123 commented on Sep 30, 2026

    @glennmichael123
    Contributor

    Opened #1014 from the clean post-#1013 JSON attribution. Seven alternating samples show representative_json at 357.280 ms with JIT versus 350.702 ms with JIT disabled, while only 22,215 of roughly 23.16 million VM dispatches transfer. The first optimizer rejection is the managed "row-" constant; #1014 makes that graph-plannable without embedding a raw GC pointer and proves exact OSR frame import/recovery/rooting. The exact function then exposes a separate 147-node/128-slot region-capacity blocker, which will remain a distinct measured child rather than being hidden by a larger scratch array.

  19. glennmichael123 commented on Sep 30, 2026

    @glennmichael123
    Contributor

    JSON/JIT progress: ac67db5 landed on main under #1017. It removes the primitive-string length guard churn from the frozen representative_json loop, reducing exact-parent median 400.774 ms -> 364.558 ms (9.0%) and OSR entries/deoptimizations 243,515 -> 17,113 with checksum 324952086 unchanged. JIT is still 2.2% behind forced VM, so #1017 remains open for the residual overhead; no parent acceptance claim is being advanced yet.

  20. glennmichael123 commented on Sep 30, 2026

    @glennmichael123
    Contributor

    Pushed 8ca0d15 for #1018. The profiled guarded bit_and path removes 240,107 runtime callbacks per scored frozen representative_json invocation, improves the exact-parent seven-sample median by 1.9%, preserves checksum 324952086, and brings JIT marginally ahead of forced VM. Qualification and anti-specialization controls continue before either child is closed.

  21. glennmichael123 commented on Sep 30, 2026

    @glennmichael123
    Contributor

    JSON native-loop tranche completed and published on main.

    Closed children:

    Evidence:

    This parent remains open. The measured recursive stringify gap is now isolated in #1019 with callback, OOM, GC, no-GIL, and TSan guardrails rather than being hidden inside this umbrella.

  22. glennmichael123 commented on Oct 2, 2026

    @glennmichael123
    Contributor

    JSON.stringify child #1019 has landed its bounded explicit-frame implementation on main in b617e7cf, with follow-up evidence and collector hardening through 388a01f6.

    Verified correctness:

    • focused JSON semantics: 16/16
    • JSON test262: 165 pass, 0 fail
    • forced tree/required-VM: 165 agree, 0 diverge
    • 50,000-level object and array witnesses: catchable RangeError in both tiers
    • moving-GC/OOM coverage, no-GIL witness, and unsuppressed TSan witness: pass

    Clean seven-pair exact-parent CPU evidence preserves all checksums and reports candidate/parent retired-instruction ratios of 0.9988x for depth-4096 and 0.9994x for shallow-4096; the six raw schema-4 artifacts and generated reports are indexed in docs/benchmarks.md.

    The direct-JSC acceptance run remains open. Two complete 126-sample attempts were preserved as rejected evidence rather than cherry-picking noisy samples. #1021 now hardens the collector against competing host work, and its upstream Home-driven validator blocker is independently tracked at home-lang/home#834.

    No replacement matrix is being started while unrelated command-bearing Bun/Playwright jobs remain active. #1019 and #1021 stay open until the Home gate is green and one genuinely clean seven-triplet matrix completes with exact checksums.

  23. glennmichael123 commented on Oct 2, 2026

    @glennmichael123
    Contributor

    #1019 follow-up: test-only commit 9453de45 adds exhaustive allocation-failure coverage for the new stringify frame-cache spill, reentrant fallback, cleanup, and retry paths. Focused gate: 2/2 pass at a 238 MB bounded peak; runtime behavior and test262 score are unchanged.

  24. glennmichael123 commented on Oct 2, 2026

    @glennmichael123
    Contributor

    JSON frame-stack follow-up: #1019 is still complete through exact-parent correctness/CPU evidence; its remaining direct-JSC acceptance is blocked by #1021. The Home compiler abort behind that tool gate is now reduced to a stale diagnostic-array index and documented with the logical fix shape in home-lang/home#834. Fresh reproduction remains exit 134 at a bounded 138 MB peak, so this is not memory pressure. No zig-js consumer workaround has been added.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestperformanceMeasured runtime, memory, scaling, or build performanceruntimeJavaScript runtime and builtin implementation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions