镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

Frontend: avoid copying unescaped string literals #536

Closed

Description

@chrisbbreuer

Parent: #496
Related: #493, #460, #461

Problem

Lexer.lexString eagerly appends every byte of every quoted JavaScript string literal into arena-owned ArrayList storage, even when the literal has no escape and its exact value is already a stable slice of the Context-owned source. Wide JSON-like object literals, generated tables, bundles, and modules therefore pay avoidable allocation, copying, growth, and retained parse-arena bytes proportional to their unescaped string payload.

Scope

  • Add parser-only growth rows for ordinary and escaped quoted-string programs with frozen source construction, checksums, warmup, and timed parse boundary.
  • Return an exact zero-copy source slice for unescaped literals.
  • Lazily materialize decoded storage at the first escape and copy raw spans in batches rather than appending every byte.
  • Preserve quote removal, escape decoding, line continuations, JSON-superset U+2028/U+2029, legacy-octal flags, WTF-8 behavior, diagnostics, source locations, and source lifetime.

Security and correctness

  • No borrowed slice may outlive the owned source backing used by the AST/compiled artifact.
  • Malformed and unterminated literals must retain exact failure classification/location.
  • Escaped decoding must remain OOM-propagating and failure-atomic.
  • Do not special-case benchmark text, intern attacker-controlled literals globally, omit source ownership, or weaken validation.

Acceptance

  • Benchmark source and structural checks land before the runtime change.
  • Fail-on-first-allocation proves an unescaped literal tokenizes without allocation; escaped literals still allocate and decode exactly.
  • 1K/2K/4K ordinary rows show bounded near-linear work and materially lower allocation/instruction cost versus the exact parent; escaped control remains semantically exact.
  • Lexer/parser/frontend suites, forced execution paths, OOM diagnostics, full units, and affected test262 string/source parsing pass with no regressions.

Activity

  1. added
    enhancementNew feature or request
    performanceMeasured runtime, memory, scaling, or build performance
    runtimeJavaScript runtime and builtin implementation
    on Aug 12, 2026
  2. chrisbbreuer commented on Aug 12, 2026

    @chrisbbreuer
    MemberAuthor

    Benchmark-first commit landed on main as 4b5a705b: bench(frontend): add quoted-string growth rows (#536).

    The parser-only runner now freezes 1,024/2,048/4,096 ordinary quoted-string programs plus a 4,096-literal escaped control. Source construction is outside warmup and timing. Each parse validates the top-level declaration, array-literal width, every StringLiteral node, and the sum of decoded bytes.

    Frozen checksums:

    • ordinary 1,024: 50,090
    • ordinary 2,048: 101,290
    • ordinary 4,096: 203,690
    • escaped 4,096 control: 203,690

    No runtime change or performance claim is included in this commit. The next commit will implement the lexer ownership change and compare it against this exact parent.

  3. chrisbbreuer commented on Aug 12, 2026

    @chrisbbreuer
    MemberAuthor

    Completed by 23df967d: perf(frontend): borrow unescaped string literals (#536).

    The lexer now returns unescaped StringLiteral token text as an exact source-backed slice. The first escape switches to arena-owned decoded storage, copies ordinary text as contiguous spans, and preserves the existing validation and legacy-octal paths. The dedicated zero-byte FixedBufferAllocator test proves an ordinary literal performs no allocation; the same allocator returns OutOfMemory for an escaped literal, and the allocation-backed control decodes raw\nspan\u0021tail exactly.

    Exact-parent diagnostic growth evidence used 7 order-balanced pairs, 100 parses per sample, identical frozen checksums, and source construction outside warmup/timing:

    • 1,024 ordinary literals: 465,656,309 -> 339,921,574 retired instructions (0.730x); 4,685,824 -> 3,768,320 bytes median peak RSS (0.804x).
    • 2,048 ordinary literals: 903,796,640 -> 648,690,222 instructions (0.718x); 6,668,288 -> 4,505,600 bytes RSS (0.676x).
    • 4,096 ordinary literals: 1,799,642,376 -> 1,295,094,077 instructions (0.720x); 9,289,728 -> 6,176,768 bytes RSS (0.665x).
    • 4,096 escaped control: 1,981,117,216 -> 1,713,182,483 instructions (0.865x); median RSS remains 9,371,648 bytes; decoded checksum remains 20,369,000.

    These are non-public diagnostic measurements. The standardized attribution collector intentionally rejected this parser-only runner because it does not emit native counter rows; no metrics were fabricated or substituted.

    Validation:

    • lexer suite: 10/10
    • parser suite: 28/28
    • focused frontend suite: 12/12
    • focused TSan allocation/lifetime witness: 1/1
    • full units: 1,594 passed, 1 expected skip, 0 failed or leaked
    • exact-parent test262 string literals, string types, directive prologues, and module code: 758 -> 758; flips 0

    The benchmark-first requirement is satisfied by 4b5a705b. Source ownership is unchanged: tokens and AST nodes already borrow the Context-owned input backing, and decoded text remains arena-owned. All acceptance items are satisfied.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestperformanceMeasured runtime, memory, scaling, or build performanceruntimeJavaScript runtime and builtin implementation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions