镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

OutputHandler chunks can split a surrogate pair, delivering unpaired surrogates to the callback #109

Closed

Description

@matt-edmondson

What's wrong

AsyncProcessStreamReader.ReadToEnd (RunCommand/AsyncProcessStreamReader.cs:142-153) reads into a 4096-char buffer and passes each read to onData unchanged. A read can stop between the high and low halves of a surrogate pair. When it does, one chunk ends with a lone high surrogate and the next chunk starts with a lone low surrogate.

Reproduction (run on Linux with a plain OutputHandler)

sh -c "head -c 4095 /dev/zero | tr '\0' a; printf '\360\237\230\200'; head -c 5000 /dev/zero | tr '\0' b"

The callback receives 3 chunks:

  • The first chunk ends with the high surrogate of 😀.
  • The second chunk starts with the matching low surrogate.

Why it matters

A callback that encodes or serializes each chunk on its own writes two U+FFFD replacement characters instead of the emoji. Common examples are Encoding.UTF8.GetBytes(chunk) written to a log or socket, or one JSON payload per chunk.

LineOutputHandler is not affected because it reassembles lines. This is a different problem from the BOM and decode issues in #76 and from the concurrent delegates in #95.

Suggested fix / acceptance criteria

  • In ReadToEnd, when a chunk ends with a high surrogate and the stream has not ended, hold that char back and prepend it to the next chunk. Flush it at end of stream.
  • Add a test that runs the command above and asserts no delivered chunk ends with a high surrogate or starts with a low surrogate.
  • Optionally, document that chunks are always well-formed UTF-16.

Activity

  1. matt-edmondson commented on Oct 6, 2026

    @matt-edmondson
    ContributorAuthor

    Triage


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions