You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
OutputHandler chunks can split a surrogate pair, delivering unpaired surrogates to the callback #109
AsyncProcessStreamReader.ReadToEnd (RunCommand/AsyncProcessStreamReader.cs:142-153) reads into a 4096-char buffer and passes each read to onData unchanged. A read can stop between the high and low halves of a surrogate pair. When it does, one chunk ends with a lone high surrogate and the next chunk starts with a lone low surrogate.
Reproduction (run on Linux with a plain OutputHandler)
sh -c "head -c 4095 /dev/zero | tr '\0' a; printf '\360\237\230\200'; head -c 5000 /dev/zero | tr '\0' b"
The callback receives 3 chunks:
The first chunk ends with the high surrogate of 😀.
The second chunk starts with the matching low surrogate.
Why it matters
A callback that encodes or serializes each chunk on its own writes two U+FFFD replacement characters instead of the emoji. Common examples are Encoding.UTF8.GetBytes(chunk) written to a log or socket, or one JSON payload per chunk.
LineOutputHandler is not affected because it reassembles lines. This is a different problem from the BOM and decode issues in #76 and from the concurrent delegates in #95.
Suggested fix / acceptance criteria
In ReadToEnd, when a chunk ends with a high surrogate and the stream has not ended, hold that char back and prepend it to the next chunk. Flush it at end of stream.
Add a test that runs the command above and asserts no delivered chunk ends with a high surrogate or starts with a low surrogate.
Optionally, document that chunks are always well-formed UTF-16.
Priority: Medium. Raw OutputHandler callbacks get malformed UTF-16 at 4096-char read boundaries, so emoji or other non-BMP text gets corrupted in logs, sockets, or per-chunk JSON. LineOutputHandler is not affected, and nothing crashes.
Area / suggested owner:AsyncProcessStreamReader.ReadToEnd, the stream-reading core.
Next step: Hold back a trailing high surrogate and prepend it to the next chunk, flushing at EOF. Add the regression test using the shell repro from the issue.
Activity
matt-edmondson commentedon Oct 6, 2026
Triage
OutputHandlercallbacks get malformed UTF-16 at 4096-char read boundaries, so emoji or other non-BMP text gets corrupted in logs, sockets, or per-chunk JSON.LineOutputHandleris not affected, and nothing crashes.AsyncProcessStreamReader.ReadToEnd, the stream-reading core.Generated by Claude Code