Skip to content

Scan raw command literals without copying the remaining input - #745

Merged
nevans merged 1 commit into
ruby:masterfrom
OskarEichler:codex/raw-split-imap
Sep 8, 2026
Merged

nevans merged 1 commit into
ruby:masterfrom
OskarEichler:codex/raw-split-imap

Conversation

@OskarEichler

Copy link
Copy Markdown
Contributor

Summary

Scan RawData literal prefixes with an offset into one binary input string. The current loop copies the remaining tail and deletes the literal prefix for every literal, causing repeated copying as command size grows. Keep literal extraction, validation, flags and output ownership unchanged.

Reproduction

require 'net/imap'
input = ("raw {1024}\r\n" + 'x' * 1024) * 1000
2.times { Net::IMAP::RawData.split(input) }
timings = 7.times.map do
  GC.start
  start = Process.clock_gettime(Process::CLOCK_MONOTONIC)
  3.times { Net::IMAP::RawData.split(input) }
  Process.clock_gettime(Process::CLOCK_MONOTONIC) - start
end
p timings.sort[3]

Verification

  • 1,232 before/after outcomes match, including exact part classes/bytes/flags/frozen state and error classes/messages; 1,232 input-ownership checks pass. A bounded deterministic corpus includes empty, mixed, binary, truncated and invalid inputs. For 250/500/1,000 1-KiB literals, median times for three splits (seven samples) were 0.005862/0.021277/0.077983s before and 0.001138/0.002287/0.004667s after on Ruby 4.0.6. Largest input: 1,036,000 bytes. These are local synthetic measurements, not production claims. Related merged ♻️ Improve RawData.new, Add RawData.split #679 introduced split; no duplicate open optimization was found.
  • Existing rake test on this isolated branch: 1726 tests, 12594 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications, Ruby 4.0.6 via rbenv. The baseline also passes all 1,726 tests; assertion counts vary slightly across runs.
  • Supplemental RuboCop Lint retains the same 48 existing findings. Ruby syntax and git diff --check pass. No repository tests, dependencies or workflows were added or modified; focused reproductions live outside the repository under the consumer's no-new-tests policy.
  • Independent branch based on 6d2ef7a636a1e2449187a83b06ac7a5baa54ead2; the runtime diff from released 0.6.6 is documentation-only before this patch.

Compatibility and limits

No intended public behavior, dependency or Ruby-minimum change. The private extract_literal helper gains an offset keyword. RawData validation remains unchanged; allocation totals and application-level throughput were not measured. Other Ruby/OS versions were not run locally. No production or external IMAP service was used. Local verification does not imply upstream CI approval or comprehensive behavioral coverage.

@nevans nevans added the performance related to CPU use, memory use, latency, etc label Aug 29, 2026
nevans added a commit that referenced this pull request Sep 8, 2026
This is an addendum to #745.  This keeps it more in the original style,
without sacrificing performance, as far as I can tell.

While many style guides (and people) may avoid the regexp globals, this
code base uses them more often than not.  And their function here should
be obvious from the local variable names.  I find this version
_(subjectively)_ easier to read.

Also, using two offsets rather (`text_start` and `literal_start`) and
assigning all of the regexp vars at the start of the loop is arguably
simpler.

And anyway, both versions are `O(n)`, and optimizing `RawData` beyond
that isn't very important.  So, while it's nice for this code to be
faster, keeping the structure easier to understand is higher priority.
@nevans
nevans merged commit 7ca3be6 into ruby:master Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance related to CPU use, memory use, latency, etc

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants