镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

Add GZIP as a message body compression codec - #11449

Open
Metastarx wants to merge 1 commit into
apache:developfrom
Metastarx:patch/feat-add-a-GZIP-message-compression-code
Open

Metastarx wants to merge 1 commit into
apache:developfrom
Metastarx:patch/feat-add-a-GZIP-message-compression-code

Conversation

@Metastarx

Copy link
Copy Markdown
Contributor

Which Issue(s) This PR Fixes

Brief Description

RocketMQ could already compress message bodies with LZ4, ZSTD and ZLIB, but all three need a third-party library or produce framing that external tools do not read by default. This change adds GZIP as a fourth selectable codec, implemented purely with java.util.zip, so a compressed body is interchangeable with the gzip output that other languages and command line tools produce. The enum, the compressor factory, the system flag bit 4 and the producer's compression-type plumbing are all extended, and the new GzipCompressor follows the same structure as the existing codecs, including level handling through the inherited Deflater and the same stream shutdown and error propagation. Existing behaviour is untouched: ZLIB stays the default, value 0 still resolves to ZLIB, and value 4 now maps to GZIP inside the three bits the system flag already reserves.

How Did You Test This Change?

mvn test -pl client,common - passed (Tests run: 604, Failures: 0, Errors: 0, Skipped: 8)

Signed-off-by: zhang hongyuan <100520587+Metastarx@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 16, 2026 17:25

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@RockteMQ-AI RockteMQ-AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review by github-manager-bot

Summary

This PR adds GZIP as a fourth selectable message-body compression codec, implemented purely on java.util.zip so the compressed output is interchangeable with standard gzip tools in other languages. The change extends the CompressionType enum (value 4), the CompressorFactory, the system-flag bit allocation, and the producer compression-type plumbing.

Findings

  • [Info] GzipCompressor.java:43-46 — The LevelGzipOutputStream inner class reaches into the protected def field inherited from DeflaterOutputStream to call setLevel() after the super constructor has already created the Deflater. This is the standard workaround for GZIPOutputStream not exposing a level-aware constructor, and it works correctly. Worth a one-line comment noting that def is inherited from DeflaterOutputStream so future readers do not have to trace the hierarchy.

  • [Info] GzipCompressor.java:50-73 — The compress path calls finish() then close() inside the try, and the finally block closes both streams again. This is safe (double-close is a no-op for these streams) and correctly releases the native Deflater on the error path. Consistent with the existing ZlibCompressor pattern.

  • [Info] GzipCompressor.java:76-110 — The decompress buffer is allocated at src.length, which is the compressed size. For payloads that expand significantly (e.g. highly compressed binary data), this means many small read() iterations. The existing ZlibCompressor uses the same pattern, so this is consistent. If large-message decompression becomes a hot path later, a larger fixed buffer (e.g. 8 KB) could reduce syscall overhead, but this is not a blocker.

  • [Info] CompressionType.java:82-84 — The comment "4 is the first value that still fits in the three bits reserved by COMPRESSION_TYPE_COMPARATOR (0x7)" is helpful for future contributors. The 3-bit mask supports values 0-7, so values 5-7 remain available for future codecs.

  • [Info] MessageSysFlag.java:49 — COMPRESSION_GZIP_TYPE = 0x4 << 8 correctly occupies the next slot in the 3-bit compression field. Wire-protocol compatible: older brokers that do not recognize value 4 will hit the existing default branch in getCompressionType() and throw, which is the expected fail-fast behavior.

Suggestions

  1. Broker-side awareness: Confirm that the broker MessageDecoder and any downstream consumer tooling (e.g. mqadmin checkMsg, export tools) can handle COMPRESSION_GZIP_TYPE in the sysflag. If the broker decodes compression type only for logging or routing, a new value should not break anything, but it is worth verifying.

  2. Documentation: Consider adding GZIP to the user-facing documentation (e.g. the compression configuration guide) so users know they can set compressType=GZIP on the producer.

  3. CompressionFlagTest.testCompressionFlagNotMatch: The update from 0x4 << 8 to 0x5 << 8 for the "not match" case is correct — value 4 is now valid, so 5 is the right choice for the negative test.

Overall

The implementation is clean, well-tested (round-trip at level bounds, empty body, large body, repeatability, invalid data, null input), and follows the established patterns of the existing codecs. No functional concerns.


Automated review by github-manager-bot

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: add a GZIP message compression codec

3 participants