[I] [Enhancement] Avoid per-message getBytes() for topic and CRC32 property key on the broker write path [rocketmq]

["wang-jiahua (via GitHub)" <gi...@apache.org>]

wang-jiahua opened a new issue, #11002:
URL: https://github.com/apache/rocketmq/issues/11002

   ### Before Creating the Enhancement Request
   
   - [x] I have confirmed that this should be classified as an enhancement rather than a bug/feature.
   
   ### Summary
   
   Cache the topic bytes in `MessageExtEncoder` and the CRC32 property-key bytes in `MessageDecoder` to avoid two per-message `String.getBytes()` allocations on the broker write path.
   
   ### Motivation
   
   - `MessageExtEncoder.encode()` / `encodeWithoutProperties()` call `msgInner.getTopic().getBytes(UTF_8)` for every message. Broker traffic is highly repetitive per thread, and the encoder instance is already `ThreadLocal`, so the same topic string is re-encoded over and over.
   - `MessageDecoder.createCrc32()` re-encodes the constant property key `MessageConst.PROPERTY_CRC32` on every message when CRC32 stamping is enabled.
   
   ### Solution
   
   - `MessageExtEncoder`: keep a single-slot `cachedTopic`/`cachedTopicData` pair (no synchronization needed — the encoder is `ThreadLocal`). Hit returns the cached bytes; miss falls back to `getBytes` and refreshes the slot, so mixed-topic traffic is never worse than today apart from one string comparison.
   - `MessageDecoder`: pre-encode `PROPERTY_CRC32` into a `private static final byte[]`.
   
   ### Verification
   
   - `AppendCallbackTest` 4/4, `AppendPropCRCTest` 2/2 (covers the CRC32 constant path), `LmqDispatchTest` 5/5, `CompactionLogTest` 4/4, `MessageDecoderTest` 7/7; checkstyle clean.
   - 4-node cluster A/B (256-thread sync producer, 1KB, per-arm clean store + restart): broker TPS and young GC per million messages are flat versus baseline — the saving (one short `byte[]` per message) is below GC-count resolution at this load, so this is a cleanup-level allocation reduction, not a measurable throughput win.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Enhancement] Avoid per-message getBytes() for topic and CRC32 property key on the broker write path [rocketmq]

["RockteMQ-AI (via GitHub)" <gi...@apache.org>]

RockteMQ-AI commented on issue #11002:
URL: https://github.com/apache/rocketmq/issues/11002#issuecomment-5523361975

   **Issue Evaluation**
   
   Category: `type/enhancement` | Status: **Evaluated**
   
   **Feasibility:** Feasible
   **Scope:** `store/MessageExtEncoder` (topic bytes caching) + `common/protocol/MessageDecoder` (CRC32 key bytes)
   **Compatibility:** No breaking changes — output is identical
   
   Solid optimization targeting two per-message `String.getBytes()` allocations on the broker write path. The approach is well-reasoned:
   
   - **`MessageExtEncoder`**: Single-slot cache leveraging the existing `ThreadLocal` nature of the encoder — no synchronization needed, and mixed-topic traffic degrades gracefully (one string comparison per miss).
   - **`MessageDecoder`**: Pre-encoding `PROPERTY_CRC32` into a `static final byte[]` is trivially safe.
   
   The verification is thorough (multiple test suites pass, A/B cluster test shows flat TPS/GC). As noted, this is a cleanup-level allocation reduction rather than a throughput win, but it's a net positive with zero risk.
   
   Good candidate for contribution.
   
   ---
   *Automated evaluation by @RockteMQ-AI*


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org