[I] [Enhancement] Reduce Remoting header encoding allocation via FastCodesHeader/RocketMQSerializable optimizations [rocketmq]

["wang-jiahua (via GitHub)" <gi...@apache.org>]

wang-jiahua opened a new issue, #10522:
URL: https://github.com/apache/rocketmq/issues/10522

   ### Before Creating the Enhancement Request
   
   - [x] I have confirmed that this should be classified as an enhancement rather than a bug/feature.
   
   ### Summary
   
   Enhance the `FastCodesHeader` interface and `RocketMQSerializable` utility to reduce allocation and CPU overhead during Remoting header encoding/decoding.
   
   ### Motivation
   
   In high-throughput scenarios, every RPC request/response goes through the Remoting header encode/decode path. Current implementation has several inefficiencies:
   
   1. `FastCodesHeader.writeIfNotNull` always converts numeric values (`Long`/`Integer`) to `String` via `value.toString()`, creating transient String objects.
   2. `RocketMQSerializable.readStr` allocates a new `String` even for single-byte ASCII values.
   3. `RocketMQSerializable.mapDeserialize` pre-allocates a `HashMap` with capacity 128, which is oversized for typical headers and wastes memory.
   4. There is no direct way to write primitive `long`/`int` values to the ByteBuf without first boxing them into `Long`/`Integer`.
   
   These small overheads are amplified by millions of RPCs per second in production.
   
   ### Describe the Solution You'd Like
   
   1. Add `writeDecimalLong(ByteBuf, long)` and `writeDecimalInt(ByteBuf, int)` to `RocketMQSerializable` to write primitive decimal numbers directly to ByteBuf without String allocation.
   2. Add `writeLong(ByteBuf, String, long)` and `writeInt(ByteBuf, String, int)` helpers to `FastCodesHeader`.
   3. Optimize `FastCodesHeader.writeIfNotNull` to route `Long`/`Integer` values through the new primitive writers.
   4. Add a `SINGLE_BYTE_STRINGS` cache for ASCII single-byte strings in `RocketMQSerializable.readStr`.
   5. Reduce the initial capacity of `HashMap` in `mapDeserialize` from 128 to 24.
   
   All changes are additive and keep existing method signatures backward compatible.
   
   ### Describe Alternatives You've Considered
   
   - Use `String.valueOf` + `writeStr`: still allocates String objects.
   - Use `ByteBuf.writeAscii` for numbers: Netty's API does not provide direct decimal writing with length prefix required by RocketMQ protocol.
   - Keep `HashMap(128)` in `mapDeserialize`: wastes memory for small maps; 24 is sufficient for typical headers.
   
   ### Additional Context
   
   This is part of a larger effort to reduce Broker allocation rate. Related previous PRs: #10443, #10491, #10485, #10514, #10513, #10517.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Enhancement] Reduce Remoting header encoding allocation via FastCodesHeader/RocketMQSerializable optimizations [rocketmq]

["RongtongJin (via GitHub)" <gi...@apache.org>]

RongtongJin closed issue #10522: [Enhancement] Reduce Remoting header encoding allocation via FastCodesHeader/RocketMQSerializable optimizations
URL: https://github.com/apache/rocketmq/issues/10522


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] [Enhancement] Reduce Remoting header encoding allocation via FastCodesHeader/RocketMQSerializable optimizations [rocketmq]

["RockteMQ-AI (via GitHub)" <gi...@apache.org>]

RockteMQ-AI commented on issue #10522:
URL: https://github.com/apache/rocketmq/issues/10522#issuecomment-4727491890

   **Issue Evaluation**
   
   Category: `type/enhancement` | Status: **Evaluated**
   
   **Feasibility:** Feasible
   **Scope:** Remoting module — `FastCodesHeader`, `RocketMQSerializable`, header encode/decode path
   **Compatibility:** Backward compatible (additive changes only)
   
   Well-structured proposal targeting a genuine hot path. The observations are accurate:
   
   1. **Primitive decimal writers** — Avoiding `Long.toString()` / `Integer.toString()` boxing on every RPC header encode is a valid micro-optimization. Writing decimal digits directly to `ByteBuf` eliminates transient `String` allocation.
   2. **`SINGLE_BYTE_STRINGS` cache** — Reasonable for the common case of short ASCII header values. The 128-entry cache covers all single-byte ASCII and is small enough to have negligible memory footprint.
   3. **HashMap capacity 128 → 24** — Typical Remoting headers have 5-15 entries; 24 (with 0.75 load factor = 32 buckets) is a better default. This reduces per-RPC memory waste without impacting larger headers (HashMap resizes as needed).
   4. **`FastCodesHeader` primitive helpers** — Clean API extension that keeps the existing `writeIfNotNull` contract.
   
   This is consistent with the ongoing allocation-reduction effort (#10443, #10491, #10485, #10514, #10513, #10517). The changes are additive and low-risk.
   
   ---
   *Automated evaluation by RockteMQ-AI*


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org