[GitHub] [rocketmq] lizhanhui opened a new issue #2614: Avoid unnecessary data copy

[GitBox <gi...@apache.org>]

lizhanhui opened a new issue #2614:
URL: https://github.com/apache/rocketmq/issues/2614


   This is an improvement issue.
   
   When we deserialize network data back to RemotingCommand, we always array-copy the body part.
   
   ```java
    public static RemotingCommand decode(final ByteBuffer byteBuffer) {
           int length = byteBuffer.limit();
           int oriHeaderLen = byteBuffer.getInt();
           int headerLength = getHeaderLength(oriHeaderLen);
   
           byte[] headerData = new byte[headerLength];
           byteBuffer.get(headerData);
   
           RemotingCommand cmd = headerDecode(headerData, getProtocolType(oriHeaderLen));
   
           int bodyLength = length - 4 - headerLength;
           byte[] bodyData = null;
           if (bodyLength > 0) {
               bodyData = new byte[bodyLength];
               byteBuffer.get(bodyData);
           }
           cmd.body = bodyData;
   
           return cmd;
       }
   ```
   
   For messages with a large body, this brings in significant overhead.  
   
   Instead of immediately copying data from direct ByteBuf to heap array, we should keep referencing the sliced frame and release it after use.
   
   This case holds valid for many request code, message sending, pull message(for the client)...  
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] lizhiboo commented on issue #2614: Avoid unnecessary data copy

[GitBox <gi...@apache.org>]

lizhiboo commented on issue #2614:
URL: https://github.com/apache/rocketmq/issues/2614#issuecomment-784186562


   > ```java
   > headerDecode
   > ```
   
   IMO, decode operation can not avoid. Even if use sliced frame and deliver to next processor, decode also needed finally. Sliced frame and increase ref maybe used for gateway scene.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [rocketmq] duhenglucky closed issue #2614: Avoid unnecessary data copy

[GitBox <gi...@apache.org>]

duhenglucky closed issue #2614:
URL: https://github.com/apache/rocketmq/issues/2614


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@rocketmq.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org