You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/04/22 16:46:01 UTC
[GitHub] [pulsar] BewareMyPower commented on pull request #10330: Fix message buffer being copied from direct memory to heap memory
BewareMyPower commented on pull request #10330:
URL: https://github.com/apache/pulsar/pull/10330#issuecomment-825008206
Here're the monitor statistics after and before this change. The first case (before 00:15) uses this PR's `addBrokerEntryMetadata` method.
<img width="1261" alt="截屏2021-04-23 上午12 19 32" src="https://user-images.githubusercontent.com/18204803/115751365-b7aa1e00-a3cb-11eb-9448-a7312bda0fce.png">
<img width="841" alt="截屏2021-04-23 上午12 19 46" src="https://user-images.githubusercontent.com/18204803/115751387-bb3da500-a3cb-11eb-8b10-26661e6e0984.png">
I used a producer with nearly 200 MB/s write speed. As we can see, the heap memory of second case (after 00:15) increased very fast and the GC happened very frequently.
The difference of these two cases are only the `pulsar-common.jar`.
I'm not an expert of Netty and also very confused why the `CompositeByteBuf` make the whole buffer be copied into the heap memory.
Here's a dumped result that was generated by `jmap -dump:format=b`.
We can see there're many messages in heap memory. The first two bytes are `[14, 2]`, which are equivalent to `magicBrokerEntryMetadata`. See https://github.com/apache/pulsar/blob/37e3180f06188c4d34cd347d46d66bcde3516445/pulsar-common/src/main/java/org/apache/pulsar/common/protocol/Commands.java#L117
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org