[GitHub] [pulsar] tongsucn opened a new issue #13285: [cpp-client] Optimize MessageBuild and SharedBuffer to avoid unnecessary memory copy.

[GitBox <gi...@apache.org>]

tongsucn opened a new issue #13285:
URL: https://github.com/apache/pulsar/issues/13285


   ## Background
   
   In out project, a common use case of Pulsar C++ client is that sending some serialized data to Pulsar, e.g.
   
   ```cpp
   // Payload is a Prorobuf message object.
   Payload pld;
   pld.set_a(xxx);
   pld.set_b(yyy);
   ...
   
   // Build Pulsar message.
   std::string serialized;
   pld.SerializeToString(&serialized);
   auto msg = pulsar::MessageBuilder().setContent(serialized).build();  // Unnecessary copy.
   
   // Send to Pulsar.
   ...
   ```
   
   Here, `.setContent(serialized)` copies `serialized` into the newly-built  message object. If the serialized data is big (say, ~1MB), the performance cost will be significant.
   
   ## Solution
   
   Following optimization should be applied:
   
   1. Add a new function ` MessageBuilder& MessageBuilder::setContent(std::string&& data);`, which takes ownership of the given data without any copy.
   2. Use `std::string` instead of `std::vector` as internal storage of `SharedBuffer` and add a new constructor `SharedBuffer(std::string&&)`, so that `std::string&&` data can be used to move-construct `SharedBuffer` object.
   
   Then, instead
   
   ```cpp
   auto msg = pulsar::MessageBuilder().setContent(serialized).build();
   ```
   
   we can use
   
   ```cpp
   auto msg = pulsar::MessageBuilder().setContent(std::move(serialized)).build();
   ```
   
   to avoid unnecessary memory copy.
   
   If this looks good to you, I can create a new pull request to merge this optimization.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] merlimat closed issue #13285: [cpp-client] Optimize MessageBuild and SharedBuffer to avoid unnecessary memory copy.

[GitBox <gi...@apache.org>]

merlimat closed issue #13285:
URL: https://github.com/apache/pulsar/issues/13285


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org