[GitHub] [pulsar] sijie commented on a change in pull request #4400: PIP 37: [pulsar-client] support large message size

[GitBox <gi...@apache.org>]

sijie commented on a change in pull request #4400: PIP 37: [pulsar-client] support large message size
URL: https://github.com/apache/pulsar/pull/4400#discussion_r373326745
 
 

 ##########
 File path: pulsar-client-api/src/main/java/org/apache/pulsar/client/api/ProducerBuilder.java
 ##########
 @@ -295,6 +295,32 @@
      * @return the producer builder instance
      */
     ProducerBuilder<T> enableBatching(boolean enableBatching);
+    
+    /**
+     * If message size is higher than allowed max publish-payload size by broker then enableChunking helps producer to
+     * split message into multiple chunks and publish them to broker separately and in order. So, it allows client to
+     * successfully publish large size of messages in pulsar.
+     * 
+     * 
+     * This feature allows publisher to publish large size of message by splitting it to multiple chunks and let
+     * consumer stitch them together to form a original large published message. Therefore, it's necessary to configure
+     * recommended configuration at pulsar producer and consumer. Recommendation to use this feature:
+     * 
+     * <pre>
+     * 1. This feature is right now only supported by non-shared subscription and persistent-topic.
 
 Review comment:
   This makes me feel that we are not on the right track for implementing this feature. If a user is required to do these many things in order to enable this feature, it seems that we are not doing something correctly. Also, this seems to make the producer and consumer logic very complicated.
   
   Can this chunking feature be implemented in a high level? E.g. over a producer wrapper or a consumer wrapper? E.g. you can chunk a large message into smaller messages and add the chunk id in the message properties. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services