[GitHub] [pulsar-client-go] cocktail828 commented on a diff in pull request #806: [issue 447] double check message size when client enable compress

[GitBox <gi...@apache.org>]

cocktail828 commented on code in PR #806:
URL: https://github.com/apache/pulsar-client-go/pull/806#discussion_r921743447


##########
pulsar/producer_partition.go:
##########
@@ -520,7 +520,8 @@ func (p *partitionProducer) internalSend(request *sendRequest) {
 	}
 
 	// if msg is too large
-	if len(payload) > int(p._getConn().GetMaxMessageSize()) {
+	if len(payload) > int(p._getConn().GetMaxMessageSize()) && 

Review Comment:
   Yep, this modification just an translation the implement of `https://github.com/apache/pulsar/blob/782132561ac9fc8430ae3ef12913999e5871d3d2/pulsar-client-cpp/lib/ProducerImpl.cc#L431`.
   It indeed do compress twice.
   
   // Wire format
   // [TOTAL_SIZE] [CMD_SIZE][CMD] [MAGIC_NUMBER][CHECKSUM] [PAYLOAD]
   //                                                                                                                |
   //            compressed or uncompressed         {[METADATA_SIZE_0] [METADATA_0] [PAYLOAD_0]} ...
   Acording to the wire format above, SDK cannot compress messages one by one but should compress the whole batchs together.
   
   Another way to avoid 'twice-compress', I think is flushing batchs immediately before and after add big message to batchBuilder.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org