[GitHub] [pulsar] rdhabalia commented on a change in pull request #12403: [PIP 107][Client] Introduce chunk message ID

[GitBox <gi...@apache.org>]

rdhabalia commented on a change in pull request #12403:
URL: https://github.com/apache/pulsar/pull/12403#discussion_r745973332



##########
File path: pulsar-common/src/main/proto/PulsarApi.proto
##########
@@ -61,6 +61,9 @@ message MessageIdData {
     optional int32 batch_index = 4 [default = -1];
     repeated int64 ack_set = 5;
     optional int32 batch_size = 6;
+
+    // For the chunk message id, we need to specify the first chunk message id.
+    optional MessageIdData first_chunk_message_id = 7;

Review comment:
       this seems a fundamental issue and it doesn't solve the actual problem. you need messageId `(entryId, ledgerId)` of the first chunk to be present into every chunk of entire message so, you can know what's the first messageId among all chunks when you seek the message. and in this PR you are trying to set that first chunk's messageId (entryId, ledgerId) into every chunk on `ProducerImpl::ackReceived` which doesn't work because you are considering best case where you will receive the first chunk's ack before you send other chunks and that's not guaranteed. Becasuse if you have 10 chunks which will be individual messages and you publish them async and you will get ack of first chunk's message after you publish all other 9 chunks, in that case none of the chunk will have `first_chunk_message_id` and it will not work as Pulsar will claim it should work. 
   This is a tricky to solve at client side because we can not update message metadata based on previous message's ack from broker unless you block the publish of other 9 chunks until you receive ack from first chunk, but that will highly impact publish performance and not usable feature.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org