[GitHub] rdhabalia opened a new issue #3731: [pulsar-broker: message-dispatching] Slow message redelivery for subscription with large backlog and random unack messages.

[GitBox <gi...@apache.org>]

rdhabalia opened a new issue #3731: [pulsar-broker: message-dispatching] Slow message redelivery for subscription with large backlog and random unack messages.
URL: https://github.com/apache/pulsar/issues/3731
 
 
   ### Issue
   **Slow message redelivery for subscription with large backlog and random unack messages.**
   
   Many times consumers don't ack messages properly which leads large number of unack-message list for that consumer and it also leads to huge backlog because `markDeletePosition` can't be advanced due to large number of ack-holes in acked-messages list at the broker.
   
   Because of that broker ends up owning large number of unacked messages in the list and broker tries redeliver all those messages when redelivery triggers. Right now, broker tries to read random messages from the large number of unack-message list which also makes bookie to perform random read across many ledgers. Therefore, message dispatch rate becomes slow and it directly impacts consumer's delivery rate which many times become a bottleneck to drain the huge backlog.
   
   ### Solution
   We need to optimize message-replay for redelivery message in such a way that broker does less number of random reads or at least try to read from the same ledger that can help to utilize bookie-read-cache.
   
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services