[GitHub] [pulsar] codelipenghui commented on issue #6209: [broker] add feature BrokerDeduplicationAcrossProducers

[GitBox <gi...@apache.org>]

codelipenghui commented on issue #6209: [broker] add feature BrokerDeduplicationAcrossProducers
URL: https://github.com/apache/pulsar/pull/6209#issuecomment-604392425
 
 
   @EugenDueck Sorry for the late response. Multiple producer deduplication looks a little complicated.  I think the highest sequence ID maintained in the broker is insufficient to meet such a demand. Maybe we can use the same mechanism of acknowledging the state of the message. The broker maintains a bitset in memory for the not continuous sequence and a mark sequence ID for the continuous sequence that producers sent to the topic. The producers may cause discontinuities(0, 2, 3, 1, 5) and no longer send 5. So that we can add a time window and if the time window passed, sequence ID 4 will no longer be stored. And this also needs this PR.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services