You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "BewareMyPower (via GitHub)" <gi...@apache.org> on 2024/03/06 06:51:22 UTC

[I] [Bug] Messages lost if a multi-topics consumer does not receive all messages after seek [pulsar]

BewareMyPower opened a new issue, #22208:
URL: https://github.com/apache/pulsar/issues/22208

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
   
   
   ### Version
   
   master (207335a449f2bc9cdf6782c67f93f8c2fb267271)
   
   ### Minimal reproduce step
   
   Add `Thread.sleep(100);` after https://github.com/apache/pulsar/blob/e2f94dc98dbecb4dc401ba837c54f497ca9d896f/pulsar-broker/src/test/java/org/apache/pulsar/client/api/MultiTopicsConsumerTest.java#L344
   
   Then, run testSeekCustomTopicMessageId
   
   ### What did you expect to see?
   
   The test should pass
   
   ### What did you see instead?
   
   The test is blocked because not all messages are received.
   
   ### Anything else?
   
   The reason is the current `seek` behavior of a multi-topics consumer.
   
   Since https://github.com/apache/pulsar/pull/19158, a `TopicMessageId` can be accepted by a `seek` call on a multi-topics consumer. In this case, the consumer will perform seek operation on a specific internal consumer and other internal consumers are not affected.
   
   However, since all incoming messages will be cleared before a seek operation, there is a race that some of them are not received by consumer before the next seek operation.
   
   For example, given a multi-topics consumer that subscribes `topic1`, `topic2`, `topic3` and each topic has 5 messages.
   - topic1: 0,1,2
   - topic2: 3,4,5
   
   1. Consumer seeks to 1 on topic1. Then the internal consumer will receiving from 2.
   2. Wait for a while, `incomingMessages` might become `[2]` because internal consumer on topic1 could pre-fetch the message `2`.
   3. Consumer seeks to 4 on topic2.
   
   After that, consumer could only receive `[5]` because message `2` was pre-fetched in step 2 and cleared in `topic2`.
   
   From expectations of users,  they have sought to `topic1: 1` and `topic2:4`, so after that, they should receive `[2, 5]` rather than `[5]`.
   
   ### Are you willing to submit a PR?
   
   - [ ] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Messages lost if a multi-topics consumer does not receive all messages after seek [pulsar]

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower commented on issue #22208:
URL: https://github.com/apache/pulsar/issues/22208#issuecomment-2595752088

   #22201 does not fix this issue. I just found this bug when I found testSeekCustomTopicMessageId failed in that PR.
   
   See https://github.com/apache/pulsar/pull/22201#discussion_r1512907076


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Messages lost if a multi-topics consumer does not receive all messages after seek [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #22208:
URL: https://github.com/apache/pulsar/issues/22208#issuecomment-2595239900

   @BewareMyPower is this fixed by #22201 ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Messages lost if a multi-topics consumer does not receive all messages after seek [pulsar]

Posted by "codelipenghui (via GitHub) via commits" <co...@pulsar.apache.org>.
codelipenghui closed issue #22208: [Bug] Messages lost if a multi-topics consumer does not receive all messages after seek
URL: https://github.com/apache/pulsar/issues/22208


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org