You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "Inbasasi (via GitHub)" <gi...@apache.org> on 2023/08/17 05:08:05 UTC
[GitHub] [pulsar] Inbasasi opened a new issue, #21009: [Bug] Pulsar Consumer receives same message more than once in SHARED subscription
Inbasasi opened a new issue, #21009:
URL: https://github.com/apache/pulsar/issues/21009
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Version
Apache Pulsar Version - 2.11.1
### Minimal reproduce step
The consumer is as below,
`pulsarClient.newConsumer(Schema.JSON(Schedule.class)).ackTimeout(600000, TimeUnit.MILLISECONDS)
.consumerName("test-consumer").topic(pulsarProperties.getTopic())
.subscriptionName("test-subscription")
.subscriptionType(SubscriptionType.Shared)
.deadLetterPolicy(DeadLetterPolicy.builder().maxRedeliverCount(5)
.deadLetterTopic(pulsarProperties.getTopic() + "-dlq").build())
.autoUpdatePartitions(true).receiverQueueSize(2000)
.subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
.messageListener(consumer).subscribe();`
### What did you expect to see?
The consumer should consume a message only once & that is happening as expected most of the time.
But occasionally, The consumer is consuming the same message twice which is causing consistency issues with the application logic.
### What did you see instead?
The consumer is consuming the same message twice which is causing consistency issues with the application logic.
I confirmed the same by logging the message ID in the consumed message.
` 2023-08-14 02:32:15.221 INFO [1a512eb2-2c23-4b91-bd2e-cc656df3a9c8] [] [pulsar-external-listener-12-1] [RecurringScheduledEventHandler.java:38] c.f.s.s.p.RecurringScheduledEventHandler - >> handling mesage ID 100260:12688:0 - pulsar-105-23 `
`2023-08-14 02:32:15.225 INFO [720cf56f-9b4c-49f7-8f91-81c0fa9bba35] [] [pulsar-external-listener-12-1] [RecurringScheduledEventHandler.java:38] c.f.s.s.p.RecurringScheduledEventHandler - >> handling mesage ID 100260:12688:0 - pulsar-105-23 `
Here, The same message ID `100260:12688:0` is consumed by the same consumer thread `pulsar-external-listener-12-1` within few ms by `1a512eb2-2c23-4b91-bd2e-cc656df3a9c8` & `720cf56f-9b4c-49f7-8f91-81c0fa9bba35`
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui closed issue #21009: [Bug] Pulsar Consumer receives same message more than once in SHARED subscription
Posted by "codelipenghui (via GitHub)" <gi...@apache.org>.
codelipenghui closed issue #21009: [Bug] Pulsar Consumer receives same message more than once in SHARED subscription
URL: https://github.com/apache/pulsar/issues/21009
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] Inbasasi commented on issue #21009: [Bug] Pulsar Consumer receives same message more than once in SHARED subscription
Posted by "Inbasasi (via GitHub)" <gi...@apache.org>.
Inbasasi commented on issue #21009:
URL: https://github.com/apache/pulsar/issues/21009#issuecomment-1681623001
Can someone please help with whether I am making any configuration issues here, which caused the message consumption more than once?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #21009: [Bug] Pulsar Consumer receives same message more than once in SHARED subscription
Posted by "codelipenghui (via GitHub)" <gi...@apache.org>.
codelipenghui commented on issue #21009:
URL: https://github.com/apache/pulsar/issues/21009#issuecomment-1701934298
@Inbasasi, I'm not sure if the last comment can help you or not. I will close the issue first and feel free to reopen it if needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] codelipenghui commented on issue #21009: [Bug] Pulsar Consumer receives same message more than once in SHARED subscription
Posted by "codelipenghui (via GitHub)" <gi...@apache.org>.
codelipenghui commented on issue #21009:
URL: https://github.com/apache/pulsar/issues/21009#issuecomment-1681965002
@Inbasasi Without transactions, the shared subscription on a persistent topic can only provide the at-least-once semantic. You can try to check the broker logs, to see if there are other consumers disconnected or if the topic has been unloaded to another broker. Actually, there are many cases to introduce the duplicated messages, then we introduced the [transactions](https://github.com/apache/pulsar/wiki/PIP-31%3A-Transaction-Support) to support exactly-once semantic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org