[jira] [Commented] (CAMEL-16790) camel-pulsar is prone to uneven message distribution with large backlog

["Claus Ibsen (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CAMEL-16790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17376266#comment-17376266 ] 

Claus Ibsen commented on CAMEL-16790:
-------------------------------------

This appear to be mostly a pulsar issue so lets see what they say

> camel-pulsar is prone to uneven message distribution with large backlog
> -----------------------------------------------------------------------
>
>                 Key: CAMEL-16790
>                 URL: https://issues.apache.org/jira/browse/CAMEL-16790
>             Project: Camel
>          Issue Type: Improvement
>          Components: camel-pulsar
>         Environment: camel 3.5.0, pulsar 2.7.1, unix
>            Reporter: Kai Levy
>            Priority: Minor
>
> I have filed a bug in the pulsar project about uneven distribution of messages using the Java client:  [https://github.com/apache/pulsar/issues/11008] - there are some reproduction instructions in this ticket. This means that for large backlogs, only the instances that were connected when the messages were produced will be able to process the messages.
> The issue occurs when you use a consumer with `messageListener` and set `receiverQueueSize > 0`. It appears that the camel-pulsar component uses that message listener: [https://github.com/apache/camel/blob/main/components/camel-pulsar/src/main/java/org/apache/camel/component/pulsar/utils/consumers/CommonCreationStrategyImpl.java#L54]
>  
> We observed the issue in our camel application using camel 3.5.0, but I believe it occurs on more versions. If the issue is not fixed in the java client, a convenient workaround could be to use a threadpool calling `consumer.receive()` in a loop instead of `messageListener`.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)