[jira] [Comment Edited] (AMQ-762) Message Group based load balancing not well distributed across brokers

["Matt Pavlovich (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AMQ-762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17243541#comment-17243541 ] 

Matt Pavlovich edited comment on AMQ-762 at 12/3/20, 10:12 PM:
---------------------------------------------------------------

This would require significant cross-communication between brokers. 

A couple of alternative approaches:

1. Have each app register a consumer against each broker and allow Message Groups to filter out per-broker.
2. Pre-filter messages into separate queues and optionally use composite queue consumption to consume from multiple queues from a single consumer
3. Filter messages using a destinationInterceptor + selector

NetworkConnectors also support selectors, which could help in scenarios where header values are known ahead of time.

This ticket is scheduled to close in 30 days.


was (Author: mattrpav):
An approach that seems to have success here would be to have each app register a consumer against each broker and allow Message Groups to filter out per-broker.

NetworkConnectors also support selectors, which could help in scenarios where header values are known ahead of time.

This ticket is scheduled to close in 30 days.

> Message Group based load balancing not well distributed across brokers
> ----------------------------------------------------------------------
>
>                 Key: AMQ-762
>                 URL: https://issues.apache.org/jira/browse/AMQ-762
>             Project: ActiveMQ
>          Issue Type: Improvement
>    Affects Versions: 4.0
>         Environment: Active MQ 4.0, Lingo 1.1
>            Reporter: Sanjiv Jivan
>            Priority: Major
>              Labels: close-pending
>             Fix For: 5.x
>
>         Attachments: ASF.LICENSE.NOT.GRANTED--lingocluster.zip
>
>
> I started 2 servers, each of which have an embedded broker. A shell based chield sends messages to 30 different message groups (using command "register <message group>" in the samepl app provided. Only 2 mesages are received by server1, 3 by server2 and 25 by server3. The load balancing distribution is highly unenen. 
> As suggested, I also tried setting a low destination queue prefetch value (consumer.prefetch=1) but the result was the same.
> To run sample :
> 1. Unzip attached file and run "maven.bat" from the oot directory (Maven 1.0)
> 2. Open 3 DOS boxes in the dist\bin folder and run "startoptimizerPooled.bat", "startOptimizerPooled2.bat" and "startOptimizerPooled3.bat" in each DOS box respectively.
> 3. Step 2 starts a network of 3 servers apps which have an embedded broker. The Spring configuration files for each of these servers is in the dist\conf directory.
> 4. Open another DOS box in dist\bin and start a test client by running "startClientShell.bat"
> 5. This command driver test client accepts commads in the form 
> "register <message group>"
> "close <message group>"
> and "exit"
> NOTE: The command "close <message group>" is supposed to close/reset  the message group by issueing a "JMSXGroupSeq" header as described here : http://www.activemq.org/site/message-groups.html
> 6. Try sending several messages to the server by issuing several commands like "regeister A", "register B", "register C" and so on.. You'll see the highly uneven distibution of messages. Only one or two messages are received my 2 servers while the third one receives a majority of the messages.
> Please let me know if you have trouble running the sample or replicating the issue. 
> Thanks,
> Sanjiv



--
This message was sent by Atlassian Jira
(v8.3.4#803005)