Uneven distributions of consumers across multiple activemq broker nodes

[Lax <LV...@ncr.com>]

Our application has 8 workflow nodes which get connected to 2 activemq broker
nodes. Each workflow node has 40 consumers defined. Ideally, when we start 2
activemq broker nodes followed by 8 workflow nodes - we should see
(4*40=160) consumers on each of the node. Because 4 workflow nodes should
get connected to 1 activemq broker and the other 4 workflow nodes should get
connected to other activemq broker. This kind of even/equal distribution
happens once in a while only. Most of the times we improper distribution
like (3*40) on one activemq broker and (5*40) on the other activemq broker.
Below are the 2 options we tried. Both the options have the same problem.


set :jms_common_broker_url,
"failover:(tcp://<node1_hostname>:61616?keepAlive=true,tcp://<node2_hostname>:61616?keepAlive=true)"


set :jms_common_broker_url, "failover:(tcp://<gtm_dns_server>:61616)"

Can anyone please let us know the options we have to make even consumers
distribution on the activemq broker nodes?



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev-f2368404.html

Re: Uneven distributions of consumers across multiple activemq broker nodes

[Łukasz Dywicki <lu...@code-house.org>]

Its kinda expected with 2 nodes to have such situations as failover is
well... random. With more nodes in failover you will get better load
distribution.

By default failover transport will pick random element from list:
https://activemq.apache.org/failover-transport-reference#using-randomize

So its already shuffled. My silly proposal is to double amount of
elements in failover URI ie. failover:(a,b,a,b) and see if it makes
things better.

Cheers,
Łukasz
--
Code-House
http://code-house.org


On 09.05.2019 02:38, Lax wrote:
> Our application has 8 workflow nodes which get connected to 2 activemq broker
> nodes. Each workflow node has 40 consumers defined. Ideally, when we start 2
> activemq broker nodes followed by 8 workflow nodes - we should see
> (4*40=160) consumers on each of the node. Because 4 workflow nodes should
> get connected to 1 activemq broker and the other 4 workflow nodes should get
> connected to other activemq broker. This kind of even/equal distribution
> happens once in a while only. Most of the times we improper distribution
> like (3*40) on one activemq broker and (5*40) on the other activemq broker.
> Below are the 2 options we tried. Both the options have the same problem.
> 
> 
> set :jms_common_broker_url,
> "failover:(tcp://<node1_hostname>:61616?keepAlive=true,tcp://<node2_hostname>:61616?keepAlive=true)"
> 
> 
> set :jms_common_broker_url, "failover:(tcp://<gtm_dns_server>:61616)"
> 
> Can anyone please let us know the options we have to make even consumers
> distribution on the activemq broker nodes?
> 
> 
> 
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-Dev-f2368404.html
>