You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by G1 <jn...@ibibo.com> on 2009/09/20 18:47:44 UTC
Horizontally scaling JMS listener/consumer application.....in
Active MQ environment
Hi,
In the enterprise we use Active MQ, v5.3 for the messaging infrastructure.
I am integrating 2 applications.
Appl-P ...producer, publishes jms messages on a topic ::
VirtualTopic.Appl-P.SomeName
And I have Appl-C...the consumer consuming the same (by directly listening
on the topic :: VirtualTopic.Appl-P.SomeName
In an attempt to horizontally scale Appl-C;
I hosted Appl-C on more than 1 machine and changed it to consume off queue's
created from the Virtual Topic;
Consumer.Instance-A.VirtualTopic.Appl-P.SomeName,
Consumer.Instance-B.VirtualTopic.Appl-P.SomeName and so forth
But Active-MQ delivers the same message to all the Consumer.Queue's (albeit
within a short delay of each other)
and thus the multiple instances of Appl-C attempt procesing the same message
more than once (which is not desired)
I thought off keeping the multiple instances of Appl-C reading off the topic
(VirtualTopic.Appl-P.SomeName) directly;
(instead of the individual Consumer.Queue's)
but I do not think that will gaurantee that only one instance of Appl-C will
process (and others ignore)
Any suggestions on how to horizontally scale the consumer, Appl-C
ensuring Active-MQ does not deliver the same message to only one instance (@
a time)
Would it be possible to achieve this, exclusively within the Active-MQ
broker? (using features of ver 5.3)
(I do not wish to develop any load balancer component, to intercept
messages, before they are delivered to Appl-C)
~g1
PS: Appl-C .... does not use any database (so there is no way to share data
or state between the multiple instances of Appl-C
--
View this message in context: http://www.nabble.com/Horizontally-scaling-JMS-listener-consumer-application.....in-Active-MQ-environment-tp25530088p25530088.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Re: Horizontally scaling JMS listener/consumer application.....in
Active MQ environment
Posted by Bruce Snyder <br...@gmail.com>.
On Fri, Sep 25, 2009 at 10:28 PM, G1 <jn...@ibibo.com> wrote:
>
> if I can have 2 consumer's on a queue (say A and B) and the message is
> gauranteed to be delivered to only one of
> ..... does the AMQ broker decide which one
> ..... can I as the AMQ user (publisher) control which consumer shud receive
> (in effect achieve a load balance)
In addition to the prefetch, there are options to make a consumer exclusive:
http://activemq.apache.org/exclusive-consumer.html
You can also configure consumers to use a weighted priority:
http://activemq.apache.org/consumer-priority.html
Bruce
--
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'
ActiveMQ in Action: http://bit.ly/2je6cQ
Blog: http://bruceblog.org/
Twitter: http://twitter.com/brucesnyder
Re: Horizontally scaling JMS listener/consumer application.....in Active MQ environment
Posted by Rob Davies <ra...@gmail.com>.
On 26 Sep 2009, at 05:28, G1 wrote:
>
> if I can have 2 consumer's on a queue (say A and B) and the message is
> gauranteed to be delivered to only one of
> ..... does the AMQ broker decide which one
> ..... can I as the AMQ user (publisher) control which consumer shud
> receive
> (in effect achieve a load balance)
>
> ~g1
>
>
> Gabe Westmaas-2 wrote:
>>
>> Both queues and topics can have multiple consumers, ....while the
>> purpose
>> of a queue is that each message is only delivered to one consumer.
>>
>> Gabe
>>
>
> --
> View this message in context: http://www.nabble.com/Horizontally-scaling-JMS-listener-consumer-application.....in-Active-MQ-environment-tp25530088p25622225.html
> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
>
The broker controls which Consumer gets the message because it knows
which of the consumers is the least loaded (messages are delivered
round-robin - but if Consumer A is at its prefetch limit when a
message is due to be dispatched - it will be skipped if Consumer B
isn't at its prefetch limit etc).
Usual practice is to use selectors to have some sort of control of
message delivery, or temporary queues, or unique named queues (e.g.
Foo.ConsumerA) - there's a lot of different ways of achieving the same
thing actually ;)
cheers,
Rob
Rob Davies
http://twitter.com/rajdavies
I work here: http://fusesource.com
My Blog: http://rajdavies.blogspot.com/
I'm writing this: http://www.manning.com/snyder/
Re: Horizontally scaling JMS listener/consumer application.....in
Active MQ environment
Posted by G1 <jn...@ibibo.com>.
if I can have 2 consumer's on a queue (say A and B) and the message is
gauranteed to be delivered to only one of
..... does the AMQ broker decide which one
..... can I as the AMQ user (publisher) control which consumer shud receive
(in effect achieve a load balance)
~g1
Gabe Westmaas-2 wrote:
>
> Both queues and topics can have multiple consumers, ....while the purpose
> of a queue is that each message is only delivered to one consumer.
>
> Gabe
>
--
View this message in context: http://www.nabble.com/Horizontally-scaling-JMS-listener-consumer-application.....in-Active-MQ-environment-tp25530088p25622225.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Re: Horizontally scaling JMS listener/consumer application.....in Active MQ environment
Posted by Gabe Westmaas <ga...@rackspace.com>.
Both queues and topics can have multiple consumers, however the purpose of a topic is so that each consumer gets a copy of the message, while the purpose of a queue is that each message is only delivered to one consumer.
Gabe
-----Original Message-----
From: "G1" <jn...@ibibo.com>
Sent: Wednesday, September 23, 2009 3:51am
To: dev@activemq.apache.org
Subject: Re: Horizontally scaling JMS listener/consumer application.....in Active MQ environment
Hi msmyers,
Are you referring to that I use Queue for the producer appl,
Instead of Appl-P publishing to a VirtualTopic.Appl-P.SomeName, are you
suggesting that it send to a Queue?
But then, how can I have multiple subscriber's @ the listnener end (i want
to horizontally scale the consumer Appl-C)
~g1
msmyers wrote:
>
> Uhh. Don't you want to use a Queue and not a Topic?
>
>
>
--
View this message in context: http://www.nabble.com/Horizontally-scaling-JMS-listener-consumer-application.....in-Active-MQ-environment-tp25530088p25531023.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Re: Horizontally scaling JMS listener/consumer application.....in
Active MQ environment
Posted by G1 <jn...@ibibo.com>.
Hi msmyers,
Are you referring to that I use Queue for the producer appl,
Instead of Appl-P publishing to a VirtualTopic.Appl-P.SomeName, are you
suggesting that it send to a Queue?
But then, how can I have multiple subscriber's @ the listnener end (i want
to horizontally scale the consumer Appl-C)
~g1
msmyers wrote:
>
> Uhh. Don't you want to use a Queue and not a Topic?
>
>
>
--
View this message in context: http://www.nabble.com/Horizontally-scaling-JMS-listener-consumer-application.....in-Active-MQ-environment-tp25530088p25531023.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
Re: Horizontally scaling JMS listener/consumer application.....in
Active MQ environment
Posted by msmyers <ms...@zipwhip.com>.
Uhh. Don't you want to use a Queue and not a Topic?
G1 wrote:
>
> Hi,
>
> In the enterprise we use Active MQ, v5.3 for the messaging infrastructure.
>
> I am integrating 2 applications.
>
> Appl-P ...producer, publishes jms messages on a topic ::
> VirtualTopic.Appl-P.SomeName
>
> And I have Appl-C...the consumer consuming the same (by directly listening
> on the topic :: VirtualTopic.Appl-P.SomeName
>
> In an attempt to horizontally scale Appl-C;
>
> I hosted Appl-C on more than 1 machine and changed it to consume off
> queue's created from the Virtual Topic;
>
> Consumer.Instance-A.VirtualTopic.Appl-P.SomeName,
> Consumer.Instance-B.VirtualTopic.Appl-P.SomeName and so forth
>
> But Active-MQ delivers the same message to all the Consumer.Queue's
> (albeit within a short delay of each other)
>
> and thus the multiple instances of Appl-C attempt procesing the same
> message more than once (which is not desired)
>
> I thought off keeping the multiple instances of Appl-C reading off the
> topic (VirtualTopic.Appl-P.SomeName) directly;
> (instead of the individual Consumer.Queue's)
> but I do not think that will gaurantee that only one instance of Appl-C
> will process (and others ignore)
>
> Any suggestions on how to horizontally scale the consumer, Appl-C
> ensuring Active-MQ does not deliver the same message to only one instance
> (@ a time)
>
> Would it be possible to achieve this, exclusively within the Active-MQ
> broker? (using features of ver 5.3)
> (I do not wish to develop any load balancer component, to intercept
> messages, before they are delivered to Appl-C)
>
> ~g1
>
> PS: Appl-C .... does not use any database (so there is no way to share
> data or state between the multiple instances of Appl-C
>
>
--
View this message in context: http://www.nabble.com/Horizontally-scaling-JMS-listener-consumer-application.....in-Active-MQ-environment-tp25530088p25530503.html
Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.