You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by TonyTobin <ca...@hotmail.com> on 2010/04/21 17:35:39 UTC
Understanding the Exclusive Consumer
In the ActiveMQ doc http://activemq.apache.org/exclusive-consumer.html it
states
The broker will pick a single MessageConsumer to get all the messages for a
queue to ensure ordering. If that consumer fails, the broker will auto
failover and choose another consumer.
Is this a consumer master slave relationship, the broker will send all
messages to the designated consumer, and the consumer passes the messages
onto the other consumers. Or does the broker send the messages to all the
consumers, but only one consumer acknowledges them, so if that consumer
dies the other consumer can then pick up where the first consumer died.
If there is a link that covers Queue consumer clusters in more depth, I
would be grateful if you could post it.
Thanks Tony
Thanks Tony
--
View this message in context: http://old.nabble.com/Understanding-the-Exclusive-Consumer-tp28288049p28288049.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Understanding the Exclusive Consumer
Posted by TonyTobin <ca...@hotmail.com>.
Thanks Gary
You have been a great help
Gary Tully wrote:
>
> There are no real options there, it is handled by the broker and it
> simply replays any unacked messages to the next consumer.
> For the consumer app, there is the possibility that the second consumer
> can
> get a duplicate if there is an outstanding ack at the time of failover, so
> the consumers needs to be able to deal with this. Even with transactions,
> with an outstanding commit reply, the consumer cannot know if the ack was
> delivered so the message may or may not get redelivered. XA 2PC is the
> only
> way to avoid the duplicate problem if the consumers need
> to guarantee persistence. Taking an optimistic approach and dealing with
> duplicates when they occur by remembering the message id in the store data
> is the other option.
>
> Not sure of other links that deal with this, but the info above may give
> you
> key words that can help in your research.
>
>
> On 22 April 2010 10:13, TonyTobin <ca...@hotmail.com> wrote:
>
>>
>> Thanks Gary,
>>
>> That makes sence.
>>
>>
>> Are there any decent links on the different strategies for message
>> persistence within consumers.
>>
>> A consumer is building up messages and then falls over, how the other
>> consumer takes over. Managing failover within the consumer cluster.
>>
>> Thank you for your help
>>
>> Tony
>>
>>
>> Gary Tully wrote:
>> >
>> > yes, but the consumers all have to be present, they already content for
>> > the
>> > lock.
>> >
>> > imagine an app that has a hot standby, both the primary and hot standby
>> > have
>> > started consumers. The broker delivers messages to the first consumer,
>> the
>> > primary (because it got there first) and when/if it fails, it will
>> deliver
>> > messages to the second consumer (the hot standby). So the broker
>> chooses
>> > but
>> > only from among the current active consumers.
>> >
>> > On 22 April 2010 09:09, TonyTobin <ca...@hotmail.com> wrote:
>> >
>> >>
>> >> Thank you for your reply Gary,
>> >>
>> >> In the documentation for exclusive Consumer, it states
>> >>
>> >> "The broker will pick a single MessageConsumer to get all the messages
>> >> for
>> >> a
>> >> queue to ensure ordering. If that consumer fails, the broker will auto
>> >> failover and choose another consumer"
>> >>
>> >> Doesnt this mean that the broker selects the next consumer, rather
>> than
>> >> the
>> >> consumer going for the lock.
>> >>
>> >> Thanks again for your reply.
>> >>
>> >>
>> >>
>> >> Gary Tully wrote:
>> >> >
>> >> > exclusive consumer is a simple solution to the case where there are
>> >> many
>> >> > consumers but only one should be active at any time. Rather than
>> some
>> >> > application logic keeping track of which one should be active, they
>> all
>> >> > have
>> >> > a go and the broker ensures exclusive access.
>> >> >
>> >> > So each consumer connects but only one of them gets the exclusive
>> lock
>> >> and
>> >> > gets the messages. When it dies, the next consumer in line gets the
>> >> > messages. If effectively allows consumers to stack waiting for the
>> >> current
>> >> > consumer to die.
>> >> >
>> >> > On 21 April 2010 16:35, TonyTobin <ca...@hotmail.com> wrote:
>> >> >
>> >> >>
>> >> >> In the ActiveMQ doc
>> >> http://activemq.apache.org/exclusive-consumer.htmlit
>> >> >> states
>> >> >>
>> >> >> The broker will pick a single MessageConsumer to get all the
>> messages
>> >> for
>> >> >> a
>> >> >> queue to ensure ordering. If that consumer fails, the broker will
>> auto
>> >> >> failover and choose another consumer.
>> >> >>
>> >> >> Is this a consumer master slave relationship, the broker will send
>> all
>> >> >> messages to the designated consumer, and the consumer passes the
>> >> messages
>> >> >> onto the other consumers. Or does the broker send the messages to
>> all
>> >> the
>> >> >> consumers, but only one consumer acknowledges them, so if that
>> >> consumer
>> >> >> dies the other consumer can then pick up where the first consumer
>> >> died.
>> >> >>
>> >> >> If there is a link that covers Queue consumer clusters in more
>> depth,
>> >> I
>> >> >> would be grateful if you could post it.
>> >> >>
>> >> >> Thanks Tony
>> >> >>
>> >> >> Thanks Tony
>> >> >>
>> >> >>
>> >> >> --
>> >> >> View this message in context:
>> >> >>
>> >>
>> http://old.nabble.com/Understanding-the-Exclusive-Consumer-tp28288049p28288049.html
>> >> >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >> > --
>> >> > http://blog.garytully.com
>> >> >
>> >> > Open Source Integration
>> >> > http://fusesource.com
>> >> >
>> >> >
>> >>
>> >> --
>> >> View this message in context:
>> >>
>> http://old.nabble.com/Understanding-the-Exclusive-Consumer-tp28288049p28325468.html
>> >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>> >>
>> >>
>> >
>> >
>> > --
>> > http://blog.garytully.com
>> >
>> > Open Source Integration
>> > http://fusesource.com
>> >
>> >
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Understanding-the-Exclusive-Consumer-tp28288049p28326125.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
>
>
> --
> http://blog.garytully.com
>
> Open Source Integration
> http://fusesource.com
>
>
--
View this message in context: http://old.nabble.com/Understanding-the-Exclusive-Consumer-tp28288049p28326754.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Understanding the Exclusive Consumer
Posted by Gary Tully <ga...@gmail.com>.
There are no real options there, it is handled by the broker and it
simply replays any unacked messages to the next consumer.
For the consumer app, there is the possibility that the second consumer can
get a duplicate if there is an outstanding ack at the time of failover, so
the consumers needs to be able to deal with this. Even with transactions,
with an outstanding commit reply, the consumer cannot know if the ack was
delivered so the message may or may not get redelivered. XA 2PC is the only
way to avoid the duplicate problem if the consumers need
to guarantee persistence. Taking an optimistic approach and dealing with
duplicates when they occur by remembering the message id in the store data
is the other option.
Not sure of other links that deal with this, but the info above may give you
key words that can help in your research.
On 22 April 2010 10:13, TonyTobin <ca...@hotmail.com> wrote:
>
> Thanks Gary,
>
> That makes sence.
>
>
> Are there any decent links on the different strategies for message
> persistence within consumers.
>
> A consumer is building up messages and then falls over, how the other
> consumer takes over. Managing failover within the consumer cluster.
>
> Thank you for your help
>
> Tony
>
>
> Gary Tully wrote:
> >
> > yes, but the consumers all have to be present, they already content for
> > the
> > lock.
> >
> > imagine an app that has a hot standby, both the primary and hot standby
> > have
> > started consumers. The broker delivers messages to the first consumer,
> the
> > primary (because it got there first) and when/if it fails, it will
> deliver
> > messages to the second consumer (the hot standby). So the broker chooses
> > but
> > only from among the current active consumers.
> >
> > On 22 April 2010 09:09, TonyTobin <ca...@hotmail.com> wrote:
> >
> >>
> >> Thank you for your reply Gary,
> >>
> >> In the documentation for exclusive Consumer, it states
> >>
> >> "The broker will pick a single MessageConsumer to get all the messages
> >> for
> >> a
> >> queue to ensure ordering. If that consumer fails, the broker will auto
> >> failover and choose another consumer"
> >>
> >> Doesnt this mean that the broker selects the next consumer, rather than
> >> the
> >> consumer going for the lock.
> >>
> >> Thanks again for your reply.
> >>
> >>
> >>
> >> Gary Tully wrote:
> >> >
> >> > exclusive consumer is a simple solution to the case where there are
> >> many
> >> > consumers but only one should be active at any time. Rather than some
> >> > application logic keeping track of which one should be active, they
> all
> >> > have
> >> > a go and the broker ensures exclusive access.
> >> >
> >> > So each consumer connects but only one of them gets the exclusive lock
> >> and
> >> > gets the messages. When it dies, the next consumer in line gets the
> >> > messages. If effectively allows consumers to stack waiting for the
> >> current
> >> > consumer to die.
> >> >
> >> > On 21 April 2010 16:35, TonyTobin <ca...@hotmail.com> wrote:
> >> >
> >> >>
> >> >> In the ActiveMQ doc
> >> http://activemq.apache.org/exclusive-consumer.htmlit
> >> >> states
> >> >>
> >> >> The broker will pick a single MessageConsumer to get all the messages
> >> for
> >> >> a
> >> >> queue to ensure ordering. If that consumer fails, the broker will
> auto
> >> >> failover and choose another consumer.
> >> >>
> >> >> Is this a consumer master slave relationship, the broker will send
> all
> >> >> messages to the designated consumer, and the consumer passes the
> >> messages
> >> >> onto the other consumers. Or does the broker send the messages to all
> >> the
> >> >> consumers, but only one consumer acknowledges them, so if that
> >> consumer
> >> >> dies the other consumer can then pick up where the first consumer
> >> died.
> >> >>
> >> >> If there is a link that covers Queue consumer clusters in more depth,
> >> I
> >> >> would be grateful if you could post it.
> >> >>
> >> >> Thanks Tony
> >> >>
> >> >> Thanks Tony
> >> >>
> >> >>
> >> >> --
> >> >> View this message in context:
> >> >>
> >>
> http://old.nabble.com/Understanding-the-Exclusive-Consumer-tp28288049p28288049.html
> >> >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > http://blog.garytully.com
> >> >
> >> > Open Source Integration
> >> > http://fusesource.com
> >> >
> >> >
> >>
> >> --
> >> View this message in context:
> >>
> http://old.nabble.com/Understanding-the-Exclusive-Consumer-tp28288049p28325468.html
> >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >>
> >>
> >
> >
> > --
> > http://blog.garytully.com
> >
> > Open Source Integration
> > http://fusesource.com
> >
> >
>
> --
> View this message in context:
> http://old.nabble.com/Understanding-the-Exclusive-Consumer-tp28288049p28326125.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
--
http://blog.garytully.com
Open Source Integration
http://fusesource.com
Re: Understanding the Exclusive Consumer
Posted by TonyTobin <ca...@hotmail.com>.
Thanks Gary,
That makes sence.
Are there any decent links on the different strategies for message
persistence within consumers.
A consumer is building up messages and then falls over, how the other
consumer takes over. Managing failover within the consumer cluster.
Thank you for your help
Tony
Gary Tully wrote:
>
> yes, but the consumers all have to be present, they already content for
> the
> lock.
>
> imagine an app that has a hot standby, both the primary and hot standby
> have
> started consumers. The broker delivers messages to the first consumer, the
> primary (because it got there first) and when/if it fails, it will deliver
> messages to the second consumer (the hot standby). So the broker chooses
> but
> only from among the current active consumers.
>
> On 22 April 2010 09:09, TonyTobin <ca...@hotmail.com> wrote:
>
>>
>> Thank you for your reply Gary,
>>
>> In the documentation for exclusive Consumer, it states
>>
>> "The broker will pick a single MessageConsumer to get all the messages
>> for
>> a
>> queue to ensure ordering. If that consumer fails, the broker will auto
>> failover and choose another consumer"
>>
>> Doesnt this mean that the broker selects the next consumer, rather than
>> the
>> consumer going for the lock.
>>
>> Thanks again for your reply.
>>
>>
>>
>> Gary Tully wrote:
>> >
>> > exclusive consumer is a simple solution to the case where there are
>> many
>> > consumers but only one should be active at any time. Rather than some
>> > application logic keeping track of which one should be active, they all
>> > have
>> > a go and the broker ensures exclusive access.
>> >
>> > So each consumer connects but only one of them gets the exclusive lock
>> and
>> > gets the messages. When it dies, the next consumer in line gets the
>> > messages. If effectively allows consumers to stack waiting for the
>> current
>> > consumer to die.
>> >
>> > On 21 April 2010 16:35, TonyTobin <ca...@hotmail.com> wrote:
>> >
>> >>
>> >> In the ActiveMQ doc
>> http://activemq.apache.org/exclusive-consumer.htmlit
>> >> states
>> >>
>> >> The broker will pick a single MessageConsumer to get all the messages
>> for
>> >> a
>> >> queue to ensure ordering. If that consumer fails, the broker will auto
>> >> failover and choose another consumer.
>> >>
>> >> Is this a consumer master slave relationship, the broker will send all
>> >> messages to the designated consumer, and the consumer passes the
>> messages
>> >> onto the other consumers. Or does the broker send the messages to all
>> the
>> >> consumers, but only one consumer acknowledges them, so if that
>> consumer
>> >> dies the other consumer can then pick up where the first consumer
>> died.
>> >>
>> >> If there is a link that covers Queue consumer clusters in more depth,
>> I
>> >> would be grateful if you could post it.
>> >>
>> >> Thanks Tony
>> >>
>> >> Thanks Tony
>> >>
>> >>
>> >> --
>> >> View this message in context:
>> >>
>> http://old.nabble.com/Understanding-the-Exclusive-Consumer-tp28288049p28288049.html
>> >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>> >>
>> >>
>> >
>> >
>> > --
>> > http://blog.garytully.com
>> >
>> > Open Source Integration
>> > http://fusesource.com
>> >
>> >
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Understanding-the-Exclusive-Consumer-tp28288049p28325468.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
>
>
> --
> http://blog.garytully.com
>
> Open Source Integration
> http://fusesource.com
>
>
--
View this message in context: http://old.nabble.com/Understanding-the-Exclusive-Consumer-tp28288049p28326125.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Understanding the Exclusive Consumer
Posted by Gary Tully <ga...@gmail.com>.
yes, but the consumers all have to be present, they already content for the
lock.
imagine an app that has a hot standby, both the primary and hot standby have
started consumers. The broker delivers messages to the first consumer, the
primary (because it got there first) and when/if it fails, it will deliver
messages to the second consumer (the hot standby). So the broker chooses but
only from among the current active consumers.
On 22 April 2010 09:09, TonyTobin <ca...@hotmail.com> wrote:
>
> Thank you for your reply Gary,
>
> In the documentation for exclusive Consumer, it states
>
> "The broker will pick a single MessageConsumer to get all the messages for
> a
> queue to ensure ordering. If that consumer fails, the broker will auto
> failover and choose another consumer"
>
> Doesnt this mean that the broker selects the next consumer, rather than the
> consumer going for the lock.
>
> Thanks again for your reply.
>
>
>
> Gary Tully wrote:
> >
> > exclusive consumer is a simple solution to the case where there are many
> > consumers but only one should be active at any time. Rather than some
> > application logic keeping track of which one should be active, they all
> > have
> > a go and the broker ensures exclusive access.
> >
> > So each consumer connects but only one of them gets the exclusive lock
> and
> > gets the messages. When it dies, the next consumer in line gets the
> > messages. If effectively allows consumers to stack waiting for the
> current
> > consumer to die.
> >
> > On 21 April 2010 16:35, TonyTobin <ca...@hotmail.com> wrote:
> >
> >>
> >> In the ActiveMQ doc http://activemq.apache.org/exclusive-consumer.htmlit
> >> states
> >>
> >> The broker will pick a single MessageConsumer to get all the messages
> for
> >> a
> >> queue to ensure ordering. If that consumer fails, the broker will auto
> >> failover and choose another consumer.
> >>
> >> Is this a consumer master slave relationship, the broker will send all
> >> messages to the designated consumer, and the consumer passes the
> messages
> >> onto the other consumers. Or does the broker send the messages to all
> the
> >> consumers, but only one consumer acknowledges them, so if that consumer
> >> dies the other consumer can then pick up where the first consumer died.
> >>
> >> If there is a link that covers Queue consumer clusters in more depth, I
> >> would be grateful if you could post it.
> >>
> >> Thanks Tony
> >>
> >> Thanks Tony
> >>
> >>
> >> --
> >> View this message in context:
> >>
> http://old.nabble.com/Understanding-the-Exclusive-Consumer-tp28288049p28288049.html
> >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >>
> >>
> >
> >
> > --
> > http://blog.garytully.com
> >
> > Open Source Integration
> > http://fusesource.com
> >
> >
>
> --
> View this message in context:
> http://old.nabble.com/Understanding-the-Exclusive-Consumer-tp28288049p28325468.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
--
http://blog.garytully.com
Open Source Integration
http://fusesource.com
Re: Understanding the Exclusive Consumer
Posted by TonyTobin <ca...@hotmail.com>.
Thank you for your reply Gary,
In the documentation for exclusive Consumer, it states
"The broker will pick a single MessageConsumer to get all the messages for a
queue to ensure ordering. If that consumer fails, the broker will auto
failover and choose another consumer"
Doesnt this mean that the broker selects the next consumer, rather than the
consumer going for the lock.
Thanks again for your reply.
Gary Tully wrote:
>
> exclusive consumer is a simple solution to the case where there are many
> consumers but only one should be active at any time. Rather than some
> application logic keeping track of which one should be active, they all
> have
> a go and the broker ensures exclusive access.
>
> So each consumer connects but only one of them gets the exclusive lock and
> gets the messages. When it dies, the next consumer in line gets the
> messages. If effectively allows consumers to stack waiting for the current
> consumer to die.
>
> On 21 April 2010 16:35, TonyTobin <ca...@hotmail.com> wrote:
>
>>
>> In the ActiveMQ doc http://activemq.apache.org/exclusive-consumer.html it
>> states
>>
>> The broker will pick a single MessageConsumer to get all the messages for
>> a
>> queue to ensure ordering. If that consumer fails, the broker will auto
>> failover and choose another consumer.
>>
>> Is this a consumer master slave relationship, the broker will send all
>> messages to the designated consumer, and the consumer passes the messages
>> onto the other consumers. Or does the broker send the messages to all the
>> consumers, but only one consumer acknowledges them, so if that consumer
>> dies the other consumer can then pick up where the first consumer died.
>>
>> If there is a link that covers Queue consumer clusters in more depth, I
>> would be grateful if you could post it.
>>
>> Thanks Tony
>>
>> Thanks Tony
>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Understanding-the-Exclusive-Consumer-tp28288049p28288049.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
>
>
> --
> http://blog.garytully.com
>
> Open Source Integration
> http://fusesource.com
>
>
--
View this message in context: http://old.nabble.com/Understanding-the-Exclusive-Consumer-tp28288049p28325468.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Understanding the Exclusive Consumer
Posted by Gary Tully <ga...@gmail.com>.
exclusive consumer is a simple solution to the case where there are many
consumers but only one should be active at any time. Rather than some
application logic keeping track of which one should be active, they all have
a go and the broker ensures exclusive access.
So each consumer connects but only one of them gets the exclusive lock and
gets the messages. When it dies, the next consumer in line gets the
messages. If effectively allows consumers to stack waiting for the current
consumer to die.
On 21 April 2010 16:35, TonyTobin <ca...@hotmail.com> wrote:
>
> In the ActiveMQ doc http://activemq.apache.org/exclusive-consumer.html it
> states
>
> The broker will pick a single MessageConsumer to get all the messages for a
> queue to ensure ordering. If that consumer fails, the broker will auto
> failover and choose another consumer.
>
> Is this a consumer master slave relationship, the broker will send all
> messages to the designated consumer, and the consumer passes the messages
> onto the other consumers. Or does the broker send the messages to all the
> consumers, but only one consumer acknowledges them, so if that consumer
> dies the other consumer can then pick up where the first consumer died.
>
> If there is a link that covers Queue consumer clusters in more depth, I
> would be grateful if you could post it.
>
> Thanks Tony
>
> Thanks Tony
>
>
> --
> View this message in context:
> http://old.nabble.com/Understanding-the-Exclusive-Consumer-tp28288049p28288049.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
--
http://blog.garytully.com
Open Source Integration
http://fusesource.com