You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by kalki70 <lr...@pobox.com> on 2009/02/10 18:06:23 UTC
Automatic recovery when using Message Groups
Hi,
I have been looking at this feature, that so far looks great for our
applications.
If I have several consumers, messages with specific JMSXGroupId are routed
to the same consumer.
If a consumer fails, its message groups are now reassigned to a different
consumer, so the service keeps working.
But... if the failed consumer comes back, it will never received any
message, because the message groups are now assigned to different consumers.
Is there a way to make this consumer to recover its previous message groups?
Thanks,
Luis
--
View this message in context: http://www.nabble.com/Automatic-recovery-when-using-Message-Groups-tp21938109p21938109.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Automatic recovery when using Message Groups
Posted by Dejan Bosanac <de...@nighttale.net>.
I've filed a Jira enhacement request for this
https://issues.apache.org/activemq/browse/AMQ-2106
Cheers
--
Dejan Bosanac
Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net
On Thu, Feb 12, 2009 at 6:12 PM, kalki70 <lr...@pobox.com> wrote:
>
> Hi, Dejan,
>
> I agree consumers don't know which groups they will be assigned. As you
> said, if a consumer goes down, other will take those groups, but the
> problem
> comes when the consumer that went down comes back again. It will never
> receive any more messages. So what I proposed was something like the
> consumer saying :
> "Hey, I crashed, but I came back. Give me back the message groups I had
> before. By the way, last time I subscribed my consumer id was xxxxx".
>
> The feature you mention, that when a new consumer subscribes, the broker
> tries to distribute evenly the messages groups, is just like having now my
> Christmas gift :jumping:. It would be really wonderful :clap:, because it
> solves the problem of adding more consumers dynamically.
>
> The policy using consumersBeforeDispatchStarts and timeBeforeDispatchStarts
> seemed to me just a temporary solution to the problem, but this one looks
> much more powerful. Not only consumers that go down can come back and
> process messages, also more consumers can be added at any time.
>
> Best regards,
>
> Luis
>
> Dejan Bosanac wrote:
> >
> > Hi Luis,
> >
> > I'm not sure this is the correct use case. Consumers don't explicitly
> know
> > what groups they will be assigned to when they first subscribe, so there
> > is
> > usually no difference what consumer is consuming messages from what
> group.
> > When your consumer goes down, the other one will take its group and
> > everything will work fine.
> >
> > Maybe we should add a feature that would instruct the broker to try to
> > make
> > an even distribution of groups among consumers. So when the new consumer
> > is
> > subscribed, the broker will see if it can reassign certain groups to it
> > depending on the number of groups assigned to existing consumers. I have
> > an
> > impression, that this would make for you as well?
> >
> > Cheers
> > --
> > Dejan Bosanac
> >
> > Open Source Integration - http://fusesource.com/
> > ActiveMQ in Action - http://www.manning.com/snyder/
> > Blog - http://www.nighttale.net
> >
>
> --
> View this message in context:
> http://www.nabble.com/Automatic-recovery-when-using-Message-Groups-tp21938109p21980479.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
Re: Automatic recovery when using Message Groups
Posted by kalki70 <lr...@pobox.com>.
Hi, Dejan,
I agree consumers don't know which groups they will be assigned. As you
said, if a consumer goes down, other will take those groups, but the problem
comes when the consumer that went down comes back again. It will never
receive any more messages. So what I proposed was something like the
consumer saying :
"Hey, I crashed, but I came back. Give me back the message groups I had
before. By the way, last time I subscribed my consumer id was xxxxx".
The feature you mention, that when a new consumer subscribes, the broker
tries to distribute evenly the messages groups, is just like having now my
Christmas gift :jumping:. It would be really wonderful :clap:, because it
solves the problem of adding more consumers dynamically.
The policy using consumersBeforeDispatchStarts and timeBeforeDispatchStarts
seemed to me just a temporary solution to the problem, but this one looks
much more powerful. Not only consumers that go down can come back and
process messages, also more consumers can be added at any time.
Best regards,
Luis
Dejan Bosanac wrote:
>
> Hi Luis,
>
> I'm not sure this is the correct use case. Consumers don't explicitly know
> what groups they will be assigned to when they first subscribe, so there
> is
> usually no difference what consumer is consuming messages from what group.
> When your consumer goes down, the other one will take its group and
> everything will work fine.
>
> Maybe we should add a feature that would instruct the broker to try to
> make
> an even distribution of groups among consumers. So when the new consumer
> is
> subscribed, the broker will see if it can reassign certain groups to it
> depending on the number of groups assigned to existing consumers. I have
> an
> impression, that this would make for you as well?
>
> Cheers
> --
> Dejan Bosanac
>
> Open Source Integration - http://fusesource.com/
> ActiveMQ in Action - http://www.manning.com/snyder/
> Blog - http://www.nighttale.net
>
--
View this message in context: http://www.nabble.com/Automatic-recovery-when-using-Message-Groups-tp21938109p21980479.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Automatic recovery when using Message Groups
Posted by Dejan Bosanac <de...@nighttale.net>.
Hi Luis,
I'm not sure this is the correct use case. Consumers don't explicitly know
what groups they will be assigned to when they first subscribe, so there is
usually no difference what consumer is consuming messages from what group.
When your consumer goes down, the other one will take its group and
everything will work fine.
Maybe we should add a feature that would instruct the broker to try to make
an even distribution of groups among consumers. So when the new consumer is
subscribed, the broker will see if it can reassign certain groups to it
depending on the number of groups assigned to existing consumers. I have an
impression, that this would make for you as well?
Cheers
--
Dejan Bosanac
Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net
On Thu, Feb 12, 2009 at 4:03 PM, kalki70 <lr...@pobox.com> wrote:
>
> Hi, Dejan,
>
> My opinion is that the functionality of Message Groups is partially broken
> if there is no automatic recovery. What should the user do? Restart
> ActiveMQ
> ? :-(
> I don't think it is so hard to implement. Upon subscription the consumer
> could be informed its consumer id, which can be an optional parameter for
> subscriptions. It would be responsibility of the consumer, when it
> subscribes again, to provide its previous consumer id, if it wants to
> assume
> the identity of the previous consumer.
> If it doesn't provide it, it will be assigned a new consumer id. Assigned
> consumer ids that are not claimed back could be released based on some
> criteria, for instance, they could have an expiration time.
>
> What do you think?
>
> Best regards,
>
> Luis
>
> Dejan Bosanac wrote:
> >
> > Hi Luis,
> >
> > unfortunately there is no way for the consumer to recover its previous
> > message groups. I'm not sure how this could be implemented, since the
> > consumer will have a new consumer id upon recovery, so it would be
> > impossible to relate it to the previous subscription.
> >
> > Cheers
> > --
> > Dejan Bosanac
> >
> > Open Source Integration - http://fusesource.com/
> > ActiveMQ in Action - http://www.manning.com/snyder/
> > Blog - http://www.nighttale.net
> >
>
> --
> View this message in context:
> http://www.nabble.com/Automatic-recovery-when-using-Message-Groups-tp21938109p21977778.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
Re: Automatic recovery when using Message Groups
Posted by kalki70 <lr...@pobox.com>.
Hi, Dejan,
My opinion is that the functionality of Message Groups is partially broken
if there is no automatic recovery. What should the user do? Restart ActiveMQ
? :-(
I don't think it is so hard to implement. Upon subscription the consumer
could be informed its consumer id, which can be an optional parameter for
subscriptions. It would be responsibility of the consumer, when it
subscribes again, to provide its previous consumer id, if it wants to assume
the identity of the previous consumer.
If it doesn't provide it, it will be assigned a new consumer id. Assigned
consumer ids that are not claimed back could be released based on some
criteria, for instance, they could have an expiration time.
What do you think?
Best regards,
Luis
Dejan Bosanac wrote:
>
> Hi Luis,
>
> unfortunately there is no way for the consumer to recover its previous
> message groups. I'm not sure how this could be implemented, since the
> consumer will have a new consumer id upon recovery, so it would be
> impossible to relate it to the previous subscription.
>
> Cheers
> --
> Dejan Bosanac
>
> Open Source Integration - http://fusesource.com/
> ActiveMQ in Action - http://www.manning.com/snyder/
> Blog - http://www.nighttale.net
>
--
View this message in context: http://www.nabble.com/Automatic-recovery-when-using-Message-Groups-tp21938109p21977778.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Automatic recovery when using Message Groups
Posted by Dejan Bosanac <de...@nighttale.net>.
Hi Luis,
unfortunately there is no way for the consumer to recover its previous
message groups. I'm not sure how this could be implemented, since the
consumer will have a new consumer id upon recovery, so it would be
impossible to relate it to the previous subscription.
Cheers
--
Dejan Bosanac
Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net
On Tue, Feb 10, 2009 at 6:06 PM, kalki70 <lr...@pobox.com> wrote:
>
> Hi,
>
> I have been looking at this feature, that so far looks great for our
> applications.
> If I have several consumers, messages with specific JMSXGroupId are routed
> to the same consumer.
> If a consumer fails, its message groups are now reassigned to a different
> consumer, so the service keeps working.
>
> But... if the failed consumer comes back, it will never received any
> message, because the message groups are now assigned to different
> consumers.
> Is there a way to make this consumer to recover its previous message
> groups?
>
> Thanks,
>
> Luis
>
> --
> View this message in context:
> http://www.nabble.com/Automatic-recovery-when-using-Message-Groups-tp21938109p21938109.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>