You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by sspind <ss...@yandex.com> on 2015/11/10 22:06:00 UTC
Message Group rebalancing
Hi there,
a question related to message groups: is it possible to rebalance message
groups when a new consumer registers with the broker?
In my setup I have two consumers that consume from the same queue where all
messages have a message group ID. If one consumer loses connection all
messages go to the other. If the consumer reconnects messages will still go
to the other one. At least if the broker saw the respective group ID
already. I know that new messages will again be given to either of the
consumers. However, in my scenario there is only a relatively small set of
"long lived" group IDs. So it can happen that all messages will be routed to
one consumer while the other one is idle.
Looking forward to your replies!
sspind
--
View this message in context: http://activemq.2283324.n4.nabble.com/Message-Group-rebalancing-tp4703823.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.
Re: Message Group rebalancing
Posted by Tim Bain <tb...@alumni.duke.edu>.
https://issues.apache.org/jira/browse/AMQ-2106 already covers this request,
and in the comments there, Gary's opinion was that the only safe way to do
this was to have the producer close the group, as Pankaj suggests.
Unfortunately, the current implementation appears to begin dispatching
messages from before the close request to the new consumer instead of
letting the old consumer finish them:
https://issues.apache.org/jira/browse/AMQ-3027. If you want that issue
fixed, add your vote to it in JIRA.
Tim
On Mon, Nov 16, 2015 at 10:38 AM, Takawale, Pankaj <
pankaj.takawale@dowjones.com> wrote:
> You could achieve it from your application logic. You could have your
> producer or consumers break the JMSXGroupID stickiness by sending
> JMSXGroupSeq=-1
> message after producing or receiving N messages from the same groupID.
> After stickiness break up, new consumers would get a chance to acquire the
> groups.
>
>
>
> On Wed, Nov 11, 2015 at 9:49 AM, Rob Davies <ra...@gmail.com> wrote:
>
> > Nice feature - submit a JIRA!
> >
> > > On 11 Nov 2015, at 14:28, Tim Bain <tb...@alumni.duke.edu> wrote:
> > >
> > > I've never heard of such a feature (but maybe someone else here has),
> > but I
> > > think it would be a great addition if it doesn't exist. So if you
> don't
> > > get a response within a couple days, I encourage you to submit an
> > > enhancement request in JIRA.
> > >
> > > Tim
> > > On Nov 10, 2015 2:20 PM, "sspind" <ss...@yandex.com> wrote:
> > >
> > >> Hi there,
> > >>
> > >> a question related to message groups: is it possible to rebalance
> > message
> > >> groups when a new consumer registers with the broker?
> > >> In my setup I have two consumers that consume from the same queue
> where
> > all
> > >> messages have a message group ID. If one consumer loses connection all
> > >> messages go to the other. If the consumer reconnects messages will
> > still go
> > >> to the other one. At least if the broker saw the respective group ID
> > >> already. I know that new messages will again be given to either of the
> > >> consumers. However, in my scenario there is only a relatively small
> set
> > of
> > >> "long lived" group IDs. So it can happen that all messages will be
> > routed
> > >> to
> > >> one consumer while the other one is idle.
> > >>
> > >> Looking forward to your replies!
> > >>
> > >> sspind
> > >>
> > >>
> > >>
> > >> --
> > >> View this message in context:
> > >>
> >
> http://activemq.2283324.n4.nabble.com/Message-Group-rebalancing-tp4703823.html
> > >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> > >>
> >
> >
>
Re: Message Group rebalancing
Posted by "Takawale, Pankaj" <pa...@dowjones.com>.
You could achieve it from your application logic. You could have your
producer or consumers break the JMSXGroupID stickiness by sending
JMSXGroupSeq=-1
message after producing or receiving N messages from the same groupID.
After stickiness break up, new consumers would get a chance to acquire the
groups.
On Wed, Nov 11, 2015 at 9:49 AM, Rob Davies <ra...@gmail.com> wrote:
> Nice feature - submit a JIRA!
>
> > On 11 Nov 2015, at 14:28, Tim Bain <tb...@alumni.duke.edu> wrote:
> >
> > I've never heard of such a feature (but maybe someone else here has),
> but I
> > think it would be a great addition if it doesn't exist. So if you don't
> > get a response within a couple days, I encourage you to submit an
> > enhancement request in JIRA.
> >
> > Tim
> > On Nov 10, 2015 2:20 PM, "sspind" <ss...@yandex.com> wrote:
> >
> >> Hi there,
> >>
> >> a question related to message groups: is it possible to rebalance
> message
> >> groups when a new consumer registers with the broker?
> >> In my setup I have two consumers that consume from the same queue where
> all
> >> messages have a message group ID. If one consumer loses connection all
> >> messages go to the other. If the consumer reconnects messages will
> still go
> >> to the other one. At least if the broker saw the respective group ID
> >> already. I know that new messages will again be given to either of the
> >> consumers. However, in my scenario there is only a relatively small set
> of
> >> "long lived" group IDs. So it can happen that all messages will be
> routed
> >> to
> >> one consumer while the other one is idle.
> >>
> >> Looking forward to your replies!
> >>
> >> sspind
> >>
> >>
> >>
> >> --
> >> View this message in context:
> >>
> http://activemq.2283324.n4.nabble.com/Message-Group-rebalancing-tp4703823.html
> >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >>
>
>
Re: Message Group rebalancing
Posted by Rob Davies <ra...@gmail.com>.
Nice feature - submit a JIRA!
> On 11 Nov 2015, at 14:28, Tim Bain <tb...@alumni.duke.edu> wrote:
>
> I've never heard of such a feature (but maybe someone else here has), but I
> think it would be a great addition if it doesn't exist. So if you don't
> get a response within a couple days, I encourage you to submit an
> enhancement request in JIRA.
>
> Tim
> On Nov 10, 2015 2:20 PM, "sspind" <ss...@yandex.com> wrote:
>
>> Hi there,
>>
>> a question related to message groups: is it possible to rebalance message
>> groups when a new consumer registers with the broker?
>> In my setup I have two consumers that consume from the same queue where all
>> messages have a message group ID. If one consumer loses connection all
>> messages go to the other. If the consumer reconnects messages will still go
>> to the other one. At least if the broker saw the respective group ID
>> already. I know that new messages will again be given to either of the
>> consumers. However, in my scenario there is only a relatively small set of
>> "long lived" group IDs. So it can happen that all messages will be routed
>> to
>> one consumer while the other one is idle.
>>
>> Looking forward to your replies!
>>
>> sspind
>>
>>
>>
>> --
>> View this message in context:
>> http://activemq.2283324.n4.nabble.com/Message-Group-rebalancing-tp4703823.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
Re: Message Group rebalancing
Posted by Tim Bain <tb...@alumni.duke.edu>.
I've never heard of such a feature (but maybe someone else here has), but I
think it would be a great addition if it doesn't exist. So if you don't
get a response within a couple days, I encourage you to submit an
enhancement request in JIRA.
Tim
On Nov 10, 2015 2:20 PM, "sspind" <ss...@yandex.com> wrote:
> Hi there,
>
> a question related to message groups: is it possible to rebalance message
> groups when a new consumer registers with the broker?
> In my setup I have two consumers that consume from the same queue where all
> messages have a message group ID. If one consumer loses connection all
> messages go to the other. If the consumer reconnects messages will still go
> to the other one. At least if the broker saw the respective group ID
> already. I know that new messages will again be given to either of the
> consumers. However, in my scenario there is only a relatively small set of
> "long lived" group IDs. So it can happen that all messages will be routed
> to
> one consumer while the other one is idle.
>
> Looking forward to your replies!
>
> sspind
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Message-Group-rebalancing-tp4703823.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>