You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@activemq.apache.org by daltonch <ch...@2kgt.com> on 2012/12/20 22:11:27 UTC

Enabling Message Groups

I have a single queue, configured with 50 consumers.

When building the message I do,
message.setStringProperty("JMSXGroupID", jmsGroupID);

Where jmsGroupID is selected based on some string in the message body. There
are about 20 different values for jmsGroupID at this time. However, when
processing messages if I have several thousand with 1 jmsGroupID, lets call
it A, and a handful with another, lets call it B. Message's with jmsGroupID
of B will not process off along side messages with jmsGroupID A. Only 1
consumer is ever used, as you can see from this screen capture from
jmsadmin. 

https://lh5.googleusercontent.com/-HX0MGQZ6uCY/UNN9m-ZKFuI/AAAAAAAAH78/VpVZF3PpG1A/s640/consumers.PNG

All 50 consumers are listed but only 1 is actually processing messages. I
assume I have something configured incorrectly but I'm not sure where to
start? As I understand it when another message comes in with JMSXGroupID of
B it should be processed by an idle consumer. This does not appear to be
happening for me.

Thanks,
Chad



--
View this message in context: http://activemq.2283324.n4.nabble.com/Enabling-Message-Groups-tp4660972.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Enabling Message Groups

Posted by Girish <gi...@gmail.com>.

Thanks Garry for confirming about maxPageSize.

One more related question, there is memoryLimit attribute for queue policy
too, I am assuming this will supersede the maxPageSize attribute ? i.e. only
the number of messages < maxPageSize that in total size upto memoryLimit
will be loaded from the persistent store , am I right?



--
View this message in context: http://activemq.2283324.n4.nabble.com/Enabling-Message-Groups-tp4660972p4672434.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Enabling Message Groups

Posted by Gary Tully <ga...@gmail.com>.

that is correct. selectors and message groups are dispatched from
memory and the number of messages in memory is bound by maxPageSize
and prefetch size.

note: if you increase the prefetch size for a consumer it will also
result in more messages getting paged in.

On 7 October 2013 07:06, Girish <gi...@gmail.com> wrote:
> Okay, so looks like this is governed by maxPageSize for the queue .. the
> messages from the fetched page alone seem to be getting considered for group
> assignment and load balancing ... similar issue/limitation exists for
> message selectors
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/Enabling-Message-Groups-tp4660972p4672313.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.

-- 
http://redhat.com
http://blog.garytully.com

Re: Enabling Message Groups

Posted by Girish <gi...@gmail.com>.

Okay, so looks like this is governed by maxPageSize for the queue .. the
messages from the fetched page alone seem to be getting considered for group
assignment and load balancing ... similar issue/limitation exists for
message selectors



--
View this message in context: http://activemq.2283324.n4.nabble.com/Enabling-Message-Groups-tp4660972p4672313.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Enabling Message Groups

Posted by Girish <gi...@gmail.com>.

Hi, I am seeing the same behavior i.e. if there are few thousand messages
belonging to 1 group on the queue and then a new messages comes in for a
different group it doesnt get assigned a new consumer which is not what
message group feature claims to support ..... did you find any workaround or
solution for this problem, would be great if you could share.



--
View this message in context: http://activemq.2283324.n4.nabble.com/Enabling-Message-Groups-tp4660972p4672279.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Enabling Message Groups

Posted by daltonch <ch...@2kgt.com>.

So I kicked up the prefetch and was able to get some messages into the other
consumers so I'm fairly confident that JMSXGroupID is working as designed,
as I now have 5 active consumers, which makes sense given my latest message
traffic. So I guess my question is say I have 100k messages on the queue all
destined for the same Message group, not unheard of in my setup. Then I get
the 100,001 message and it belongs to another group ID. Should it be
evaluated immediately and sent to another consumer? If so I don't think I am
seeing that behavior. 



--
View this message in context: http://activemq.2283324.n4.nabble.com/Enabling-Message-Groups-tp4660972p4661035.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: Enabling Message Groups

Posted by Raul Kripalani <ra...@evosent.com>.

As far as I recall, consumers are selected using a hash function. If the
hashes for both values match, it could justify this behaviour.

You can change the bucketing algorithm with something like:

<policyEntry queue=">"> <messageGroupMapFactory>...
</messageGroupMapFactory> </policyEntry>

Check the AMQ Javadoc for implementations of the MessageGroupMapFactory
interface to see what options are available.

Sorry I can't give you a full reference now because I'm in the road, but
explore down that path.

But before that, quickly run a sanity test with different JMSXGroupID
values to discard any errors with the setup altogether.

Regards,
Raúl.
On 20 Dec 2012 21:11, "daltonch" <ch...@2kgt.com> wrote:

> I have a single queue, configured with 50 consumers.
>
> When building the message I do,
> message.setStringProperty("JMSXGroupID", jmsGroupID);
>
> Where jmsGroupID is selected based on some string in the message body.
> There
> are about 20 different values for jmsGroupID at this time. However, when
> processing messages if I have several thousand with 1 jmsGroupID, lets call
> it A, and a handful with another, lets call it B. Message's with jmsGroupID
> of B will not process off along side messages with jmsGroupID A. Only 1
> consumer is ever used, as you can see from this screen capture from
> jmsadmin.
>
>
> https://lh5.googleusercontent.com/-HX0MGQZ6uCY/UNN9m-ZKFuI/AAAAAAAAH78/VpVZF3PpG1A/s640/consumers.PNG
>
> All 50 consumers are listed but only 1 is actually processing messages. I
> assume I have something configured incorrectly but I'm not sure where to
> start? As I understand it when another message comes in with JMSXGroupID of
> B it should be processed by an idle consumer. This does not appear to be
> happening for me.
>
> Thanks,
> Chad
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Enabling-Message-Groups-tp4660972.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>