MessageGroup behavior

[vramanx <vr...@hotmail.com>]

Is it possible to have a fixed set of consumers in a MessageGroup listening
to all the messages, but the processing will be sequential for a given group
id?

for example,

we have an order processing system and the group id is the order id.
messages for an order should be processed sequentially. as long as on
consumer is processing a message for an order id, further messages for that
order id are not dispatched.





--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Re: MessageGroup behavior

[Archibald <ar...@gmx.net>]

Agree.
Sorry for that.

Br, A.



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Re: MessageGroup behavior

[Justin Bertram <jb...@apache.org>]

I believe the original author on this thread was using 5.x, but you're
using Artemis so it might be best to start a new thread.  In any event,
it's generally not good manners to hijack someone else's thread with your
own issue.  It usually confuses the original issue and delays resolution
for the thread's author.


Justin

On Thu, Mar 1, 2018 at 8:33 AM, Archibald <ar...@gmx.net> wrote:

> Even if, would not make any difference in my case.
> The topic consumer is the broker itself by diverting messages into queues.
>
> But each queue has multiple consumers which process messages
> concurrently...
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>

Re: MessageGroup behavior

[Archibald <ar...@gmx.net>]

Even if, would not make any difference in my case.
The topic consumer is the broker itself by diverting messages into queues.

But each queue has multiple consumers which process messages concurrently...



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Re: MessageGroup behavior

[Tim Bain <tb...@alumni.duke.edu>]

Don't topic consumers process messages one by one by default?

Tim

On Mar 1, 2018 5:56 AM, "Archibald" <ar...@gmx.net> wrote:

> I'm struggling with a similiar problem:
>
> I have one producer sending event messages.
> I have a couple of different consumers which shall react on those events.
> Producer is not aware of any consumers and as such publishes events to a
> topic.
> The broker is configured with diverts which split and forward events to
> dedicated consumer queues.
>
> Those events are related to an asset and I want the consumers to process
> events for the same asset one by one.
>
> First try:
> Producers uses JMSXGroupID on the message being send to the topic. But the
> message is only being delivered to one of the consumer queues (and one
> particular consumer).
>
> Second try:
> The producer sends message without JMSXGroupID. I've configured a
> transformer on each divert which adds the JMSXGroupID property. But it
> seems
> that at this stage the property is being ignored by Artemis.
>
> What can I do?
>
> Thanks,
> Archibald
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>

Re: MessageGroup behavior

[Archibald <ar...@gmx.net>]

I'm struggling with a similiar problem:

I have one producer sending event messages. 
I have a couple of different consumers which shall react on those events.
Producer is not aware of any consumers and as such publishes events to a
topic.
The broker is configured with diverts which split and forward events to
dedicated consumer queues.

Those events are related to an asset and I want the consumers to process
events for the same asset one by one.

First try:
Producers uses JMSXGroupID on the message being send to the topic. But the
message is only being delivered to one of the consumer queues (and one
particular consumer).

Second try:
The producer sends message without JMSXGroupID. I've configured a
transformer on each divert which adds the JMSXGroupID property. But it seems
that at this stage the property is being ignored by Artemis.

What can I do?

Thanks, 
Archibald  



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Re: MessageGroup behavior

[Tim Bain <tb...@alumni.duke.edu>]

I didn't follow the bit about grouping your workers. You have N ActiveMQ
consumers, and the message group will be assigned to one (and only one) of
them. Where does grouping workers play into it?

Tim

On Mar 9, 2018 3:18 PM, "vramanx" <vr...@hotmail.com> wrote:

Thanks for the clarification Tim! From the documentation, I thought that the
consumers have to be tagged with static GroupID which is then later set on
the producer side.

So on the consumer side we just need to group workers as a processing unit
and the system will pick an available group to assign a message for a new
group id.

That meets our scenario. Thanks again for the clarification.



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Re: MessageGroup behavior

[vramanx <vr...@hotmail.com>]

Thanks for the clarification Tim! From the documentation, I thought that the
consumers have to be tagged with static GroupID which is then later set on
the producer side.

So on the consumer side we just need to group workers as a processing unit
and the system will pick an available group to assign a message for a new
group id.

That meets our scenario. Thanks again for the clarification.



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Re: MessageGroup behavior

[Tim Bain <tb...@alumni.duke.edu>]

Re-reading your original question, I realized that I assumed that you were
aware of the JMS-standard feature called message groups, which ensure that
all messages with a given group ID are consumed (in order) by a single
consumer. That feature will meet all of your requirements except for the
ability for multiple consumers to participate in the consumption, so if you
weren't aware of it, please consider whether it will meet your needs. See
http://activemq.apache.org/message-groups.html for a description of
ActiveMQ's implementation of this JMS-standard feature.

If you already knew about that feature and really wanted a multi-consumer
variant, read on.

For message groups, I didn't mean to imply that the assignment of consumers
to group IDs was done at configuration time rather than runtime. The broker
assigns whichever consumer is available to an unassigned group,
automatically and without any advance configuration.

What we don't do is allow multiple consumers to be jointly assigned to the
group but ensure that no more than one is consuming a message from the
group at the same time, which is what I understand you to be proposing.
There isn't a good way for the broker to dispatch messages in the same
group to two consumers but control when they process the message, so the
only way it could work is to not dispatch the next message until the
previous one has been acknowledged. If consumers have large prefetch
buffers (for performance reasons) this means either a long time between
messages in the group being consumed (i.e. poor performance) or all the
messages in the group going to the same consumer (which is what you already
get from the existing message group feature). In addition, you would need
to define somehow which consumers are in the group and which are not. All
of this is possible, and you're welcome to submit JIRA enhancement requests
against either/both of ActiveMQ 5.x and Artemis to propose it, but I
personally don't see how it's a win over the existing single-consumer
message group feature. Unless there's something I'm not considering?

Tim

On Feb 26, 2018 1:49 PM, "vramanx" <vr...@hotmail.com> wrote:

> The group id is dynamic. So we cant really have a set of consumers with a
> pre-defined set of group ids.
>
> One way to handle this is to spin up a singleton actor consumer for a group
> id if none exists.
>
> Will be great if ActiveMQ supports this concept of "dynamic group id" out
> of
> the box so there is a fixed set of consumers for the MessageGroup. As long
> as there is one consumer is processing a message for a group id, then new
> messages will not be dispatched.
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>

Re: MessageGroup behavior

[vramanx <vr...@hotmail.com>]

The group id is dynamic. So we cant really have a set of consumers with a
pre-defined set of group ids.

One way to handle this is to spin up a singleton actor consumer for a group
id if none exists.

Will be great if ActiveMQ supports this concept of "dynamic group id" out of
the box so there is a fixed set of consumers for the MessageGroup. As long
as there is one consumer is processing a message for a group id, then new
messages will not be dispatched.



--
Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html

Re: MessageGroup behavior

[Tim Bain <tb...@alumni.duke.edu>]

Message groups are tied to one and only one consumer (unless that consumer
disconnects, in which case a new consumer can be chosen). I'm not aware of
any feature that would allow any consumer to take the next message in the
group but only if no other message in the group is being processed, which
is what I think you're asking for.

Tim

On Sat, Feb 24, 2018 at 11:29 PM, vramanx <vr...@hotmail.com> wrote:

> Is it possible to have a fixed set of consumers in a MessageGroup listening
> to all the messages, but the processing will be sequential for a given
> group
> id?
>
> for example,
>
> we have an order processing system and the group id is the order id.
> messages for an order should be processed sequentially. as long as on
> consumer is processing a message for an order id, further messages for that
> order id are not dispatched.
>
>
>
>
>
> --
> Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-
> f2341805.html
>