ActiveMQ broker automatically move messages with same JMSXGroupID to DLQ

[tedswow2000 <te...@gmail.com>]

Hi,
I am wondering activemq's broker can handle the following use case:

before broker sending message to the consumer, the broker will detect the
dlq, if there is message in the dlq with the same JMSXGroupID as the one is
going to be sent to the consumer, the broker will reroute the message to dlq
rather than the actual consumer...


thanks very much if someone can help.



--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-broker-automatically-move-messages-with-same-JMSXGroupID-to-DLQ-tp4685943.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ broker automatically move messages with same JMSXGroupID to DLQ

[artnaseef <ar...@artnaseef.com>]

That feature does not exist.  It would also be tricky to implement as
ActiveMQ stores messages for serial access, not random access.



--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-broker-automatically-move-messages-with-same-JMSXGroupID-to-DLQ-tp4685943p4685950.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ broker automatically move messages with same JMSXGroupID to DLQ

[Tim Bain <tb...@alumni.duke.edu>]

If there are multiple consumers, having each consumer reject out-of-order
messages as Art suggested is only going to work if you use message groups,
so take his last paragraph as good general advice but ignore it if you
choose to implement the rest of his suggestions.

But if you're using message groups, you can just keep a list of the
entities for which your initial message failed, and check each new message
against that blacklist.  You'll have to decide whether you need persistence
of that information across consumer restarts and message groups being
handed to another consumer (use a database or some other persistent store
instead of an in-memory list), how long an entity should be kept in the
blacklist, and whether blacklisting should happen only for a failure on the
first message about an entity or on any failure.

Good luck.
On Sep 11, 2015 10:42 PM, "artnaseef" <ar...@artnaseef.com> wrote:

> Hey Rosy - ActiveMQ is a messaging solution, primarily focused on getting
> messages from producers to consumers.  Advanced logic of that type really
> belongs in an application.  There's no straight-forward way to add that to
> ActiveMQ itself, and it would be difficult to diagnose if such
> functionality
> didn't work perfectly.
>
> I would look at the processing of the messages and find a way within the
> application to detect when processing of a message should not continue due
> to a prior, related message failing.  If the messages do not have sequence
> numbers on them, it would be very simple to add a processor (e.g. using
> camel) that reads the messages and assigns incremental sequence numbers to
> each message in the same steam (e.g.  message 1, 2, and 3 for
> student_1000).
> Then the application can track the last successful update for each student
> and reject updates that are out-of-sequence.
>
> Actually, even JMS Groups aren't the best messaging pattern; they sound
> like
> a good idea, but in practice JMS Groups can create a lot of problems.
> Idempotent processing is the gold-standard enabling fully competing
> consumers and true horizontal scaling.
>
> Hope this helps.
>
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/ActiveMQ-broker-automatically-move-messages-with-same-JMSXGroupID-to-DLQ-tp4685943p4701948.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Re: ActiveMQ broker automatically move messages with same JMSXGroupID to DLQ

["rosy.salame@murex.com" <ro...@murex.com>]

Thanks a lot for your reply; now, it makes sense.  I just have one question
concerning your last point; I didn't understand why JMS groups can create
lots of problems? In your opinion, we should not use it?

Thank you,

Rosy



--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-broker-automatically-move-messages-with-same-JMSXGroupID-to-DLQ-tp4685943p4701975.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ broker automatically move messages with same JMSXGroupID to DLQ

[artnaseef <ar...@artnaseef.com>]

Hey Rosy - ActiveMQ is a messaging solution, primarily focused on getting
messages from producers to consumers.  Advanced logic of that type really
belongs in an application.  There's no straight-forward way to add that to
ActiveMQ itself, and it would be difficult to diagnose if such functionality
didn't work perfectly.

I would look at the processing of the messages and find a way within the
application to detect when processing of a message should not continue due
to a prior, related message failing.  If the messages do not have sequence
numbers on them, it would be very simple to add a processor (e.g. using
camel) that reads the messages and assigns incremental sequence numbers to
each message in the same steam (e.g.  message 1, 2, and 3 for student_1000). 
Then the application can track the last successful update for each student
and reject updates that are out-of-sequence.

Actually, even JMS Groups aren't the best messaging pattern; they sound like
a good idea, but in practice JMS Groups can create a lot of problems. 
Idempotent processing is the gold-standard enabling fully competing
consumers and true horizontal scaling.

Hope this helps.




--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-broker-automatically-move-messages-with-same-JMSXGroupID-to-DLQ-tp4685943p4701948.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ broker automatically move messages with same JMSXGroupID to DLQ

["rosy.salame@murex.com" <ro...@murex.com>]

Hello,

I'm facing the same issue.
Is there an option on Active MQ level to preserve the order of execution of
messages in case of an exception? . In other words, assume that we have
inside message ID=1 info about an object called student having for example
ID=Student_1000 and this message failed and entered in DLQ for a certain
reason but we have in the principal queue message ID= 2 and message ID = 3
having the same ID of this student (ID=Student_1000) . We should not allow
those messages from getting processed because they are containing info about
same ID of object as inside message ID = 1; ideally, they should be
redirected directly to DLQ to preserve the order of execution because if we
allow this processing, we will loose the order of execution in case we are
performing an update. Please note that I'm using message groups of Active
MQ.

How to do that on Active MQ level?

Many thanks,

Rosy



--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-broker-automatically-move-messages-with-same-JMSXGroupID-to-DLQ-tp4685943p4701915.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.