You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Joan Pujol <jo...@gmail.com> on 2020/06/02 14:19:23 UTC
Message group and broker redelivery
Hi,
Searching in mailing list archives for message group and message
redelivery I've seen different responses and it's not clear to me the
supposed way to work.
If I had a queue with message groups is ordered guaranteed if broker
redeliveries are present?
That is if I have:
m1,m2, and m3 queued with the same group key and m2 is redelivered
after 5 mins, will m3 wait or not?
If not, is there someone with this use case implying message order and
redelivery, and can explain how it manages it?
A lot of thanks in advance,
--
Joan Jesús Pujol Espinar
Re: Message group and broker redelivery
Posted by Justin Bertram <jb...@apache.org>.
The broker doesn't really support dependency between messages as you seem
to need. As you know, grouped messages will be sent to the same consumer,
but as far as the broker is concerned those messages are still 100%
independent of each other. It's not clear to me why a redelivery-delay of 0
wouldn't work for you. You could always implement delays in your own
message processing code as necessary. However, if using a redelivery-delay
of 0 is not possible or desirable for whatever reason then I would go with
the second option you listed - checking the dependency order and triggering
a redelivery if the order is broken. That should eventually put the
messages in the group back into order.
The broker's behavior with regards to blocking during a redelivery delay is
not configurable. It prefers to maximize message throughput in this case.
Justin
On Tue, Jun 2, 2020 at 10:34 AM Joan Pujol <jo...@gmail.com> wrote:
> Ok, and if using a redelivery-delay is not an option any recommended
> way to manage it?
> My best option by now is doing a pre-processing by not enqueuing
> dependent messages and have a listener to enqueue them when it's
> dependant was processed
> Or checking before processing a message if there is another in which
> it depends, forcing the dependent message to be redelivered also,...
>
> About what you comment regarding performance, it's not a very
> high-performance environment with very high throughput. It's better to
> be resilient and have strict order guarantee and exponential backoff
> retries to cover problems with external integrations.
> And by using message-groups it allows us to only lock messages that
> are on the same message-group, while allowing another messages to
> process normally, that is exactly the use case we want.
>
> Cheers,
>
> On Tue, 2 Jun 2020 at 17:22, Justin Bertram <jb...@apache.org> wrote:
> >
> > > If I had a queue with message groups is ordered guaranteed if broker
> > redeliveries are present?
> >
> > If you use a redelivery-delay > 0 then delivery order will be broken. If
> > you use a redelivery-delay of 0 then delivery order will not be broken.
> > Therefore, if you want to maintain strict order then use a
> redelivery-delay
> > of 0.
> >
> > > That is if I have: m1,m2, and m3 queued with the same group key and m2
> is
> > redelivered after 5 mins, will m3 wait or not?
> >
> > Delivery for m3 will not wait. If the broker blocked delivery of
> subsequent
> > messages on the queue during a redelivery delay that would significantly
> > impact message throughput performance in a negative way. This would not
> be
> > tenable for an enterprise message broker serving hundreds or perhaps
> > thousands of clients each of whom may regularly be triggering
> redeliveries
> > on shared queues.
> >
> >
> > Justin
> >
> > On Tue, Jun 2, 2020 at 10:04 AM Joan Pujol <jo...@gmail.com> wrote:
> >
> > > ActiveMQ Artemis
> > >
> > > On Tue, 2 Jun 2020 at 16:22, Justin Bertram <jb...@apache.org>
> wrote:
> > > >
> > > > Are you asking about ActiveMQ 5.x or ActiveMQ Artemis?
> > > >
> > > >
> > > > Justin
> > > >
> > > > On Tue, Jun 2, 2020 at 9:19 AM Joan Pujol <jo...@gmail.com>
> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Searching in mailing list archives for message group and message
> > > > > redelivery I've seen different responses and it's not clear to me
> the
> > > > > supposed way to work.
> > > > >
> > > > > If I had a queue with message groups is ordered guaranteed if
> broker
> > > > > redeliveries are present?
> > > > >
> > > > > That is if I have:
> > > > > m1,m2, and m3 queued with the same group key and m2 is redelivered
> > > > > after 5 mins, will m3 wait or not?
> > > > > If not, is there someone with this use case implying message order
> and
> > > > > redelivery, and can explain how it manages it?
> > > > >
> > > > > A lot of thanks in advance,
> > > > > --
> > > > > Joan Jesús Pujol Espinar
> > > > >
> > > > >
> > >
> > >
> > >
> > > --
> > > Joan Jesús Pujol Espinar
> > > http://www.joanpujol.cat
> > > https://play.google.com/store/apps/developer?id=lujop&hl=ca
> > >
> > > --
> > > Joan Jesús Pujol Espinar
> > > http://www.joanpujol.cat
> > > http://lujop.deviantart.com
> > >
> > >
>
>
>
> --
> Joan Jesús Pujol Espinar
> http://www.joanpujol.cat
> https://play.google.com/store/apps/developer?id=lujop&hl=ca
>
> --
> Joan Jesús Pujol Espinar
> http://www.joanpujol.cat
> http://lujop.deviantart.com
>
>
Re: Message group and broker redelivery
Posted by Joan Pujol <jo...@gmail.com>.
Ok, and if using a redelivery-delay is not an option any recommended
way to manage it?
My best option by now is doing a pre-processing by not enqueuing
dependent messages and have a listener to enqueue them when it's
dependant was processed
Or checking before processing a message if there is another in which
it depends, forcing the dependent message to be redelivered also,...
About what you comment regarding performance, it's not a very
high-performance environment with very high throughput. It's better to
be resilient and have strict order guarantee and exponential backoff
retries to cover problems with external integrations.
And by using message-groups it allows us to only lock messages that
are on the same message-group, while allowing another messages to
process normally, that is exactly the use case we want.
Cheers,
On Tue, 2 Jun 2020 at 17:22, Justin Bertram <jb...@apache.org> wrote:
>
> > If I had a queue with message groups is ordered guaranteed if broker
> redeliveries are present?
>
> If you use a redelivery-delay > 0 then delivery order will be broken. If
> you use a redelivery-delay of 0 then delivery order will not be broken.
> Therefore, if you want to maintain strict order then use a redelivery-delay
> of 0.
>
> > That is if I have: m1,m2, and m3 queued with the same group key and m2 is
> redelivered after 5 mins, will m3 wait or not?
>
> Delivery for m3 will not wait. If the broker blocked delivery of subsequent
> messages on the queue during a redelivery delay that would significantly
> impact message throughput performance in a negative way. This would not be
> tenable for an enterprise message broker serving hundreds or perhaps
> thousands of clients each of whom may regularly be triggering redeliveries
> on shared queues.
>
>
> Justin
>
> On Tue, Jun 2, 2020 at 10:04 AM Joan Pujol <jo...@gmail.com> wrote:
>
> > ActiveMQ Artemis
> >
> > On Tue, 2 Jun 2020 at 16:22, Justin Bertram <jb...@apache.org> wrote:
> > >
> > > Are you asking about ActiveMQ 5.x or ActiveMQ Artemis?
> > >
> > >
> > > Justin
> > >
> > > On Tue, Jun 2, 2020 at 9:19 AM Joan Pujol <jo...@gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > Searching in mailing list archives for message group and message
> > > > redelivery I've seen different responses and it's not clear to me the
> > > > supposed way to work.
> > > >
> > > > If I had a queue with message groups is ordered guaranteed if broker
> > > > redeliveries are present?
> > > >
> > > > That is if I have:
> > > > m1,m2, and m3 queued with the same group key and m2 is redelivered
> > > > after 5 mins, will m3 wait or not?
> > > > If not, is there someone with this use case implying message order and
> > > > redelivery, and can explain how it manages it?
> > > >
> > > > A lot of thanks in advance,
> > > > --
> > > > Joan Jesús Pujol Espinar
> > > >
> > > >
> >
> >
> >
> > --
> > Joan Jesús Pujol Espinar
> > http://www.joanpujol.cat
> > https://play.google.com/store/apps/developer?id=lujop&hl=ca
> >
> > --
> > Joan Jesús Pujol Espinar
> > http://www.joanpujol.cat
> > http://lujop.deviantart.com
> >
> >
--
Joan Jesús Pujol Espinar
http://www.joanpujol.cat
https://play.google.com/store/apps/developer?id=lujop&hl=ca
--
Joan Jesús Pujol Espinar
http://www.joanpujol.cat
http://lujop.deviantart.com
Re: Message group and broker redelivery
Posted by Justin Bertram <jb...@apache.org>.
> If I had a queue with message groups is ordered guaranteed if broker
redeliveries are present?
If you use a redelivery-delay > 0 then delivery order will be broken. If
you use a redelivery-delay of 0 then delivery order will not be broken.
Therefore, if you want to maintain strict order then use a redelivery-delay
of 0.
> That is if I have: m1,m2, and m3 queued with the same group key and m2 is
redelivered after 5 mins, will m3 wait or not?
Delivery for m3 will not wait. If the broker blocked delivery of subsequent
messages on the queue during a redelivery delay that would significantly
impact message throughput performance in a negative way. This would not be
tenable for an enterprise message broker serving hundreds or perhaps
thousands of clients each of whom may regularly be triggering redeliveries
on shared queues.
Justin
On Tue, Jun 2, 2020 at 10:04 AM Joan Pujol <jo...@gmail.com> wrote:
> ActiveMQ Artemis
>
> On Tue, 2 Jun 2020 at 16:22, Justin Bertram <jb...@apache.org> wrote:
> >
> > Are you asking about ActiveMQ 5.x or ActiveMQ Artemis?
> >
> >
> > Justin
> >
> > On Tue, Jun 2, 2020 at 9:19 AM Joan Pujol <jo...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > Searching in mailing list archives for message group and message
> > > redelivery I've seen different responses and it's not clear to me the
> > > supposed way to work.
> > >
> > > If I had a queue with message groups is ordered guaranteed if broker
> > > redeliveries are present?
> > >
> > > That is if I have:
> > > m1,m2, and m3 queued with the same group key and m2 is redelivered
> > > after 5 mins, will m3 wait or not?
> > > If not, is there someone with this use case implying message order and
> > > redelivery, and can explain how it manages it?
> > >
> > > A lot of thanks in advance,
> > > --
> > > Joan Jesús Pujol Espinar
> > >
> > >
>
>
>
> --
> Joan Jesús Pujol Espinar
> http://www.joanpujol.cat
> https://play.google.com/store/apps/developer?id=lujop&hl=ca
>
> --
> Joan Jesús Pujol Espinar
> http://www.joanpujol.cat
> http://lujop.deviantart.com
>
>
Re: Message group and broker redelivery
Posted by Joan Pujol <jo...@gmail.com>.
ActiveMQ Artemis
On Tue, 2 Jun 2020 at 16:22, Justin Bertram <jb...@apache.org> wrote:
>
> Are you asking about ActiveMQ 5.x or ActiveMQ Artemis?
>
>
> Justin
>
> On Tue, Jun 2, 2020 at 9:19 AM Joan Pujol <jo...@gmail.com> wrote:
>
> > Hi,
> >
> > Searching in mailing list archives for message group and message
> > redelivery I've seen different responses and it's not clear to me the
> > supposed way to work.
> >
> > If I had a queue with message groups is ordered guaranteed if broker
> > redeliveries are present?
> >
> > That is if I have:
> > m1,m2, and m3 queued with the same group key and m2 is redelivered
> > after 5 mins, will m3 wait or not?
> > If not, is there someone with this use case implying message order and
> > redelivery, and can explain how it manages it?
> >
> > A lot of thanks in advance,
> > --
> > Joan Jesús Pujol Espinar
> >
> >
--
Joan Jesús Pujol Espinar
http://www.joanpujol.cat
https://play.google.com/store/apps/developer?id=lujop&hl=ca
--
Joan Jesús Pujol Espinar
http://www.joanpujol.cat
http://lujop.deviantart.com
Re: Message group and broker redelivery
Posted by Joan Pujol <jo...@joanpujol.cat>.
ActiveMQ Artemis.
On Tue, 2 Jun 2020 at 16:22, Justin Bertram <jb...@apache.org> wrote:
>
> Are you asking about ActiveMQ 5.x or ActiveMQ Artemis?
>
>
> Justin
>
> On Tue, Jun 2, 2020 at 9:19 AM Joan Pujol <jo...@gmail.com> wrote:
>
> > Hi,
> >
> > Searching in mailing list archives for message group and message
> > redelivery I've seen different responses and it's not clear to me the
> > supposed way to work.
> >
> > If I had a queue with message groups is ordered guaranteed if broker
> > redeliveries are present?
> >
> > That is if I have:
> > m1,m2, and m3 queued with the same group key and m2 is redelivered
> > after 5 mins, will m3 wait or not?
> > If not, is there someone with this use case implying message order and
> > redelivery, and can explain how it manages it?
> >
> > A lot of thanks in advance,
> > --
> > Joan Jesús Pujol Espinar
> >
> >
--
Joan Jesús Pujol Espinar
http://www.joanpujol.cat
https://play.google.com/store/apps/developer?id=lujop&hl=ca
Re: Message group and broker redelivery
Posted by Justin Bertram <jb...@apache.org>.
Are you asking about ActiveMQ 5.x or ActiveMQ Artemis?
Justin
On Tue, Jun 2, 2020 at 9:19 AM Joan Pujol <jo...@gmail.com> wrote:
> Hi,
>
> Searching in mailing list archives for message group and message
> redelivery I've seen different responses and it's not clear to me the
> supposed way to work.
>
> If I had a queue with message groups is ordered guaranteed if broker
> redeliveries are present?
>
> That is if I have:
> m1,m2, and m3 queued with the same group key and m2 is redelivered
> after 5 mins, will m3 wait or not?
> If not, is there someone with this use case implying message order and
> redelivery, and can explain how it manages it?
>
> A lot of thanks in advance,
> --
> Joan Jesús Pujol Espinar
>
>