You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by sl...@cecurity.com on 2021/01/06 17:05:41 UTC
Redelivery policy parameters on artemis 2.10.1
Hello,
I have an unexpected behavior on redelivery with artemis.
The documentation states that redelivery is attempted 10 times by
default and that -1 means infinite
(https://activemq.apache.org/components/artemis/documentation/2.10.1/undelivered-messages.html)
(I checked the documentation matching version to be sure, but it seems
identical in latest)
I have queues with max-delivery-attempts set to -1 (in
<address-setting>, per queue). I checked the live queue properties in
JMX, it matches the xml configuration with -1 so a configuration loading
issue is not supected.
Now I have broken connexions on consumers (more on that later) and the
message is trashed after six attempts.
It produces the following (<redacted>) log statements
2021-01-06 16:56:28,016 WARN {Thread-16
(ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@f493a59)}
[org.apache.activemq.artemis.core.client] : AMQ212037: Connection
failure to /xxx.xxx.xxx.xxx:49918 has been detected: AMQ229014: Did not
receive data from /xxx.xxx.xxx.xxx:49918 within the 30,000ms connection
TTL. The connection will now be closed. [code=CONNECTION_TIMEDOUT]
2021-01-06 16:56:28,030 WARN {Thread-18 (activemq-netty-threads)}
[org.apache.activemq.artemis.core.server] : AMQ222150: Message
Reference[5011177500]:RELIABLE:CoreMessage[messageID=5011177500,durable=true,userID=6eeab1a3-5037-11eb-80d8-b02628c2b3dc,priority=4,
timestamp=Wed Jan 06 16:54:17 CET 2021,expiration=0, durable=true,
address=<address>,size=1665,properties=TypedProperties[__HDR_dlqDeliveryFailureCause=java.lang.Throwable:
Dispatch[8] to ID:<hostname>-44589-1607881800084-3:84233:1:106 exceeds
redelivery policy limit:RedeliveryPolicy {destination = null,
collisionAvoidanceFactor = 0.15, maximumRedeliveries = 6,
maximumRedeliveryDelay = -1, initialRedeliveryDelay = 1000,
useCollisionAvoidance = false, useExponentialBackOff = false,
backOffMultiplier = 5.0, redeliveryDelay = 1000, preDispatchCheck =
true},__AMQ_CID=5d351b8e-3ebb-11eb-80d8-b02628c2b3dc,_AMQ_ROUTING_TYPE=1,<more
headers> has exceeded max delivery attempts. No Dead Letter Address
configured for queue <address> so dropping it
the 6 redelivery attempts seems consistent with this other documentation :
https://access.redhat.com/documentation/en-us/red_hat_amq/6.3/html/client_connectivity_guide/amq-redelivery
So, any clue why we get this behavior ? Did I get the meaning of the
configuration directives wrong ? Is there a bug in the documentation
and/or artemis ?
Regards,
SL
Re: Redelivery policy parameters on artemis 2.10.1
Posted by Justin Bertram <jb...@apache.org>.
> Obviously the delivery failed after the server lost the connection to the
consumer...
Based on the logs you provided I don't see any direct link between the
connection timeout and the redelivery failure. In fact, when I read your
initial message I wondered why you included that log entry since it
appeared to be irrelevant. The log messages simply look coincidental to me.
To be clear, the redelivery failure message is a direct result of a
"poison" ack for the message from the client. If the broker doesn't receive
the poison ack from the client it won't attempt to send the message to the
dead-letter address. The broker clearly can't receive such a message over a
failed connection.
That said, *if* the connection which timed out actually did belong to the
consumer where delivery failed I would chalk the log message order up to
thread timing. The broker is highly multi-threaded so sometimes you see
this kind of thing where log messages don't follow the expected logical
order. For what it's worth, it's clear from the logs that the threads
logging those messages are, in fact, different.
> Can I get the expected infinite delivery by changing the client broker
url parameters like in the redhat doc ?
Yes.
Justin
On Wed, Jan 6, 2021 at 12:28 PM <sl...@cecurity.com> wrote:
> Thank you for your response.
>
> There is still a few things that are not clear for me.
>
> Obviously the delivery failed after the server lost the connection to
> the consumer, so i don't understand how, if the redelivery count was
> updated by an opaque client side delivery, the server got an up to date
> counter on the server side when the connection was severed ?
>
> Can I get the expected infinite delivery by changing the client broker
> url parameters like in the redhat doc ?
>
> Regards,
>
> SL
>
>
> Le 06/01/2021 à 18:36, Justin Bertram a écrit :
> > As far as I can tell you're seeing the overlap between how ActiveMQ
> Artemis
> > handles redelivery and how the OpenWire JMS client (written for ActiveMQ
> > 5.x) handles redelivery. To simplify, ActiveMQ Artemis handles redelivery
> > on the broker (configured via address-settings), but the OpenWire JMS
> > client handles redelivery itself (configured via its RedeliveryPolicy).
> >
> > The RedeliveryPolicy of the OpenWire JMS client will override the
> > address-settings configured on the broker because once the OpenWire JMS
> > client has the message it has control. It doesn't even tell the broker
> that
> > redelivery is happening. The only thing it tells the broker is if
> > redelivery ultimately fails at which point the message will be sent to
> the
> > configured dead-letter address (or not).
> >
> > Based on your observations as well as the log message it appears your
> > OpenWire JMS client is using a RedeliveryPolicy with a
> maximumRedeliveries
> > of 6. Therefore even though your address-settings use -1 for
> > max-delivery-attempts you won't actually get infinite redeliveries.
> >
> > Hope that helps.
> >
> >
> > Justin
> >
> > On Wed, Jan 6, 2021 at 11:06 AM <sl...@cecurity.com> wrote:
> >
> >> Hello,
> >>
> >> I have an unexpected behavior on redelivery with artemis.
> >>
> >> The documentation states that redelivery is attempted 10 times by
> >> default and that -1 means infinite
> >>
> >> (
> >>
> https://activemq.apache.org/components/artemis/documentation/2.10.1/undelivered-messages.html
> >> )
> >>
> >> (I checked the documentation matching version to be sure, but it seems
> >> identical in latest)
> >>
> >> I have queues with max-delivery-attempts set to -1 (in
> >> <address-setting>, per queue). I checked the live queue properties in
> >> JMX, it matches the xml configuration with -1 so a configuration loading
> >> issue is not supected.
> >>
> >> Now I have broken connexions on consumers (more on that later) and the
> >> message is trashed after six attempts.
> >>
> >> It produces the following (<redacted>) log statements
> >>
> >> 2021-01-06 16:56:28,016 WARN {Thread-16
> >>
> >>
> (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@f493a59
> >> )}
> >> [org.apache.activemq.artemis.core.client] : AMQ212037: Connection
> >> failure to /xxx.xxx.xxx.xxx:49918 has been detected: AMQ229014: Did not
> >> receive data from /xxx.xxx.xxx.xxx:49918 within the 30,000ms connection
> >> TTL. The connection will now be closed. [code=CONNECTION_TIMEDOUT]
> >> 2021-01-06 16:56:28,030 WARN {Thread-18 (activemq-netty-threads)}
> >> [org.apache.activemq.artemis.core.server] : AMQ222150: Message
> >>
> >>
> Reference[5011177500]:RELIABLE:CoreMessage[messageID=5011177500,durable=true,userID=6eeab1a3-5037-11eb-80d8-b02628c2b3dc,priority=4,
> >> timestamp=Wed Jan 06 16:54:17 CET 2021,expiration=0, durable=true,
> >>
> >>
> address=<address>,size=1665,properties=TypedProperties[__HDR_dlqDeliveryFailureCause=java.lang.Throwable:
> >> Dispatch[8] to ID:<hostname>-44589-1607881800084-3:84233:1:106 exceeds
> >> redelivery policy limit:RedeliveryPolicy {destination = null,
> >> collisionAvoidanceFactor = 0.15, maximumRedeliveries = 6,
> >> maximumRedeliveryDelay = -1, initialRedeliveryDelay = 1000,
> >> useCollisionAvoidance = false, useExponentialBackOff = false,
> >> backOffMultiplier = 5.0, redeliveryDelay = 1000, preDispatchCheck =
> >>
> >>
> true},__AMQ_CID=5d351b8e-3ebb-11eb-80d8-b02628c2b3dc,_AMQ_ROUTING_TYPE=1,<more
> >> headers> has exceeded max delivery attempts. No Dead Letter Address
> >> configured for queue <address> so dropping it
> >>
> >> the 6 redelivery attempts seems consistent with this other
> documentation :
> >>
> >>
> >>
> https://access.redhat.com/documentation/en-us/red_hat_amq/6.3/html/client_connectivity_guide/amq-redelivery
> >>
> >> So, any clue why we get this behavior ? Did I get the meaning of the
> >> configuration directives wrong ? Is there a bug in the documentation
> >> and/or artemis ?
> >>
> >> Regards,
> >>
> >> SL
> >>
> >>
>
>
Re: Redelivery policy parameters on artemis 2.10.1
Posted by sl...@cecurity.com.
Thank you for your response.
There is still a few things that are not clear for me.
Obviously the delivery failed after the server lost the connection to
the consumer, so i don't understand how, if the redelivery count was
updated by an opaque client side delivery, the server got an up to date
counter on the server side when the connection was severed ?
Can I get the expected infinite delivery by changing the client broker
url parameters like in the redhat doc ?
Regards,
SL
Le 06/01/2021 à 18:36, Justin Bertram a écrit :
> As far as I can tell you're seeing the overlap between how ActiveMQ Artemis
> handles redelivery and how the OpenWire JMS client (written for ActiveMQ
> 5.x) handles redelivery. To simplify, ActiveMQ Artemis handles redelivery
> on the broker (configured via address-settings), but the OpenWire JMS
> client handles redelivery itself (configured via its RedeliveryPolicy).
>
> The RedeliveryPolicy of the OpenWire JMS client will override the
> address-settings configured on the broker because once the OpenWire JMS
> client has the message it has control. It doesn't even tell the broker that
> redelivery is happening. The only thing it tells the broker is if
> redelivery ultimately fails at which point the message will be sent to the
> configured dead-letter address (or not).
>
> Based on your observations as well as the log message it appears your
> OpenWire JMS client is using a RedeliveryPolicy with a maximumRedeliveries
> of 6. Therefore even though your address-settings use -1 for
> max-delivery-attempts you won't actually get infinite redeliveries.
>
> Hope that helps.
>
>
> Justin
>
> On Wed, Jan 6, 2021 at 11:06 AM <sl...@cecurity.com> wrote:
>
>> Hello,
>>
>> I have an unexpected behavior on redelivery with artemis.
>>
>> The documentation states that redelivery is attempted 10 times by
>> default and that -1 means infinite
>>
>> (
>> https://activemq.apache.org/components/artemis/documentation/2.10.1/undelivered-messages.html
>> )
>>
>> (I checked the documentation matching version to be sure, but it seems
>> identical in latest)
>>
>> I have queues with max-delivery-attempts set to -1 (in
>> <address-setting>, per queue). I checked the live queue properties in
>> JMX, it matches the xml configuration with -1 so a configuration loading
>> issue is not supected.
>>
>> Now I have broken connexions on consumers (more on that later) and the
>> message is trashed after six attempts.
>>
>> It produces the following (<redacted>) log statements
>>
>> 2021-01-06 16:56:28,016 WARN {Thread-16
>>
>> (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@f493a59
>> )}
>> [org.apache.activemq.artemis.core.client] : AMQ212037: Connection
>> failure to /xxx.xxx.xxx.xxx:49918 has been detected: AMQ229014: Did not
>> receive data from /xxx.xxx.xxx.xxx:49918 within the 30,000ms connection
>> TTL. The connection will now be closed. [code=CONNECTION_TIMEDOUT]
>> 2021-01-06 16:56:28,030 WARN {Thread-18 (activemq-netty-threads)}
>> [org.apache.activemq.artemis.core.server] : AMQ222150: Message
>>
>> Reference[5011177500]:RELIABLE:CoreMessage[messageID=5011177500,durable=true,userID=6eeab1a3-5037-11eb-80d8-b02628c2b3dc,priority=4,
>> timestamp=Wed Jan 06 16:54:17 CET 2021,expiration=0, durable=true,
>>
>> address=<address>,size=1665,properties=TypedProperties[__HDR_dlqDeliveryFailureCause=java.lang.Throwable:
>> Dispatch[8] to ID:<hostname>-44589-1607881800084-3:84233:1:106 exceeds
>> redelivery policy limit:RedeliveryPolicy {destination = null,
>> collisionAvoidanceFactor = 0.15, maximumRedeliveries = 6,
>> maximumRedeliveryDelay = -1, initialRedeliveryDelay = 1000,
>> useCollisionAvoidance = false, useExponentialBackOff = false,
>> backOffMultiplier = 5.0, redeliveryDelay = 1000, preDispatchCheck =
>>
>> true},__AMQ_CID=5d351b8e-3ebb-11eb-80d8-b02628c2b3dc,_AMQ_ROUTING_TYPE=1,<more
>> headers> has exceeded max delivery attempts. No Dead Letter Address
>> configured for queue <address> so dropping it
>>
>> the 6 redelivery attempts seems consistent with this other documentation :
>>
>>
>> https://access.redhat.com/documentation/en-us/red_hat_amq/6.3/html/client_connectivity_guide/amq-redelivery
>>
>> So, any clue why we get this behavior ? Did I get the meaning of the
>> configuration directives wrong ? Is there a bug in the documentation
>> and/or artemis ?
>>
>> Regards,
>>
>> SL
>>
>>
Re: Redelivery policy parameters on artemis 2.10.1
Posted by Justin Bertram <jb...@apache.org>.
As far as I can tell you're seeing the overlap between how ActiveMQ Artemis
handles redelivery and how the OpenWire JMS client (written for ActiveMQ
5.x) handles redelivery. To simplify, ActiveMQ Artemis handles redelivery
on the broker (configured via address-settings), but the OpenWire JMS
client handles redelivery itself (configured via its RedeliveryPolicy).
The RedeliveryPolicy of the OpenWire JMS client will override the
address-settings configured on the broker because once the OpenWire JMS
client has the message it has control. It doesn't even tell the broker that
redelivery is happening. The only thing it tells the broker is if
redelivery ultimately fails at which point the message will be sent to the
configured dead-letter address (or not).
Based on your observations as well as the log message it appears your
OpenWire JMS client is using a RedeliveryPolicy with a maximumRedeliveries
of 6. Therefore even though your address-settings use -1 for
max-delivery-attempts you won't actually get infinite redeliveries.
Hope that helps.
Justin
On Wed, Jan 6, 2021 at 11:06 AM <sl...@cecurity.com> wrote:
> Hello,
>
> I have an unexpected behavior on redelivery with artemis.
>
> The documentation states that redelivery is attempted 10 times by
> default and that -1 means infinite
>
> (
> https://activemq.apache.org/components/artemis/documentation/2.10.1/undelivered-messages.html
> )
>
> (I checked the documentation matching version to be sure, but it seems
> identical in latest)
>
> I have queues with max-delivery-attempts set to -1 (in
> <address-setting>, per queue). I checked the live queue properties in
> JMX, it matches the xml configuration with -1 so a configuration loading
> issue is not supected.
>
> Now I have broken connexions on consumers (more on that later) and the
> message is trashed after six attempts.
>
> It produces the following (<redacted>) log statements
>
> 2021-01-06 16:56:28,016 WARN {Thread-16
>
> (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@f493a59
> )}
> [org.apache.activemq.artemis.core.client] : AMQ212037: Connection
> failure to /xxx.xxx.xxx.xxx:49918 has been detected: AMQ229014: Did not
> receive data from /xxx.xxx.xxx.xxx:49918 within the 30,000ms connection
> TTL. The connection will now be closed. [code=CONNECTION_TIMEDOUT]
> 2021-01-06 16:56:28,030 WARN {Thread-18 (activemq-netty-threads)}
> [org.apache.activemq.artemis.core.server] : AMQ222150: Message
>
> Reference[5011177500]:RELIABLE:CoreMessage[messageID=5011177500,durable=true,userID=6eeab1a3-5037-11eb-80d8-b02628c2b3dc,priority=4,
> timestamp=Wed Jan 06 16:54:17 CET 2021,expiration=0, durable=true,
>
> address=<address>,size=1665,properties=TypedProperties[__HDR_dlqDeliveryFailureCause=java.lang.Throwable:
> Dispatch[8] to ID:<hostname>-44589-1607881800084-3:84233:1:106 exceeds
> redelivery policy limit:RedeliveryPolicy {destination = null,
> collisionAvoidanceFactor = 0.15, maximumRedeliveries = 6,
> maximumRedeliveryDelay = -1, initialRedeliveryDelay = 1000,
> useCollisionAvoidance = false, useExponentialBackOff = false,
> backOffMultiplier = 5.0, redeliveryDelay = 1000, preDispatchCheck =
>
> true},__AMQ_CID=5d351b8e-3ebb-11eb-80d8-b02628c2b3dc,_AMQ_ROUTING_TYPE=1,<more
> headers> has exceeded max delivery attempts. No Dead Letter Address
> configured for queue <address> so dropping it
>
> the 6 redelivery attempts seems consistent with this other documentation :
>
>
> https://access.redhat.com/documentation/en-us/red_hat_amq/6.3/html/client_connectivity_guide/amq-redelivery
>
> So, any clue why we get this behavior ? Did I get the meaning of the
> configuration directives wrong ? Is there a bug in the documentation
> and/or artemis ?
>
> Regards,
>
> SL
>
>