[jira] [Comment Edited] (QPIDJMS-563) QPID JMS - Failover - endless loop on sending a message with reject outcome

["Robbie Gemmell (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/QPIDJMS-563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17488161#comment-17488161 ] 

Robbie Gemmell edited comment on QPIDJMS-563 at 2/7/22, 2:56 PM:
-----------------------------------------------------------------

There is a 'jms.sendTimeout' URI option setting which applies a time out each individual send attempt by a live underlying producer, and also has some related handling for the time spent awaiting a new connection during failover before a send attempt can occur. I thought perhaps you could use that to work around this particular unexpected server behaviour, however in considering furhter how it operates I think since the connection _does_ actually succeed in being reestablished in this case and it does a send that it wont be effective, as send request will occur and will get explicitly failed as the connection is killed, and the connection drops+reestablishes succesfully, it will thus cancel the awaiting-failover handling process, and since that will happen essentially immediately each time it drops+reconnects it will likely never actually run that process. Unless perhaps that is, you _also_ set the 'failover.reconnectDelay' option _and_ ensure it has a _higher_ value than the 'jms.sendTimeout', meaning it doesnt even try reconnecting+sending again after a drop until after the awaiting-failover process has had chance to kill the waiting send request before a reconnection stops it (this obviously that may limit you in terms of what you reasonably want to set the two values to).


was (Author: gemmellr):
There is a 'jms.sendTimeout' URI option setting which applies a time out each individual send attempt by a live underlying producer, and also has some related handling for the time spent awaiting a new connection during failover before a send attempt can occur. I thought perhaps you could use that to work around this particular unexpected server behaviour, however in considering furhter how it operates I think since the connection _does_ actually succeed in being reestablished in this case and it does a send that it wont be effective, as send request will occur and will get explicitly failed as the connection is killed, and the connection drops+reestablishes succesfully, it will thus cancel the awaiting-failover handling process, and since that will happen essentially immediately each time it drops+reconnects it will likely never actually run that process. Unless perhaps that is, you _also_ set the 'failover.reconnectDelay' option and ensure it has a _higher_ value than the jms.sendTimeout, meaning it doesnt even try reconnecting+sending again after a drop until after the awaiting-failover process has had chance to kill the waiting send request before a reconnection stops it (this obviously that may limit you in terms of what you reasonably want to set the two values to).

> QPID JMS - Failover - endless loop on sending a message with reject outcome
> ---------------------------------------------------------------------------
>
>                 Key: QPIDJMS-563
>                 URL: https://issues.apache.org/jira/browse/QPIDJMS-563
>             Project: Qpid JMS
>          Issue Type: Bug
>          Components: qpid-jms-client
>    Affects Versions: 1.5.0
>            Reporter: Thomas Stollenwerk
>            Priority: Critical
>
> +*Summary:*+
> Having a amqp v1 message which is rejected by the amqp v1 broker results in an endless failover loop blocking the JmsMessageProducer.send method.
> +*Expected:*+
> The failover of the qpid jms client tries to connect and send the rejected message in a maximum of failover.maxReconnectAttempts.
> +*Actual:*+
> The result of the message send task is not taken into account for counting the attempts.
> The JmsMessageProducer.send gets stuck forever.
>  # Failover connects to the failover uri and then resets the attempt counter.
>  # Sending the message results in REJECT outcome.
>  # Failover connects to the next failover uri and then resets the attempt counter.
>  # Sending the message results in REJECT outcome.
>  # ... repeats and does not stop on maxReconnectAttempts.
> +*Code analysis:*+
> *FailoverProvider.java#L1282* already resets the connection attempts with *reconnectControl.connectionEstablished()* not waiting for the success of the send task, resulting in an endless failover loop.
> +*Background:*+
> In our case we are using RabbitMQ with the amqp v1 plugin. There are scenarios where the broker is answering with REJECT outcomes (overflow policy, high watermark etc). Right now we cannot use this feature because the client is stuck and the RabbitMQ is flooded with reconnects.
> We would be very grateful having a fix. 
> Thanks for the good work and best regards
> Thomas
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org