You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "Gary Tully (JIRA)" <ji...@apache.org> on 2018/06/29 09:46:00 UTC
[jira] [Comment Edited] (AMQ-6666) Failover Transport - send
timeout not working
[ https://issues.apache.org/jira/browse/AMQ-6666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16527363#comment-16527363 ]
Gary Tully edited comment on AMQ-6666 at 6/29/18 9:45 AM:
----------------------------------------------------------
The sendTimeout is confined to message send commands.
these transaction related commands have a 0 timeout so they will block pending failover connection.
In these cases, the FailoverTransport configuration related to retries should be the answer.
{code:java}
startupMaxReconnectAttempts{code}
and
{code:java}
maxReconnectAttempts{code}
These should limit the reconnect thread and combined with the
{code:java}
maxReconnectDelay, initialReconnectDelay, useExponentialBackOff, backOffMultiplier
{code}
let the connection failure bubble up to the application after some period.
was (Author: gtully):
The sendTimeout is confined to message send commands.
these transaction related commands have a 0 timeout so they will block pending failover connection.
In these cases, the FailoverTransport configuraton related to retries should be the answer.
{code:java}
startupMaxReconnectAttempts{code}
and
{code:java}
maxReconnectAttempts{code}
These should limit the reconnect thread and combined with the
{code:java}
maxReconnectDelay, initialReconnectDelay, useExponentialBackOff, backOffMultiplier
{code}
let the connection failure bubble up to the application after some period.
> Failover Transport - send timeout not working
> ---------------------------------------------
>
> Key: AMQ-6666
> URL: https://issues.apache.org/jira/browse/AMQ-6666
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.14.0
> Reporter: Martin Lichtin
> Priority: Critical
>
> Running into a situation with the Failover Transport not respecting the timeout that's been set. The symptom is endless messages of this kind:
> {noformat}
> 2017-04-29 09:48:26,128 | TRACE | .engine.cfgengine.in]-11 | FailoverTransport | sport.failover.FailoverTransport 615 | 81 - org.apache.activemq.activemq-osgi - 5.14.0 | Waiting for transport to reconnect..: TransactionInfo {commandId = 127798, responseRequired = true, type = 7, connectionId = ID:inucdev4-57330-1493370444659-3:3, transactionId = XID:[1096044365,globalId=6374726c6366672d656e67696e653130333530323030303034,branchId=6374726c6366672d656e67696e6531313036383134]}
> 2017-04-29 09:48:26,228 | TRACE | .engine.cfgengine.in]-11 | FailoverTransport | sport.failover.FailoverTransport 615 | 81 - org.apache.activemq.activemq-osgi - 5.14.0 | Waiting for transport to reconnect..: TransactionInfo {commandId = 127798, responseRequired = true, type = 7, connectionId = ID:inucdev4-57330-1493370444659-3:3, transactionId = XID:[1096044365,globalId=6374726c6366672d656e67696e653130333530323030303034,branchId=6374726c6366672d656e67696e6531313036383134]}
> 2017-04-29 09:48:26,329 | TRACE | .engine.cfgengine.in]-11 | FailoverTransport | sport.failover.FailoverTransport 615 | 81 - org.apache.activemq.activemq-osgi - 5.14.0 | Waiting for transport to reconnect..: TransactionInfo {commandId = 127798, responseRequired = true, type = 7, connectionId = ID:inucdev4-57330-1493370444659-3:3, transactionId = XID:[1096044365,globalId=6374726c6366672d656e67696e653130333530323030303034,branchId=6374726c6366672d656e67696e6531313036383134]}
> 2017-04-29 09:48:26,429 | TRACE | .engine.cfgengine.in]-11 | FailoverTransport | sport.failover.FailoverTransport 615 | 81 - org.apache.activemq.activemq-osgi - 5.14.0 | Waiting for transport to reconnect..: TransactionInfo {commandId = 127798, responseRequired = true, type = 7, connectionId = ID:inucdev4-57330-1493370444659-3:3, transactionId = XID:[1096044365,globalId=6374726c6366672d656e67696e653130333530323030303034,branchId=6374726c6366672d656e67696e6531313036383134]}
> 2017-04-29 09:48:26,530 | TRACE | .engine.cfgengine.in]-11 | FailoverTransport | sport.failover.FailoverTransport 615 | 81 - org.apache.activemq.activemq-osgi - 5.14.0 | Waiting for transport to reconnect..: TransactionInfo {commandId = 127798, responseRequired = true, type = 7, connectionId = ID:inucdev4-57330-1493370444659-3:3, transactionId = XID:[1096044365,globalId=6374726c6366672d656e67696e653130333530323030303034,branchId=6374726c6366672d656e67696e6531313036383134]}
> ...
> 2017-04-29 09:48:33,270 | TRACE | .engine.cfgengine.in]-11 | FailoverTransport | sport.failover.FailoverTransport 615 | 81 - org.apache.activemq.activemq-osgi - 5.14.0 | Waiting for transport to reconnect..: TransactionInfo {commandId = 127798, responseRequired = true, type = 7, connectionId = ID:inucdev4-57330-1493370444659-3:3, transactionId = XID:[1096044365,globalId=6374726c6366672d656e67696e653130333530323030303034,branchId=6374726c6366672d656e67696e6531313036383134]}
> 2017-04-29 09:48:33,371 | TRACE | .engine.cfgengine.in]-11 | FailoverTransport | sport.failover.FailoverTransport 615 | 81 - org.apache.activemq.activemq-osgi - 5.14.0 | Waiting for transport to reconnect..: TransactionInfo {commandId = 127798, responseRequired = true, type = 7, connectionId = ID:inucdev4-57330-1493370444659-3:3, transactionId = XID:[1096044365,globalId=6374726c6366672d656e67696e653130333530323030303034,branchId=6374726c6366672d656e67696e6531313036383134]}
> {noformat}
> The code seems to never get out of this loop:
> {noformat}
> while (transport == null && !disposed && connectionFailure == null
> && !Thread.currentThread().isInterrupted() && willReconnect()) {
> LOG.trace("Waiting for transport to reconnect..: {}", command);
> long end = System.currentTimeMillis();
> if (command.isMessage() && timeout > 0 && (end - start > timeout)) {
> timedout = true;
> LOG.info("Failover timed out after {} ms", (end - start));
> break;
> }
> try {
> reconnectMutex.wait(100);
> } catch (InterruptedException e) {
> Thread.currentThread().interrupt();
> LOG.debug("Interupted:", e);
> }
> transport = connectedTransport.get();
> }
> {noformat}
> The timeout is set to 5000ms and should have hit a long time ago, but as "command.isMessage()" returns false, it endlessly stays inside the loop.
> The "command.isMessage()" condition should likely be removed.
> Currently running tests with a patched ActiveMQ, and the situation has improved and fail-over seems to have worked (mostly). (Only seeing an issue with a topic consumer that has not reconnected.)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)