You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2019/01/21 13:51:00 UTC
[jira] [Work logged] (ARTEMIS-2069) Backup doesn't activate after
shared store is reconnected
[ https://issues.apache.org/jira/browse/ARTEMIS-2069?focusedWorklogId=187787&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-187787 ]
ASF GitHub Bot logged work on ARTEMIS-2069:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 21/Jan/19 13:50
Start Date: 21/Jan/19 13:50
Worklog Time Spent: 10m
Work Description: TomasHofman commented on issue #2287: ARTEMIS-2069 Backup doesn't activate after shared store is reconnected
URL: https://github.com/apache/activemq-artemis/pull/2287#issuecomment-446176184
No, if `lockAcquisitionTimeout` is set to -1, it will wait 2 seconds, and then retry the lock (the whole thing is in a loop).
There are two situations:
* `tryLock(pos)` returns null (= lock is already taken), wait 500 ms and repeat - *this hasn't changed*.
* `tryLock(pos)` throws IOException:
* If `lockAcquisitionTimeout == -1` it will wait 2 seconds and then retry, forever.
* If `lockAcquisitionTimeout != -1`, it will wait 2 seconds or remaining time until timeout and retry, or if remaining time is <= 0 exception is thrown.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 187787)
Time Spent: 10m
Remaining Estimate: 0h
> Backup doesn't activate after shared store is reconnected
> ---------------------------------------------------------
>
> Key: ARTEMIS-2069
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2069
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Affects Versions: 2.6.2
> Reporter: Tomas Hofman
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> *Scenario*
> # Start live backup server pair in dedicated topology with shared store HA, with journal located on NFS
> # NFS mounted on backup server fails
> # Reconnect NFS on backup server
> # Try to shut down live EAP server
> # Backup doesn't activate
> *What happens*
> Backup is waiting for live to fail by checking its file lock. In case the connection to shared storage fails, backup logs following error.
>
> |{color:#000000}05:50:57,896 ERROR [org.apache.activemq.artemis.core.server] (AMQ119000: Activation for server ActiveMQServerImpl::serverUUID=836c9b1e-f067-11e7-8763-001b21862475) AMQ224000: Failure in initialisation: java.io.IOException: Input/output error{color}|
> |{color:#000000} at sun.nio.ch.FileDispatcherImpl.lock0(Native Method) [rt.jar:1.8.0_151]{color}|
> |{color:#000000} at sun.nio.ch.FileDispatcherImpl.lock(FileDispatcherImpl.java:90) [rt.jar:1.8.0_151]{color}|
> |{color:#000000} at sun.nio.ch.FileChannelImpl.tryLock(FileChannelImpl.java:1115) [rt.jar:1.8.0_151]{color}|
> |{color:#000000} at org.apache.activemq.artemis.core.server.impl.FileLockNodeManager.tryLock(FileLockNodeManager.java:299) [artemis-server-1.5.5.008-redhat-1.jar:1.5.5.008-redhat-1]{color}|
> |{color:#000000} at org.apache.activemq.artemis.core.server.impl.FileLockNodeManager.lock(FileLockNodeManager.java:316) [artemis-server-1.5.5.008-redhat-1.jar:1.5.5.008-redhat-1]{color}|
> |{color:#000000} at org.apache.activemq.artemis.core.server.impl.FileLockNodeManager.awaitLiveNode(FileLockNodeManager.java:127) [artemis-server-1.5.5.008-redhat-1.jar:1.5.5.008-redhat-1]{color}|
> |{color:#000000} at org.apache.activemq.artemis.core.server.impl.SharedStoreBackupActivation.run(SharedStoreBackupActivation.java:77) [artemis-server-1.5.5.008-redhat-1.jar:1.5.5.008-redhat-1]{color}|
> |{color:#000000} at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$ActivationThread.run(ActiveMQServerImpl.java:2496) [artemis-server-1.5.5.008-redhat-1.jar:1.5.5.008-redhat-1]{color}|
> | |
>
> Exception is caught in {{SharedStoreBackupActivation.run}}, and causes termination of backup activation process.
> In case the NFS is reconnected later, backup server doesn't continue in activation process and it doesn't wait for live to fail. In case the live fails, backup doesn't activate, even though it has a connection to shared storage.
> Backup should retry checking live lock even in case the storage is unavailable. It should log warning/error messages that storage is unavailable, but it should not terminate the activation process. This would allow backup to continue its duties when the storage is reconnected.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)