You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Alan Conway (JIRA)" <ji...@apache.org> on 2012/07/17 19:35:34 UTC

[jira] [Created] (QPID-4144) HA broker deadlocks on broeker::QueueRegistry lock and ha::Primary lock

Alan Conway created QPID-4144:
---------------------------------

             Summary: HA broker deadlocks on broeker::QueueRegistry lock and ha::Primary lock
                 Key: QPID-4144
                 URL: https://issues.apache.org/jira/browse/QPID-4144
             Project: Qpid
          Issue Type: Bug
          Components: C++ Clustering
    Affects Versions: 0.17
            Reporter: Alan Conway
            Assignee: Alan Conway
             Fix For: 0.18
         Attachments: thread-dump.txt

Running tests repeatedly, the broker deadlocked with the attached stack trace.

The problem call sequences are:
1. QueueRegistry::destroy takes QueuerRegistry lock > ConfigurationObserver::queueDestroy > ha::Primary::queueDestroy takes Primary lock.
2. ConnectionObserver::opened cals Primary::opened lock> RemoteBackup>getQueues().eachQueue

This patch breaks the deadlock at both ends: QueueRegistry no longer holds the lock across the observer call and Primary does not hold the lock across eachQueue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


[jira] [Commented] (QPID-4144) HA broker deadlocks on broeker::QueueRegistry lock and ha::Primary lock

Posted by "Gordon Sim (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/QPID-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13418366#comment-13418366 ] 

Gordon Sim commented on QPID-4144:
----------------------------------

Looks good to me; I would approve of merging this onto 0.18 release branch.
                
> HA broker deadlocks on broeker::QueueRegistry lock and ha::Primary lock
> -----------------------------------------------------------------------
>
>                 Key: QPID-4144
>                 URL: https://issues.apache.org/jira/browse/QPID-4144
>             Project: Qpid
>          Issue Type: Bug
>          Components: C++ Clustering
>    Affects Versions: 0.17
>            Reporter: Alan Conway
>            Assignee: Alan Conway
>             Fix For: 0.18
>
>         Attachments: thread-dump.txt
>
>
> Running tests repeatedly, the broker deadlocked with the attached stack trace.
> The problem call sequences are:
> 1. QueueRegistry::destroy takes QueuerRegistry lock > ConfigurationObserver::queueDestroy > ha::Primary::queueDestroy takes Primary lock.
> 2. ConnectionObserver::opened cals Primary::opened lock> RemoteBackup>getQueues().eachQueue
> This patch breaks the deadlock at both ends: QueueRegistry no longer holds the lock across the observer call and Primary does not hold the lock across eachQueue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


[jira] [Commented] (QPID-4144) HA broker deadlocks on broeker::QueueRegistry lock and ha::Primary lock

Posted by "Justin Ross (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/QPID-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13418461#comment-13418461 ] 

Justin Ross commented on QPID-4144:
-----------------------------------

Reviewed by Gordon.  Approved for 0.18.
                
> HA broker deadlocks on broeker::QueueRegistry lock and ha::Primary lock
> -----------------------------------------------------------------------
>
>                 Key: QPID-4144
>                 URL: https://issues.apache.org/jira/browse/QPID-4144
>             Project: Qpid
>          Issue Type: Bug
>          Components: C++ Clustering
>    Affects Versions: 0.17
>            Reporter: Alan Conway
>            Assignee: Alan Conway
>             Fix For: 0.18
>
>         Attachments: thread-dump.txt
>
>
> Running tests repeatedly, the broker deadlocked with the attached stack trace.
> The problem call sequences are:
> 1. QueueRegistry::destroy takes QueuerRegistry lock > ConfigurationObserver::queueDestroy > ha::Primary::queueDestroy takes Primary lock.
> 2. ConnectionObserver::opened cals Primary::opened lock> RemoteBackup>getQueues().eachQueue
> This patch breaks the deadlock at both ends: QueueRegistry no longer holds the lock across the observer call and Primary does not hold the lock across eachQueue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


[jira] [Updated] (QPID-4144) HA broker deadlocks on broeker::QueueRegistry lock and ha::Primary lock

Posted by "Alan Conway (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/QPID-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alan Conway updated QPID-4144:
------------------------------

    Attachment: thread-dump.txt

Thread dump of the deadlocked process.
                
> HA broker deadlocks on broeker::QueueRegistry lock and ha::Primary lock
> -----------------------------------------------------------------------
>
>                 Key: QPID-4144
>                 URL: https://issues.apache.org/jira/browse/QPID-4144
>             Project: Qpid
>          Issue Type: Bug
>          Components: C++ Clustering
>    Affects Versions: 0.17
>            Reporter: Alan Conway
>            Assignee: Alan Conway
>             Fix For: 0.18
>
>         Attachments: thread-dump.txt
>
>
> Running tests repeatedly, the broker deadlocked with the attached stack trace.
> The problem call sequences are:
> 1. QueueRegistry::destroy takes QueuerRegistry lock > ConfigurationObserver::queueDestroy > ha::Primary::queueDestroy takes Primary lock.
> 2. ConnectionObserver::opened cals Primary::opened lock> RemoteBackup>getQueues().eachQueue
> This patch breaks the deadlock at both ends: QueueRegistry no longer holds the lock across the observer call and Primary does not hold the lock across eachQueue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


[jira] [Commented] (QPID-4144) HA broker deadlocks on broeker::QueueRegistry lock and ha::Primary lock

Posted by "Alan Conway (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/QPID-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416393#comment-13416393 ] 

Alan Conway commented on QPID-4144:
-----------------------------------

Fixed on trunk, request merge to 0.18 branch.

------------------------------------------------------------------------
r1362584 | aconway | 2012-07-17 13:39:54 -0400 (Tue, 17 Jul 2012) | 10 lines

QPID-4144 HA broker deadlocks on broker::QueueRegistry lock and ha::Primary lock

Running tests repeatedly, the broker deadlocked with the attached stack trace.

The problem call sequences are:
1. QueueRegistry::destroy takes QueuerRegistry lock > ConfigurationObserver::queueDestroy > ha::Primary::queueDestroy takes Primary lock.
2. ConnectionObserver::opened cals Primary::opened lock> RemoteBackup>getQueues().eachQueue

This patch breaks the deadlock at both ends: QueueRegistry no longer holds the lock across the observer call and Primary does not hold the lock across eachQueue.

------------------------------------------------------------------------
                
> HA broker deadlocks on broeker::QueueRegistry lock and ha::Primary lock
> -----------------------------------------------------------------------
>
>                 Key: QPID-4144
>                 URL: https://issues.apache.org/jira/browse/QPID-4144
>             Project: Qpid
>          Issue Type: Bug
>          Components: C++ Clustering
>    Affects Versions: 0.17
>            Reporter: Alan Conway
>            Assignee: Alan Conway
>             Fix For: 0.18
>
>         Attachments: thread-dump.txt
>
>
> Running tests repeatedly, the broker deadlocked with the attached stack trace.
> The problem call sequences are:
> 1. QueueRegistry::destroy takes QueuerRegistry lock > ConfigurationObserver::queueDestroy > ha::Primary::queueDestroy takes Primary lock.
> 2. ConnectionObserver::opened cals Primary::opened lock> RemoteBackup>getQueues().eachQueue
> This patch breaks the deadlock at both ends: QueueRegistry no longer holds the lock across the observer call and Primary does not hold the lock across eachQueue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org


[jira] [Resolved] (QPID-4144) HA broker deadlocks on broeker::QueueRegistry lock and ha::Primary lock

Posted by "Alan Conway (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/QPID-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alan Conway resolved QPID-4144.
-------------------------------

    Resolution: Fixed

On 0.18 branch:
------------------------------------------------------------------------
r1363486 | aconway | 2012-07-19 15:16:12 -0400 (Thu, 19 Jul 2012) | 10 lines

QPID-4144 HA broker deadlocks on broker::QueueRegistry lock and ha::Primary lock

Running tests repeatedly, the broker deadlocked with the attached stack trace.

The problem call sequences are:
1. QueueRegistry::destroy takes QueuerRegistry lock > ConfigurationObserver::queueDestroy > ha::Primary::queueDestroy takes Primary lock.
2. ConnectionObserver::opened cals Primary::opened lock> RemoteBackup>getQueues().eachQueue

This patch breaks the deadlock at both ends: QueueRegistry no longer holds the lock across the observer call and Primary does not hold the lock across eachQueue.

                
> HA broker deadlocks on broeker::QueueRegistry lock and ha::Primary lock
> -----------------------------------------------------------------------
>
>                 Key: QPID-4144
>                 URL: https://issues.apache.org/jira/browse/QPID-4144
>             Project: Qpid
>          Issue Type: Bug
>          Components: C++ Clustering
>    Affects Versions: 0.17
>            Reporter: Alan Conway
>            Assignee: Alan Conway
>             Fix For: 0.18
>
>         Attachments: thread-dump.txt
>
>
> Running tests repeatedly, the broker deadlocked with the attached stack trace.
> The problem call sequences are:
> 1. QueueRegistry::destroy takes QueuerRegistry lock > ConfigurationObserver::queueDestroy > ha::Primary::queueDestroy takes Primary lock.
> 2. ConnectionObserver::opened cals Primary::opened lock> RemoteBackup>getQueues().eachQueue
> This patch breaks the deadlock at both ends: QueueRegistry no longer holds the lock across the observer call and Primary does not hold the lock across eachQueue.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org