You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Weston M. Price (Created) (JIRA)" <ji...@apache.org> on 2011/12/21 00:51:30 UTC
[jira] [Created] (QPID-3705) Thread Deadlock in QpidActivation
Thread Deadlock in QpidActivation
---------------------------------
Key: QPID-3705
URL: https://issues.apache.org/jira/browse/QPID-3705
Project: Qpid
Issue Type: Bug
Components: JCA
Affects Versions: 0.15
Environment: All OS platforms, all supported application server platforms for JCA adapter.
Reporter: Weston M. Price
Fix For: 0.15
There is a deadlock condition in the Qpid JCA adapter. For inbound messaging,
endpoint activation is handled by the QpidActivation.SetupActivation class.
This class is a Work instance and is submitted to the JCA WorkManager, as such
it is run in a separate thread.
In the case where the initial activation call blocks (e.g. invalid durable
subscription names), the IOReceiver thread receives an exception prior to the
QpidActivation class every being property initialized. On an exception we
attempt to teardown and setup the activation again. However, at this point the
QpidActivation instance is locked by the JCA Work manager, as such, the
IOReceiver thread (or any other that received the exception first), can't make
forward progress because the QpidActivation class has been locked. Both the
setup() and teardown() methods are synchronized and prohibit forward progress
since the JCA Work Manager already has a lock on the instance.
Steps to Reproduce:
1. Deploy the Qpid JCA adapter
2. Create a Topic Destination via *-ds.xml file.
3. Deploy an MDB, using a durable subscription with a name that does not exist.
Actual results:
JBoss EAP (and Geronimo) locks up and as such, does not allow a graceful shutdown without
resorting to kill - <pid> or other extraordinary measures. Run jstack <pid> to
review the deadlock condition.
The QpidActivation class contains a a SetupActivation class that is only
responsible for setting up the initial connection/session etc. It shares the
same code with the QpidActivation as a whole most notably the onException(),
handleFailure() code. If the initial setup fails, or blocks, other threads
receive the onException() invocation and attempt to teardown() and setup() the
activation again. Since the instance is blocked, a deadlock occurs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Assigned] (QPID-3705) Thread Deadlock in QpidActivation
Posted by "Weston M. Price (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Weston M. Price reassigned QPID-3705:
-------------------------------------
Assignee: Weston M. Price
> Thread Deadlock in QpidActivation
> ---------------------------------
>
> Key: QPID-3705
> URL: https://issues.apache.org/jira/browse/QPID-3705
> Project: Qpid
> Issue Type: Bug
> Components: JCA
> Affects Versions: 0.15
> Environment: All OS platforms, all supported application server platforms for JCA adapter.
> Reporter: Weston M. Price
> Assignee: Weston M. Price
> Fix For: 0.15
>
> Attachments: QPID-3705-stacktrace.txt, QPID-3705.patch
>
>
> There is a deadlock condition in the Qpid JCA adapter. For inbound messaging,
> endpoint activation is handled by the QpidActivation.SetupActivation class.
> This class is a Work instance and is submitted to the JCA WorkManager, as such
> it is run in a separate thread.
> In the case where the initial activation call blocks (e.g. invalid durable
> subscription names), the IOReceiver thread receives an exception prior to the
> QpidActivation class every being property initialized. On an exception we
> attempt to teardown and setup the activation again. However, at this point the
> QpidActivation instance is locked by the JCA Work manager, as such, the
> IOReceiver thread (or any other that received the exception first), can't make
> forward progress because the QpidActivation class has been locked. Both the
> setup() and teardown() methods are synchronized and prohibit forward progress
> since the JCA Work Manager already has a lock on the instance.
> Steps to Reproduce:
> 1. Deploy the Qpid JCA adapter
> 2. Create a Topic Destination via *-ds.xml file.
> 3. Deploy an MDB, using a durable subscription with a name that does not exist.
> Actual results:
> JBoss EAP (and Geronimo) locks up and as such, does not allow a graceful shutdown without
> resorting to kill - <pid> or other extraordinary measures. Run jstack <pid> to
> review the deadlock condition.
> The QpidActivation class contains a a SetupActivation class that is only
> responsible for setting up the initial connection/session etc. It shares the
> same code with the QpidActivation as a whole most notably the onException(),
> handleFailure() code. If the initial setup fails, or blocks, other threads
> receive the onException() invocation and attempt to teardown() and setup() the
> activation again. Since the instance is blocked, a deadlock occurs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Updated] (QPID-3705) Thread Deadlock in QpidActivation
Posted by "Weston M. Price (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Weston M. Price updated QPID-3705:
----------------------------------
Attachment: QPID-3705.patch
Patch for QPID-3705
> Thread Deadlock in QpidActivation
> ---------------------------------
>
> Key: QPID-3705
> URL: https://issues.apache.org/jira/browse/QPID-3705
> Project: Qpid
> Issue Type: Bug
> Components: JCA
> Affects Versions: 0.15
> Environment: All OS platforms, all supported application server platforms for JCA adapter.
> Reporter: Weston M. Price
> Fix For: 0.15
>
> Attachments: QPID-3705-stacktrace.txt, QPID-3705.patch
>
>
> There is a deadlock condition in the Qpid JCA adapter. For inbound messaging,
> endpoint activation is handled by the QpidActivation.SetupActivation class.
> This class is a Work instance and is submitted to the JCA WorkManager, as such
> it is run in a separate thread.
> In the case where the initial activation call blocks (e.g. invalid durable
> subscription names), the IOReceiver thread receives an exception prior to the
> QpidActivation class every being property initialized. On an exception we
> attempt to teardown and setup the activation again. However, at this point the
> QpidActivation instance is locked by the JCA Work manager, as such, the
> IOReceiver thread (or any other that received the exception first), can't make
> forward progress because the QpidActivation class has been locked. Both the
> setup() and teardown() methods are synchronized and prohibit forward progress
> since the JCA Work Manager already has a lock on the instance.
> Steps to Reproduce:
> 1. Deploy the Qpid JCA adapter
> 2. Create a Topic Destination via *-ds.xml file.
> 3. Deploy an MDB, using a durable subscription with a name that does not exist.
> Actual results:
> JBoss EAP (and Geronimo) locks up and as such, does not allow a graceful shutdown without
> resorting to kill - <pid> or other extraordinary measures. Run jstack <pid> to
> review the deadlock condition.
> The QpidActivation class contains a a SetupActivation class that is only
> responsible for setting up the initial connection/session etc. It shares the
> same code with the QpidActivation as a whole most notably the onException(),
> handleFailure() code. If the initial setup fails, or blocks, other threads
> receive the onException() invocation and attempt to teardown() and setup() the
> activation again. Since the instance is blocked, a deadlock occurs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Updated] (QPID-3705) Thread Deadlock in QpidActivation
Posted by "Weston M. Price (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Weston M. Price updated QPID-3705:
----------------------------------
Attachment: QPID-3705-stacktrace.txt
JStack run showing thread deadlock in QpidActivation
> Thread Deadlock in QpidActivation
> ---------------------------------
>
> Key: QPID-3705
> URL: https://issues.apache.org/jira/browse/QPID-3705
> Project: Qpid
> Issue Type: Bug
> Components: JCA
> Affects Versions: 0.15
> Environment: All OS platforms, all supported application server platforms for JCA adapter.
> Reporter: Weston M. Price
> Fix For: 0.15
>
> Attachments: QPID-3705-stacktrace.txt
>
>
> There is a deadlock condition in the Qpid JCA adapter. For inbound messaging,
> endpoint activation is handled by the QpidActivation.SetupActivation class.
> This class is a Work instance and is submitted to the JCA WorkManager, as such
> it is run in a separate thread.
> In the case where the initial activation call blocks (e.g. invalid durable
> subscription names), the IOReceiver thread receives an exception prior to the
> QpidActivation class every being property initialized. On an exception we
> attempt to teardown and setup the activation again. However, at this point the
> QpidActivation instance is locked by the JCA Work manager, as such, the
> IOReceiver thread (or any other that received the exception first), can't make
> forward progress because the QpidActivation class has been locked. Both the
> setup() and teardown() methods are synchronized and prohibit forward progress
> since the JCA Work Manager already has a lock on the instance.
> Steps to Reproduce:
> 1. Deploy the Qpid JCA adapter
> 2. Create a Topic Destination via *-ds.xml file.
> 3. Deploy an MDB, using a durable subscription with a name that does not exist.
> Actual results:
> JBoss EAP (and Geronimo) locks up and as such, does not allow a graceful shutdown without
> resorting to kill - <pid> or other extraordinary measures. Run jstack <pid> to
> review the deadlock condition.
> The QpidActivation class contains a a SetupActivation class that is only
> responsible for setting up the initial connection/session etc. It shares the
> same code with the QpidActivation as a whole most notably the onException(),
> handleFailure() code. If the initial setup fails, or blocks, other threads
> receive the onException() invocation and attempt to teardown() and setup() the
> activation again. Since the instance is blocked, a deadlock occurs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] [Resolved] (QPID-3705) Thread Deadlock in QpidActivation
Posted by "Weston M. Price (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/QPID-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Weston M. Price resolved QPID-3705.
-----------------------------------
Resolution: Fixed
Addressed with attached patch. Fixes deadlock in QpidActivation.
> Thread Deadlock in QpidActivation
> ---------------------------------
>
> Key: QPID-3705
> URL: https://issues.apache.org/jira/browse/QPID-3705
> Project: Qpid
> Issue Type: Bug
> Components: JCA
> Affects Versions: 0.15
> Environment: All OS platforms, all supported application server platforms for JCA adapter.
> Reporter: Weston M. Price
> Assignee: Weston M. Price
> Fix For: 0.15
>
> Attachments: QPID-3705-stacktrace.txt, QPID-3705.patch
>
>
> There is a deadlock condition in the Qpid JCA adapter. For inbound messaging,
> endpoint activation is handled by the QpidActivation.SetupActivation class.
> This class is a Work instance and is submitted to the JCA WorkManager, as such
> it is run in a separate thread.
> In the case where the initial activation call blocks (e.g. invalid durable
> subscription names), the IOReceiver thread receives an exception prior to the
> QpidActivation class every being property initialized. On an exception we
> attempt to teardown and setup the activation again. However, at this point the
> QpidActivation instance is locked by the JCA Work manager, as such, the
> IOReceiver thread (or any other that received the exception first), can't make
> forward progress because the QpidActivation class has been locked. Both the
> setup() and teardown() methods are synchronized and prohibit forward progress
> since the JCA Work Manager already has a lock on the instance.
> Steps to Reproduce:
> 1. Deploy the Qpid JCA adapter
> 2. Create a Topic Destination via *-ds.xml file.
> 3. Deploy an MDB, using a durable subscription with a name that does not exist.
> Actual results:
> JBoss EAP (and Geronimo) locks up and as such, does not allow a graceful shutdown without
> resorting to kill - <pid> or other extraordinary measures. Run jstack <pid> to
> review the deadlock condition.
> The QpidActivation class contains a a SetupActivation class that is only
> responsible for setting up the initial connection/session etc. It shares the
> same code with the QpidActivation as a whole most notably the onException(),
> handleFailure() code. If the initial setup fails, or blocks, other threads
> receive the onException() invocation and attempt to teardown() and setup() the
> activation again. Since the instance is blocked, a deadlock occurs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org