You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Martin Ritchie (JIRA)" <qp...@incubator.apache.org> on 2010/01/28 12:01:35 UTC
[jira] Created: (QPID-2370) TTR-Qpid-07-NA failed due to channel
error 504 during tear down
TTR-Qpid-07-NA failed due to channel error 504 during tear down
---------------------------------------------------------------
Key: QPID-2370
URL: https://issues.apache.org/jira/browse/QPID-2370
Project: Qpid
Issue Type: Bug
Components: Java Performance Tests
Affects Versions: 0.6
Reporter: Martin Ritchie
During a full 'RunCore.sh' performance test run TTR-Qpid-07-NA hung holding the test run up. This appeared to be as a result of a Channel Error that was thrown during.
Thread-4 2010-01-26 03:29:48,722 ERROR [apache.qpid.client.AMQConnection] error:
org.apache.qpid.AMQConnectionClosedException: Error: channel is closed [error code 504: channel error] [error code 504: channel error]
This may have occured during test tear down as the following was logged by the same thread just after the stack trace.
Thread-4 2010-01-26 03:29:48,725 WARN [apache.qpid.ping.PingTestPerf] There was an exception during per thread tear down.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Updated: (QPID-2370) TTR-Qpid-07-NA failed due to channel
error 504 during tear down
Posted by "Martin Ritchie (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Martin Ritchie updated QPID-2370:
---------------------------------
Attachment: QPID-2370.CheckClosedBeforeFlow.patch
Added patch that causes a session.closed/closing check to be performed before attempting to send the ChannelFlow method.
This will not remove the race condition however, it will greatly reduce the chances of a ChannelFlow method occuring AFTER a ChannelClose has been sent.
> TTR-Qpid-07-NA failed due to channel error 504 during tear down
> ---------------------------------------------------------------
>
> Key: QPID-2370
> URL: https://issues.apache.org/jira/browse/QPID-2370
> Project: Qpid
> Issue Type: Bug
> Components: Java Performance Tests
> Affects Versions: 0.6
> Reporter: Martin Ritchie
> Assignee: Martin Ritchie
> Attachments: QPID-2370.CheckClosedBeforeFlow.patch
>
>
> During a full 'RunCore.sh' performance test run TTR-Qpid-07-NA hung holding the test run up. This appeared to be as a result of a Channel Error that was thrown during.
> Thread-4 2010-01-26 03:29:48,722 ERROR [apache.qpid.client.AMQConnection] error:
> org.apache.qpid.AMQConnectionClosedException: Error: channel is closed [error code 504: channel error] [error code 504: channel error]
> This may have occured during test tear down as the following was logged by the same thread just after the stack trace.
> Thread-4 2010-01-26 03:29:48,725 WARN [apache.qpid.ping.PingTestPerf] There was an exception during per thread tear down.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Updated: (QPID-2370) TTR-Qpid-07-NA failed due to channel
error 504 during tear down
Posted by "Martin Ritchie (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Martin Ritchie updated QPID-2370:
---------------------------------
Attachment: QPID-2370.CheckClosedBeforeFlowWithStateCheck.patch
Updated the patch to check the state of the suspension to before creating a thread. This will reduce thread creations.
> TTR-Qpid-07-NA failed due to channel error 504 during tear down
> ---------------------------------------------------------------
>
> Key: QPID-2370
> URL: https://issues.apache.org/jira/browse/QPID-2370
> Project: Qpid
> Issue Type: Bug
> Components: Java Performance Tests
> Affects Versions: 0.6
> Reporter: Martin Ritchie
> Assignee: Martin Ritchie
> Attachments: QPID-2370-Improve-BrokerDebug.patch, QPID-2370.CheckClosedBeforeFlow.patch, QPID-2370.CheckClosedBeforeFlowWithStateCheck.patch
>
>
> During a full 'RunCore.sh' performance test run TTR-Qpid-07-NA hung holding the test run up. This appeared to be as a result of a Channel Error that was thrown during.
> Thread-4 2010-01-26 03:29:48,722 ERROR [apache.qpid.client.AMQConnection] error:
> org.apache.qpid.AMQConnectionClosedException: Error: channel is closed [error code 504: channel error] [error code 504: channel error]
> This may have occured during test tear down as the following was logged by the same thread just after the stack trace.
> Thread-4 2010-01-26 03:29:48,725 WARN [apache.qpid.ping.PingTestPerf] There was an exception during per thread tear down.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Updated: (QPID-2370) TTR-Qpid-07-NA failed due to channel
error 504 during tear down
Posted by "Martin Ritchie (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Martin Ritchie updated QPID-2370:
---------------------------------
Attachment: QPID-2370-Improve-BrokerDebug.patch
Additional patch that improves the broker debug statements making diagnosis of issues easier.
> TTR-Qpid-07-NA failed due to channel error 504 during tear down
> ---------------------------------------------------------------
>
> Key: QPID-2370
> URL: https://issues.apache.org/jira/browse/QPID-2370
> Project: Qpid
> Issue Type: Bug
> Components: Java Performance Tests
> Affects Versions: 0.6
> Reporter: Martin Ritchie
> Assignee: Martin Ritchie
> Attachments: QPID-2370-Improve-BrokerDebug.patch, QPID-2370.CheckClosedBeforeFlow.patch
>
>
> During a full 'RunCore.sh' performance test run TTR-Qpid-07-NA hung holding the test run up. This appeared to be as a result of a Channel Error that was thrown during.
> Thread-4 2010-01-26 03:29:48,722 ERROR [apache.qpid.client.AMQConnection] error:
> org.apache.qpid.AMQConnectionClosedException: Error: channel is closed [error code 504: channel error] [error code 504: channel error]
> This may have occured during test tear down as the following was logged by the same thread just after the stack trace.
> Thread-4 2010-01-26 03:29:48,725 WARN [apache.qpid.ping.PingTestPerf] There was an exception during per thread tear down.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Commented: (QPID-2370) TTR-Qpid-07-NA failed due to channel
error 504 during tear down
Posted by "Martin Ritchie (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12805869#action_12805869 ]
Martin Ritchie commented on QPID-2370:
--------------------------------------
Given the number of '.'s it does make sence that this error occured at the end of the test run during the tear down. After the exception occurs the 'Unable to suspend channel' message is repeated 114 times.
Whilst killing the test run did not result in the junit test failure (note OK still logged) this is only due to the fact that the exception was not thrown out of the test case.
.................AnonymousIoService-1 2010-01-26 03:29:48,720 ERROR [qpid.client.state.AMQStateManager] No Waiters for error saving as last error:Error: channel is closed [error code 504: channel error] [error code 504: channel error]
Thread-346184 2010-01-26 03:29:48,721 WARN [apache.qpid.client.AMQSession] Unable to suspend channel
Thread-4 2010-01-26 03:29:48,722 ERROR [apache.qpid.client.AMQConnection] error:
org.apache.qpid.AMQConnectionClosedException: Error: channel is closed [error code 504: channel error] [error code 504: channel error]
at org.apache.qpid.client.handler.ConnectionCloseMethodHandler.methodReceived(ConnectionCloseMethodHandler.java:85)
at org.apache.qpid.client.handler.ClientMethodDispatcherImpl.dispatchConnectionClose(ClientMethodDispatcherImpl.java:180)
at org.apache.qpid.framing.amqp_0_9.ConnectionCloseBodyImpl.execute(ConnectionCloseBodyImpl.java:140)
at org.apache.qpid.client.state.AMQStateManager.methodReceived(AMQStateManager.java:112)
at org.apache.qpid.client.protocol.AMQProtocolHandler.methodBodyReceived(AMQProtocolHandler.java:552)
at org.apache.qpid.client.protocol.AMQProtocolSession.methodFrameReceived(AMQProtocolSession.java:533)
at org.apache.qpid.framing.AMQMethodBodyImpl.handle(AMQMethodBodyImpl.java:93)
at org.apache.qpid.client.protocol.AMQProtocolHandler.messageReceived(AMQProtocolHandler.java:519)
at org.apache.mina.common.support.AbstractIoFilterChain$TailFilter.messageReceived(AbstractIoFilterChain.java:703)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(AbstractIoFilterChain.java:362)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1200(AbstractIoFilterChain.java:54)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived(AbstractIoFilterChain.java:800)
at org.apache.mina.filter.codec.support.SimpleProtocolDecoderOutput.flush(SimpleProtocolDecoderOutput.java:60)
at org.apache.mina.filter.codec.ProtocolCodecFilter.messageReceived(ProtocolCodecFilter.java:184)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(AbstractIoFilterChain.java:362)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1200(AbstractIoFilterChain.java:54)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceived(AbstractIoFilterChain.java:800)
at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java:243)
at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(ExecutorFilter.java:305)
at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:665)
at edu.emory.mathcs.backport.java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:690)
at java.lang.Thread.run(Thread.java:619)
Thread-346182 2010-01-26 03:29:48,725 WARN [apache.qpid.client.AMQSession] Unable to suspend channel
Thread-346180 2010-01-26 03:29:48,725 WARN [apache.qpid.client.AMQSession] Unable to suspend channel
Thread-4 2010-01-26 03:29:48,725 WARN [apache.qpid.ping.PingTestPerf] There was an exception during per thread tear down.
Thread-346179 2010-01-26 03:29:48,725 WARN [apache.qpid.client.AMQSession] Unable to suspend channel
Thread-346178 2010-01-26 03:29:48,725 WARN [apache.qpid.client.AMQSession] Unable to suspend channel
Thread-346177 2010-01-26 03:29:48,725 WARN [apache.qpid.client.AMQSession] Unable to suspend channel
Thread-346176 2010-01-26 03:29:48,725 WARN [apache.qpid.client.AMQSession] Unable to suspend channel
Thread-346175 2010-01-26 03:29:48,725 WARN [apache.qpid.client.AMQSession] Unable to suspend channel
Thread-346174 2010-01-26 03:29:48,725 WARN [apache.qpid.client.AMQSession] Unable to suspend channel
Thread-346173 2010-01-26 03:29:48,726 WARN [apache.qpid.client.AMQSession] Unable to suspend channel
Thread-346172 2010-01-26 03:29:48,726 WARN [apache.qpid.client.AMQSession] Unable to suspend channel
Thread-346171 2010-01-26 03:29:48,726 WARN [apache.qpid.client.AMQSession] Unable to suspend channel
Thread-346170 2010-01-26 03:29:48,726 WARN [apache.qpid.client.AMQSession] Unable to suspend channel
Thread-346169 2010-01-26 03:29:48,726 WARN [apache.qpid.client.AMQSession] Unable to suspend channel
Thread-346181 2010-01-26 03:29:48,724 WARN [apache.qpid.client.AMQSession] Unable to suspend channel
Time: 3,601.185
OK (29307 tests)
Thread-346183 2010-01-26 03:29:48,724 WARN [apache.qpid.client.AMQSession] Unable to suspend channel
Thread-346168 2010-01-26 03:29:48,727 WARN [apache.qpid.client.AMQSession] Unable to suspend channel
> TTR-Qpid-07-NA failed due to channel error 504 during tear down
> ---------------------------------------------------------------
>
> Key: QPID-2370
> URL: https://issues.apache.org/jira/browse/QPID-2370
> Project: Qpid
> Issue Type: Bug
> Components: Java Performance Tests
> Affects Versions: 0.6
> Reporter: Martin Ritchie
>
> During a full 'RunCore.sh' performance test run TTR-Qpid-07-NA hung holding the test run up. This appeared to be as a result of a Channel Error that was thrown during.
> Thread-4 2010-01-26 03:29:48,722 ERROR [apache.qpid.client.AMQConnection] error:
> org.apache.qpid.AMQConnectionClosedException: Error: channel is closed [error code 504: channel error] [error code 504: channel error]
> This may have occured during test tear down as the following was logged by the same thread just after the stack trace.
> Thread-4 2010-01-26 03:29:48,725 WARN [apache.qpid.ping.PingTestPerf] There was an exception during per thread tear down.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Assigned: (QPID-2370) TTR-Qpid-07-NA failed due to channel
error 504 during tear down
Posted by "Martin Ritchie (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Martin Ritchie reassigned QPID-2370:
------------------------------------
Assignee: Martin Ritchie
> TTR-Qpid-07-NA failed due to channel error 504 during tear down
> ---------------------------------------------------------------
>
> Key: QPID-2370
> URL: https://issues.apache.org/jira/browse/QPID-2370
> Project: Qpid
> Issue Type: Bug
> Components: Java Performance Tests
> Affects Versions: 0.6
> Reporter: Martin Ritchie
> Assignee: Martin Ritchie
>
> During a full 'RunCore.sh' performance test run TTR-Qpid-07-NA hung holding the test run up. This appeared to be as a result of a Channel Error that was thrown during.
> Thread-4 2010-01-26 03:29:48,722 ERROR [apache.qpid.client.AMQConnection] error:
> org.apache.qpid.AMQConnectionClosedException: Error: channel is closed [error code 504: channel error] [error code 504: channel error]
> This may have occured during test tear down as the following was logged by the same thread just after the stack trace.
> Thread-4 2010-01-26 03:29:48,725 WARN [apache.qpid.ping.PingTestPerf] There was an exception during per thread tear down.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Updated: (QPID-2370) TTR-Qpid-07-NA failed due to channel
error 504 during tear down
Posted by "Martin Ritchie (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Martin Ritchie updated QPID-2370:
---------------------------------
Attachment: (was: QPID-2370.CheckClosedBeforeFlow.patch)
> TTR-Qpid-07-NA failed due to channel error 504 during tear down
> ---------------------------------------------------------------
>
> Key: QPID-2370
> URL: https://issues.apache.org/jira/browse/QPID-2370
> Project: Qpid
> Issue Type: Bug
> Components: Java Performance Tests
> Affects Versions: 0.6
> Reporter: Martin Ritchie
> Assignee: Martin Ritchie
> Attachments: QPID-2370.CheckClosedBeforeFlow.patch
>
>
> During a full 'RunCore.sh' performance test run TTR-Qpid-07-NA hung holding the test run up. This appeared to be as a result of a Channel Error that was thrown during.
> Thread-4 2010-01-26 03:29:48,722 ERROR [apache.qpid.client.AMQConnection] error:
> org.apache.qpid.AMQConnectionClosedException: Error: channel is closed [error code 504: channel error] [error code 504: channel error]
> This may have occured during test tear down as the following was logged by the same thread just after the stack trace.
> Thread-4 2010-01-26 03:29:48,725 WARN [apache.qpid.ping.PingTestPerf] There was an exception during per thread tear down.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Updated: (QPID-2370) TTR-Qpid-07-NA failed due to channel
error 504 during tear down
Posted by "Martin Ritchie (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Martin Ritchie updated QPID-2370:
---------------------------------
Attachment: QPID-2370.CheckClosedBeforeFlow.patch
I did mean to grant permission
> TTR-Qpid-07-NA failed due to channel error 504 during tear down
> ---------------------------------------------------------------
>
> Key: QPID-2370
> URL: https://issues.apache.org/jira/browse/QPID-2370
> Project: Qpid
> Issue Type: Bug
> Components: Java Performance Tests
> Affects Versions: 0.6
> Reporter: Martin Ritchie
> Assignee: Martin Ritchie
> Attachments: QPID-2370.CheckClosedBeforeFlow.patch
>
>
> During a full 'RunCore.sh' performance test run TTR-Qpid-07-NA hung holding the test run up. This appeared to be as a result of a Channel Error that was thrown during.
> Thread-4 2010-01-26 03:29:48,722 ERROR [apache.qpid.client.AMQConnection] error:
> org.apache.qpid.AMQConnectionClosedException: Error: channel is closed [error code 504: channel error] [error code 504: channel error]
> This may have occured during test tear down as the following was logged by the same thread just after the stack trace.
> Thread-4 2010-01-26 03:29:48,725 WARN [apache.qpid.ping.PingTestPerf] There was an exception during per thread tear down.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Assigned: (QPID-2370) TTR-Qpid-07-NA failed due to channel
error 504 during tear down
Posted by "Martin Ritchie (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Martin Ritchie reassigned QPID-2370:
------------------------------------
Assignee: Robbie Gemmell (was: Martin Ritchie)
Hi Robbie can you review these changes please. I commited the Broker and the client change (withStateCheck).
Cheers.
> TTR-Qpid-07-NA failed due to channel error 504 during tear down
> ---------------------------------------------------------------
>
> Key: QPID-2370
> URL: https://issues.apache.org/jira/browse/QPID-2370
> Project: Qpid
> Issue Type: Bug
> Components: Java Performance Tests
> Affects Versions: 0.6
> Reporter: Martin Ritchie
> Assignee: Robbie Gemmell
> Attachments: QPID-2370-Improve-BrokerDebug.patch, QPID-2370.CheckClosedBeforeFlow.patch, QPID-2370.CheckClosedBeforeFlowWithStateCheck.patch
>
>
> During a full 'RunCore.sh' performance test run TTR-Qpid-07-NA hung holding the test run up. This appeared to be as a result of a Channel Error that was thrown during.
> Thread-4 2010-01-26 03:29:48,722 ERROR [apache.qpid.client.AMQConnection] error:
> org.apache.qpid.AMQConnectionClosedException: Error: channel is closed [error code 504: channel error] [error code 504: channel error]
> This may have occured during test tear down as the following was logged by the same thread just after the stack trace.
> Thread-4 2010-01-26 03:29:48,725 WARN [apache.qpid.ping.PingTestPerf] There was an exception during per thread tear down.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org
[jira] Updated: (QPID-2370) TTR-Qpid-07-NA failed due to channel
error 504 during tear down
Posted by "Martin Ritchie (JIRA)" <qp...@incubator.apache.org>.
[ https://issues.apache.org/jira/browse/QPID-2370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Martin Ritchie updated QPID-2370:
---------------------------------
Status: Ready To Review (was: In Progress)
> TTR-Qpid-07-NA failed due to channel error 504 during tear down
> ---------------------------------------------------------------
>
> Key: QPID-2370
> URL: https://issues.apache.org/jira/browse/QPID-2370
> Project: Qpid
> Issue Type: Bug
> Components: Java Performance Tests
> Affects Versions: 0.6
> Reporter: Martin Ritchie
> Assignee: Martin Ritchie
> Attachments: QPID-2370-Improve-BrokerDebug.patch, QPID-2370.CheckClosedBeforeFlow.patch, QPID-2370.CheckClosedBeforeFlowWithStateCheck.patch
>
>
> During a full 'RunCore.sh' performance test run TTR-Qpid-07-NA hung holding the test run up. This appeared to be as a result of a Channel Error that was thrown during.
> Thread-4 2010-01-26 03:29:48,722 ERROR [apache.qpid.client.AMQConnection] error:
> org.apache.qpid.AMQConnectionClosedException: Error: channel is closed [error code 504: channel error] [error code 504: channel error]
> This may have occured during test tear down as the following was logged by the same thread just after the stack trace.
> Thread-4 2010-01-26 03:29:48,725 WARN [apache.qpid.ping.PingTestPerf] There was an exception during per thread tear down.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org