[jira] [Created] (AMQ-6048) ActiveMQ broker service hanged after re-establishing the connection with another ActiveMQ

["Anas Abu Shawish (JIRA)" <ji...@apache.org>]

Anas Abu Shawish created AMQ-6048:
-------------------------------------

             Summary: ActiveMQ broker service hanged after re-establishing the connection with another ActiveMQ
                 Key: AMQ-6048
                 URL: https://issues.apache.org/jira/browse/AMQ-6048
             Project: ActiveMQ
          Issue Type: Bug
    Affects Versions: 5.11.0
         Environment: Windows server 2003
            Reporter: Anas Abu Shawish


We've encountered this incident on ActiveMQ broker wrapper logs where a disconnections happened between the broker and other ActiveMQ service. The network disconnection happened for almost a day and then an outofMemory exception started to appear. Then the connection was re-established and with the other MQ service and broker started to process one file then it hanged and no logs were found afterwards until ActiveMQ broker windows service restarted.

A snap of the error appeared in wrapper.log file :

INFO   | jvm 1    | 2015/10/30 07:58:39 | Exception in thread "ActiveMQ Failover Worker: 23334009" java.lang.OutOfMemoryError: unable to create new native thread
INFO   | jvm 1    | 2015/10/30 07:58:39 | 	at java.lang.Thread.start0(Native Method)
INFO   | jvm 1    | 2015/10/30 07:58:39 | 	at java.lang.Thread.start(Thread.java:714)
INFO   | jvm 1    | 2015/10/30 07:58:39 | 	at org.apache.activemq.thread.TaskRunnerFactory.doExecuteNewThread(TaskRunnerFactory.java:165)
INFO   | jvm 1    | 2015/10/30 07:58:39 | 	at org.apache.activemq.thread.TaskRunnerFactory.execute(TaskRunnerFactory.java:155)
INFO   | jvm 1    | 2015/10/30 07:58:39 | 	at org.apache.activemq.thread.TaskRunnerFactory.execute(TaskRunnerFactory.java:146)
INFO   | jvm 1    | 2015/10/30 07:58:39 | 	at org.apache.activemq.transport.tcp.TcpTransport.doStop(TcpTransport.java:538)
INFO   | jvm 1    | 2015/10/30 07:58:39 | 	at org.apache.activemq.transport.nio.NIOTransport.doStop(NIOTransport.java:166)
INFO   | jvm 1    | 2015/10/30 07:58:39 | 	at org.apache.activemq.util.ServiceSupport.stop(ServiceSupport.java:71)
INFO   | jvm 1    | 2015/10/30 07:58:39 | 	at org.apache.activemq.transport.tcp.TcpTransport.stop(TcpTransport.java:582)
INFO   | jvm 1    | 2015/10/30 07:58:39 | 	at org.apache.activemq.transport.AbstractInactivityMonitor.stop(AbstractInactivityMonitor.java:145)
INFO   | jvm 1    | 2015/10/30 07:58:39 | 	at org.apache.activemq.transport.TransportFilter.stop(TransportFilter.java:65)
INFO   | jvm 1    | 2015/10/30 07:58:39 | 	at org.apache.activemq.transport.WireFormatNegotiator.stop(WireFormatNegotiator.java:91)
INFO   | jvm 1    | 2015/10/30 07:58:39 | 	at org.apache.activemq.transport.failover.FailoverTransport.doReconnect(FailoverTransport.java:1079)
INFO   | jvm 1    | 2015/10/30 07:58:39 | 	at org.apache.activemq.transport.failover.FailoverTransport$2.iterate(FailoverTransport.java:149)
INFO   | jvm 1    | 2015/10/30 07:58:39 | 	at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:112)
INFO   | jvm 1    | 2015/10/30 07:58:39 | 	at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:42)
INFO   | jvm 1    | 2015/10/30 07:59:39 |  INFO | Successfully reconnected to nio://wps-server1.systems.qa.hsbc:61999
INFO   | jvm 1    | 2015/10/31 07:01:13 |  INFO | *** Compressing SIF file : 'SIF_10370600_HSB_20151031_0701.CSV'
INFO   | jvm 1    | 2015/10/31 07:01:14 |  INFO | *** Sending SIF file...

Appreciate if anyone can help in identifying the root cause of the issue.

Regards, 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)