You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by Will <wi...@sonnex.name> on 2011/07/14 11:49:03 UTC

ActiveMQ 5.5.0 broker crashing

I have a problem whereby the ActiveMQ broker crashes when my system has been
running for more than ~6 hours. By "crashes" I mean it's still running but
won't allow any sending/receiving of messages and needs to be restarted.
Here's my setup:

ActiveMQ 5.5.0 with the out-of-the-box configuration, except after it
crashed the first few times I set Xmx to 2048M and
UseDedicatedTaskRunner=false, not really sure why though.

Has around 200 (non-transacted) connections from locally networked PCs but
very minimal traffic - around two messages per second.

Runs on Ubuntu 9.04, using:
java version "1.6.0_16"
Java(TM) SE Runtime Environment (build 1.6.0_16-b01)
Java HotSpot(TM) 64-Bit Server VM (build 14.2-b01, mixed mode)


My data/activemq.log shows the following when it crashes:

2011-07-14 03:57:54,605 | INFO  | Transport failed: java.io.EOFException |
org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transpo
rt: tcp:///146.169.52.132:53248
2011-07-14 03:58:00,925 | INFO  | Transport failed: java.io.EOFException |
org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transpo
rt: tcp:///146.169.52.133:50171
2011-07-14 03:58:03,795 | INFO  | Transport failed:
java.net.SocketException: Broken pipe |
org.apache.activemq.broker.TransportConnection.Transport |
 InactivityMonitor Async Task:
java.util.concurrent.ThreadPoolExecutor$Worker@4dcd6d55
2011-07-14 03:58:07,275 | INFO  | Transport failed:
java.net.SocketException: Broken pipe |
org.apache.activemq.broker.TransportConnection.Transport |
 Async Exception Handler
2011-07-14 03:58:07,274 | INFO  | Transport failed:
java.net.SocketException: Broken pipe |
org.apache.activemq.broker.TransportConnection.Transport |
 InactivityMonitor Async Task:
java.util.concurrent.ThreadPoolExecutor$Worker@5ff907ee
2011-07-14 03:58:03,802 | INFO  | Transport failed:
java.net.SocketException: Broken pipe |
org.apache.activemq.broker.TransportConnection.Transport |
 InactivityMonitor Async Task:
java.util.concurrent.ThreadPoolExecutor$Worker@3bd1c463
2011-07-14 03:58:07,274 | INFO  | Transport failed:
java.net.SocketException: Broken pipe |
org.apache.activemq.broker.TransportConnection.Transport |
 InactivityMonitor Async Task:
java.util.concurrent.ThreadPoolExecutor$Worker@3a15e82a


etc. etc. lots more Broken pipes... then some of these interspersed with
many more broken pipes:

2011-07-14 03:58:10,201 | INFO  | Transport failed:
org.apache.activemq.transport.InactivityIOException: Cannot send, channel
has already failed: /146
.169.53.87:46663 | org.apache.activemq.broker.TransportConnection.Transport
| Async Exception Handler


Then some of these (still interspersed with broken pipes and failed channel
messages):

2011-07-14 04:18:50,633 | WARN  | Failed to unregister mbean:
org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=openwire,ViewType
=address,Name=/146.169.52.126_33912 |
org.apache.activemq.broker.jmx.ManagedTransportConnection | ActiveMQ
Task-103


Then the end is nigh with a few of these:

2011-07-14 04:19:54,077 | WARN  | Failed to remove consumer: ID:"I removed
the address here"-60248-1310591381109-0:1:-1:1. Reason:
java.lang.OutOfMemoryError
: Java heap space | org.apache.activemq.broker.TransportConnection |
ActiveMQ Task-69
java.lang.OutOfMemoryError: Java heap space
        at
java.util.concurrent.CopyOnWriteArrayList.remove(CopyOnWriteArrayList.java:481)
        at
java.util.concurrent.CopyOnWriteArraySet.remove(CopyOnWriteArraySet.java:205)
        at
org.apache.activemq.broker.jmx.ManagedRegionBroker.unregisterSubscription(ManagedRegionBroker.java:381)
        at
org.apache.activemq.broker.jmx.ManagedRegionBroker.removeConsumer(ManagedRegionBroker.java:255)
        at
org.apache.activemq.broker.BrokerFilter.removeConsumer(BrokerFilter.java:117)
        at
org.apache.activemq.advisory.AdvisoryBroker.removeConsumer(AdvisoryBroker.java:225)
        at
org.apache.activemq.broker.BrokerFilter.removeConsumer(BrokerFilter.java:117)
        at
org.apache.activemq.broker.BrokerFilter.removeConsumer(BrokerFilter.java:117)
        at
org.apache.activemq.broker.MutableBrokerFilter.removeConsumer(MutableBrokerFilter.java:123)
        at
org.apache.activemq.broker.TransportConnection.processRemoveConsumer(TransportConnection.java:575)
        at
org.apache.activemq.broker.TransportConnection.processRemoveSession(TransportConnection.java:613)
        at
org.apache.activemq.broker.TransportConnection.processRemoveConnection(TransportConnection.java:726)
        at
org.apache.activemq.broker.TransportConnection.doStop(TransportConnection.java:1025)
        at
org.apache.activemq.broker.jmx.ManagedTransportConnection.doStop(ManagedTransportConnection.java:71)
        at
org.apache.activemq.broker.TransportConnection$3.run(TransportConnection.java:952)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
2011-07-14 04:19:54,077 | WARN  | Problem scavenging sessions |
org.eclipse.jetty.util.log | HashSessionScavenger-1
java.lang.OutOfMemoryError: Java heap space


After the broker "crashes" my clients dump the following when attempting to
create a temporary queue (though "createTemporaryQueue" is just the first
method of the local "Session" that they call):

Exception in thread "Thread-4" javax.jms.JMSException: Timer already
cancelled.
        at
org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49)
        at
org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1295)
        at
org.apache.activemq.ActiveMQConnection.createTempDestination(ActiveMQConnection.java:1930)
        at
org.apache.activemq.ActiveMQSession.createTemporaryQueue(ActiveMQSession.java:1330)
        at project.hub.Client.receive(Client.scala:54)
        at project.Node$.colony(Node.scala:52)
        at
project.Node$$anonfun$main$1$$anonfun$apply$1.apply$mcV$sp(Node.scala:69)
        at project.util.Ops$$anon$1.run(Ops.scala:5)
        at java.lang.Thread.run(Thread.java:636)
Caused by: java.lang.IllegalStateException: Timer already cancelled.
        at java.util.Timer.sched(Timer.java:354)
        at java.util.Timer.schedule(Timer.java:222)
        at
org.apache.activemq.thread.Scheduler.schedualPeriodically(Scheduler.java:49)
        at org.apache.activemq.broker.region.Queue.start(Queue.java:852)
        at
org.apache.activemq.broker.region.AbstractRegion.addDestination(AbstractRegion.java:129)
        at
org.apache.activemq.broker.region.RegionBroker.addDestination(RegionBroker.java:304)
        at
org.apache.activemq.broker.region.RegionBroker.addDestinationInfo(RegionBroker.java:367)
        at
org.apache.activemq.advisory.AdvisoryBroker.addDestinationInfo(AdvisoryBroker.java:160)
        at
org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:217)
        at
org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:217)
        at
org.apache.activemq.broker.MutableBrokerFilter.addDestinationInfo(MutableBrokerFilter.java:223)
        at
org.apache.activemq.broker.TransportConnection.processAddDestination(TransportConnection.java:480)
        at
org.apache.activemq.command.DestinationInfo.visit(DestinationInfo.java:122)
        at
org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:306)
        at
org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:179)
        at
org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69)
        at
org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
        at
org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:227)
        at
org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
        at
org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:220)
        at
org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:202)
        at java.lang.Thread.run(Thread.java:619)



I also get the following from a different client thread that is blocked on
"MessageConsumer.receive":

Exception in thread "Thread-10" javax.jms.JMSException: Channel was inactive
for too (>30000) long: server/146.169.2.83:61678
        at
org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
        at
org.apache.activemq.ActiveMQMessageConsumer.dequeue(ActiveMQMessageConsumer.java:458)
        at
org.apache.activemq.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:504)
        at project.hub.Client$$anonfun$2.apply$mcV$sp(Client.scala:32)
        at project.util.Ops$$anon$1.run(Ops.scala:5)
        at java.lang.Thread.run(Thread.java:636)


Another type of broker client I have dumps these:

Exception in thread "Thread-10" javax.jms.JMSException: Cannot send, channel
has already failed: server/146.169.2.83:61678
        at
org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
        at
org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1267)
        at
org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1259)
        at
org.apache.activemq.ActiveMQSession.asyncSendPacket(ActiveMQSession.java:1863)
        at
org.apache.activemq.ActiveMQMessageProducer.close(ActiveMQMessageProducer.java:147)
        at project.hub.Server$.theoryProvider(Server.scala:114)
        at
project.hub.Server$$anonfun$main$2$$anonfun$apply$1.apply$mcV$sp(Server.scala:176)
        at project.util.Ops$$anon$1.run(Ops.scala:5)
        at java.lang.Thread.run(Thread.java:619)
Caused by: org.apache.activemq.transport.InactivityIOException: Cannot send,
channel has already failed: server/146.169.2.83:61678
        at
org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:246)
        at
org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:85)
        at
org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:104)
        at
org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
        at
org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
        at
org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1265)
        ... 7 more



Any help would be greatly appreciated.

Will

--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-5-5-0-broker-crashing-tp3667090p3667090.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ 5.5.0 broker crashing

Posted by Gary Tully <ga...@gmail.com>.
yes, setting it to 0 disables it globally. But it is disabled by
default on a destination and is typically enabled on a per destination
basis using destination policy.

On 14 July 2011 17:43, TwistedYosh <yo...@gmail.com> wrote:
> Hi Gary,
>
> I am jumping on this thread has I find that the topic somewhat relates to an
> issue I am having.  Currently I have the
> schedulePeriodForDestinationPurpge=60000.  By setting it to 0 does that stop
> the purge altogether?   If that is the case that it is opposite of what I am
> trying to accomplish.
>
> thank you,
>
> Ty
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-5-5-0-broker-crashing-tp3667090p3668103.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



-- 
http://fusesource.com
http://blog.garytully.com

Re: ActiveMQ 5.5.0 broker crashing

Posted by TwistedYosh <yo...@gmail.com>.
Hi Gary,

I am jumping on this thread has I find that the topic somewhat relates to an
issue I am having.  Currently I have the
schedulePeriodForDestinationPurpge=60000.  By setting it to 0 does that stop
the purge altogether?   If that is the case that it is opposite of what I am
trying to accomplish.

thank you,

Ty 


--
View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-5-5-0-broker-crashing-tp3667090p3668103.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: ActiveMQ 5.5.0 broker crashing

Posted by Gary Tully <ga...@gmail.com>.
it disables automatic purging of destinations that have no active
consumers/producers, it is a short circut override of the policy
entry.

http://activemq.apache.org/delete-inactive-destinations.html

On 14 July 2011 14:40, James Green <ja...@gmail.com> wrote:
> What does schedulePeriodForDestinationPurge=0 do precisely?
>
> On 14 July 2011 11:30, Gary Tully <ga...@gmail.com> wrote:
>
>> That looks like a memory leak. There is a potential issue with
>> inactive destination deletion that has been resolved on trunk that can
>> be worked around by setting schedulePeriodForDestinationPurge=0 on the
>> broker element in xml configuration.
>>
>



-- 
http://fusesource.com
http://blog.garytully.com

Re: ActiveMQ 5.5.0 broker crashing

Posted by James Green <ja...@gmail.com>.
What does schedulePeriodForDestinationPurge=0 do precisely?

On 14 July 2011 11:30, Gary Tully <ga...@gmail.com> wrote:

> That looks like a memory leak. There is a potential issue with
> inactive destination deletion that has been resolved on trunk that can
> be worked around by setting schedulePeriodForDestinationPurge=0 on the
> broker element in xml configuration.
>

Re: ActiveMQ 5.5.0 broker crashing

Posted by Gary Tully <ga...@gmail.com>.
That looks like a memory leak. There is a potential issue with
inactive destination deletion that has been resolved on trunk that can
be worked around by setting schedulePeriodForDestinationPurge=0 on the
broker element in xml configuration.

Otherwise we need to track down the component that is using up your
heap. I guess there are two approaches. Try and reduce your use or
activemq to a minimum such that you can reproduce the OOM  and raise a
jira issue with the test case or use some tools like yourkit or jmap
on the broker to figure out the culprit.

On the latter, there are some nice resource on the web like:
http://olex.openlogic.com/wazi/2009/how-to-fix-memory-leaks-in-java/

Once the JMV gets its first OOM, there will be lots of errors in the
logs, activemq is not recoverable in that state without a restart.

On 14 July 2011 10:49, Will <wi...@sonnex.name> wrote:
> I have a problem whereby the ActiveMQ broker crashes when my system has been
> running for more than ~6 hours. By "crashes" I mean it's still running but
> won't allow any sending/receiving of messages and needs to be restarted.
> Here's my setup:
>
> ActiveMQ 5.5.0 with the out-of-the-box configuration, except after it
> crashed the first few times I set Xmx to 2048M and
> UseDedicatedTaskRunner=false, not really sure why though.
>
> Has around 200 (non-transacted) connections from locally networked PCs but
> very minimal traffic - around two messages per second.
>
> Runs on Ubuntu 9.04, using:
> java version "1.6.0_16"
> Java(TM) SE Runtime Environment (build 1.6.0_16-b01)
> Java HotSpot(TM) 64-Bit Server VM (build 14.2-b01, mixed mode)
>
>
> My data/activemq.log shows the following when it crashes:
>
> 2011-07-14 03:57:54,605 | INFO  | Transport failed: java.io.EOFException |
> org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transpo
> rt: tcp:///146.169.52.132:53248
> 2011-07-14 03:58:00,925 | INFO  | Transport failed: java.io.EOFException |
> org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transpo
> rt: tcp:///146.169.52.133:50171
> 2011-07-14 03:58:03,795 | INFO  | Transport failed:
> java.net.SocketException: Broken pipe |
> org.apache.activemq.broker.TransportConnection.Transport |
>  InactivityMonitor Async Task:
> java.util.concurrent.ThreadPoolExecutor$Worker@4dcd6d55
> 2011-07-14 03:58:07,275 | INFO  | Transport failed:
> java.net.SocketException: Broken pipe |
> org.apache.activemq.broker.TransportConnection.Transport |
>  Async Exception Handler
> 2011-07-14 03:58:07,274 | INFO  | Transport failed:
> java.net.SocketException: Broken pipe |
> org.apache.activemq.broker.TransportConnection.Transport |
>  InactivityMonitor Async Task:
> java.util.concurrent.ThreadPoolExecutor$Worker@5ff907ee
> 2011-07-14 03:58:03,802 | INFO  | Transport failed:
> java.net.SocketException: Broken pipe |
> org.apache.activemq.broker.TransportConnection.Transport |
>  InactivityMonitor Async Task:
> java.util.concurrent.ThreadPoolExecutor$Worker@3bd1c463
> 2011-07-14 03:58:07,274 | INFO  | Transport failed:
> java.net.SocketException: Broken pipe |
> org.apache.activemq.broker.TransportConnection.Transport |
>  InactivityMonitor Async Task:
> java.util.concurrent.ThreadPoolExecutor$Worker@3a15e82a
>
>
> etc. etc. lots more Broken pipes... then some of these interspersed with
> many more broken pipes:
>
> 2011-07-14 03:58:10,201 | INFO  | Transport failed:
> org.apache.activemq.transport.InactivityIOException: Cannot send, channel
> has already failed: /146
> .169.53.87:46663 | org.apache.activemq.broker.TransportConnection.Transport
> | Async Exception Handler
>
>
> Then some of these (still interspersed with broken pipes and failed channel
> messages):
>
> 2011-07-14 04:18:50,633 | WARN  | Failed to unregister mbean:
> org.apache.activemq:BrokerName=localhost,Type=Connection,ConnectorName=openwire,ViewType
> =address,Name=/146.169.52.126_33912 |
> org.apache.activemq.broker.jmx.ManagedTransportConnection | ActiveMQ
> Task-103
>
>
> Then the end is nigh with a few of these:
>
> 2011-07-14 04:19:54,077 | WARN  | Failed to remove consumer: ID:"I removed
> the address here"-60248-1310591381109-0:1:-1:1. Reason:
> java.lang.OutOfMemoryError
> : Java heap space | org.apache.activemq.broker.TransportConnection |
> ActiveMQ Task-69
> java.lang.OutOfMemoryError: Java heap space
>        at
> java.util.concurrent.CopyOnWriteArrayList.remove(CopyOnWriteArrayList.java:481)
>        at
> java.util.concurrent.CopyOnWriteArraySet.remove(CopyOnWriteArraySet.java:205)
>        at
> org.apache.activemq.broker.jmx.ManagedRegionBroker.unregisterSubscription(ManagedRegionBroker.java:381)
>        at
> org.apache.activemq.broker.jmx.ManagedRegionBroker.removeConsumer(ManagedRegionBroker.java:255)
>        at
> org.apache.activemq.broker.BrokerFilter.removeConsumer(BrokerFilter.java:117)
>        at
> org.apache.activemq.advisory.AdvisoryBroker.removeConsumer(AdvisoryBroker.java:225)
>        at
> org.apache.activemq.broker.BrokerFilter.removeConsumer(BrokerFilter.java:117)
>        at
> org.apache.activemq.broker.BrokerFilter.removeConsumer(BrokerFilter.java:117)
>        at
> org.apache.activemq.broker.MutableBrokerFilter.removeConsumer(MutableBrokerFilter.java:123)
>        at
> org.apache.activemq.broker.TransportConnection.processRemoveConsumer(TransportConnection.java:575)
>        at
> org.apache.activemq.broker.TransportConnection.processRemoveSession(TransportConnection.java:613)
>        at
> org.apache.activemq.broker.TransportConnection.processRemoveConnection(TransportConnection.java:726)
>        at
> org.apache.activemq.broker.TransportConnection.doStop(TransportConnection.java:1025)
>        at
> org.apache.activemq.broker.jmx.ManagedTransportConnection.doStop(ManagedTransportConnection.java:71)
>        at
> org.apache.activemq.broker.TransportConnection$3.run(TransportConnection.java:952)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>        at java.lang.Thread.run(Thread.java:619)
> 2011-07-14 04:19:54,077 | WARN  | Problem scavenging sessions |
> org.eclipse.jetty.util.log | HashSessionScavenger-1
> java.lang.OutOfMemoryError: Java heap space
>
>
> After the broker "crashes" my clients dump the following when attempting to
> create a temporary queue (though "createTemporaryQueue" is just the first
> method of the local "Session" that they call):
>
> Exception in thread "Thread-4" javax.jms.JMSException: Timer already
> cancelled.
>        at
> org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49)
>        at
> org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1295)
>        at
> org.apache.activemq.ActiveMQConnection.createTempDestination(ActiveMQConnection.java:1930)
>        at
> org.apache.activemq.ActiveMQSession.createTemporaryQueue(ActiveMQSession.java:1330)
>        at project.hub.Client.receive(Client.scala:54)
>        at project.Node$.colony(Node.scala:52)
>        at
> project.Node$$anonfun$main$1$$anonfun$apply$1.apply$mcV$sp(Node.scala:69)
>        at project.util.Ops$$anon$1.run(Ops.scala:5)
>        at java.lang.Thread.run(Thread.java:636)
> Caused by: java.lang.IllegalStateException: Timer already cancelled.
>        at java.util.Timer.sched(Timer.java:354)
>        at java.util.Timer.schedule(Timer.java:222)
>        at
> org.apache.activemq.thread.Scheduler.schedualPeriodically(Scheduler.java:49)
>        at org.apache.activemq.broker.region.Queue.start(Queue.java:852)
>        at
> org.apache.activemq.broker.region.AbstractRegion.addDestination(AbstractRegion.java:129)
>        at
> org.apache.activemq.broker.region.RegionBroker.addDestination(RegionBroker.java:304)
>        at
> org.apache.activemq.broker.region.RegionBroker.addDestinationInfo(RegionBroker.java:367)
>        at
> org.apache.activemq.advisory.AdvisoryBroker.addDestinationInfo(AdvisoryBroker.java:160)
>        at
> org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:217)
>        at
> org.apache.activemq.broker.BrokerFilter.addDestinationInfo(BrokerFilter.java:217)
>        at
> org.apache.activemq.broker.MutableBrokerFilter.addDestinationInfo(MutableBrokerFilter.java:223)
>        at
> org.apache.activemq.broker.TransportConnection.processAddDestination(TransportConnection.java:480)
>        at
> org.apache.activemq.command.DestinationInfo.visit(DestinationInfo.java:122)
>        at
> org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:306)
>        at
> org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:179)
>        at
> org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69)
>        at
> org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
>        at
> org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:227)
>        at
> org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
>        at
> org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:220)
>        at
> org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:202)
>        at java.lang.Thread.run(Thread.java:619)
>
>
>
> I also get the following from a different client thread that is blocked on
> "MessageConsumer.receive":
>
> Exception in thread "Thread-10" javax.jms.JMSException: Channel was inactive
> for too (>30000) long: server/146.169.2.83:61678
>        at
> org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
>        at
> org.apache.activemq.ActiveMQMessageConsumer.dequeue(ActiveMQMessageConsumer.java:458)
>        at
> org.apache.activemq.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:504)
>        at project.hub.Client$$anonfun$2.apply$mcV$sp(Client.scala:32)
>        at project.util.Ops$$anon$1.run(Ops.scala:5)
>        at java.lang.Thread.run(Thread.java:636)
>
>
> Another type of broker client I have dumps these:
>
> Exception in thread "Thread-10" javax.jms.JMSException: Cannot send, channel
> has already failed: server/146.169.2.83:61678
>        at
> org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
>        at
> org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1267)
>        at
> org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1259)
>        at
> org.apache.activemq.ActiveMQSession.asyncSendPacket(ActiveMQSession.java:1863)
>        at
> org.apache.activemq.ActiveMQMessageProducer.close(ActiveMQMessageProducer.java:147)
>        at project.hub.Server$.theoryProvider(Server.scala:114)
>        at
> project.hub.Server$$anonfun$main$2$$anonfun$apply$1.apply$mcV$sp(Server.scala:176)
>        at project.util.Ops$$anon$1.run(Ops.scala:5)
>        at java.lang.Thread.run(Thread.java:619)
> Caused by: org.apache.activemq.transport.InactivityIOException: Cannot send,
> channel has already failed: server/146.169.2.83:61678
>        at
> org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:246)
>        at
> org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:85)
>        at
> org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:104)
>        at
> org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
>        at
> org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
>        at
> org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1265)
>        ... 7 more
>
>
>
> Any help would be greatly appreciated.
>
> Will
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/ActiveMQ-5-5-0-broker-crashing-tp3667090p3667090.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



-- 
http://fusesource.com
http://blog.garytully.com