You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Erich Liebmann (JIRA)" <ji...@apache.org> on 2010/05/17 08:33:04 UTC
[jira] Created: (AMQ-2734) VMTransport fails with IOException
VMTransport fails with IOException
----------------------------------
Key: AMQ-2734
URL: https://issues.apache.org/activemq/browse/AMQ-2734
Project: ActiveMQ
Issue Type: Bug
Components: Broker
Affects Versions: 5.3.0
Environment: OS: Linux / 2.6.9-67.ELsmp / i386 / 8
JVM: Java HotSpot(TM) Server VM / Sun Microsystems Inc. / 1.6.0_17-b04
Reporter: Erich Liebmann
We are able to successfully publish a large number of messages to an embedded ActiveMQ instance via the VM protocol/transport. After a while, however, further publishing starts to fail with IOExceptions (caused by InterruptedExceptions). For some (unknown) reason the invocation of LinkedBlockingQueue#put() in the VMTransport#oneway() method starts to fail with InterruptedExceptions. Please find the full stack trace below.
Since we use the VM transport together with the FAILOVER transport the connection is recovered automatically (a fraction of a second later). Our connection URL currently is as follows:
failover:(vm://amqmasterbroker?create=false&waitForStart=10000)?initialReconnectDelay=100&maxReconnectAttempts=5
We intend to set the transport parameters backup=true&timeout=100 on the FAILOVER protocol to work around the problem of stuck publishing threads and message publishing errors in case of the IOException (not sure if this will help) but would want see the root-cause to be fixed by ActiveMQ as this issue is causing a lot of troubles for us in production.
2010-05-14 18:19:13,950 [EventServiceChannel$ReceiverThread[daemon]_357] DEBUG failover.FailoverTransport - Send oneway attempt: 0 failed for command:ActiveMQObjectMessage {commandId = 1929587556, responseRequired = false, messageId = ID:saturntstsgp-40467-1273493117026-2:0:35:2:8847665, originalDestination = null, originalTransactionId = null, producerId = ID:saturntstsgp-40467-127349311702
2010-05-14 18:19:13,950 [EventServiceChannel$ReceiverThread[daemon]_357] WARN failover.FailoverTransport - Transport failed to vm://amqmasterbroker?create=false&waitForStart=10000 , attempting to automatically reconnect due to: java.io.IOException
2010-05-14 18:19:13,951 [EventServiceChannel$ReceiverThread[daemon]_357] DEBUG failover.FailoverTransport - Transport failed with the following exception:
java.io.IOException
at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:45)
at org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:104)
at org.apache.activemq.transport.failover.FailoverTransport.oneway(FailoverTransport.java:471)
at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
at org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1225)
at org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1219)
at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1680)
at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:231)
at org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:269)
at org.springframework.jms.connection.CachedMessageProducer.send(CachedMessageProducer.java:117)
at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:597)
at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:574)
at org.springframework.jms.core.JmsTemplate$3.doInJms(JmsTemplate.java:541)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:471)
at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:539)
at com.scb.saturn.services.messagebroker.MessageBrokerImpl.publishMessage(MessageBrokerImpl.java:118)
at com.scb.saturn.services.messagebroker.MessageBrokerImpl.publishMessage(MessageBrokerImpl.java:102)
at sun.reflect.GeneratedMethodAccessor38.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.springframework.osgi.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:64)
at org.springframework.osgi.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:78)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.osgi.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:36)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.osgi.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:59)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy45.publishMessage(Unknown Source)
at com.scb.saturn.services.apamaclient.scenario.ScenarioService.publishToMessageBroker(ScenarioService.java:362)
at com.scb.saturn.services.apamaclient.scenario.ScenarioService.updateInstance(ScenarioService.java:97)
at com.scb.saturn.services.apamaclient.scenario.ScenarioService.propertyChange(ScenarioService.java:176)
at java.beans.PropertyChangeSupport.firePropertyChange(Unknown Source)
at java.beans.PropertyChangeSupport.firePropertyChange(Unknown Source)
at com.apama.services.scenario.internal.ScenarioDefinition.handle_instanceUpdated(ScenarioDefinition.java:1424)
at com.apama.services.scenario.internal.ScenarioDefinition.access$300(ScenarioDefinition.java:52)
at com.apama.services.scenario.internal.ScenarioDefinition$2.handleEvent(ScenarioDefinition.java:1627)
at com.apama.services.event.internal.EventServiceChannel.processEvent(EventServiceChannel.java:1057)
at com.apama.services.event.internal.EventServiceChannel$ReceiverThread.run(EventServiceChannel.java:1179)
Caused by: java.lang.InterruptedException
at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(Unknown Source)
at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(Unknown Source)
at java.util.concurrent.LinkedBlockingQueue.put(Unknown Source)
at org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:94)
... 44 more
2010-05-14 18:19:13,960 [EventServiceChannel$ReceiverThread[daemon]_357] WARN util.LoggingTransportListener - transport [amqMessageBrokerAmqConnectionTransport] has been interrupted
2010-05-14 18:19:13,961 [ActiveMQ Task] DEBUG failover.FailoverTransport - urlList connectionList:[vm://amqmasterbroker?create=false&waitForStart=10000]
2010-05-14 18:19:13,961 [ActiveMQ Task] DEBUG failover.FailoverTransport - Attempting connect to: vm://amqmasterbroker?create=false&waitForStart=10000
2010-05-14 18:19:13,962 [ActiveMQ Task] DEBUG failover.FailoverTransport - Connection established
2010-05-14 18:19:13,962 [ActiveMQ Task] WARN util.LoggingTransportListener - transport [amqMessageBrokerAmqConnectionTransport] has been resumed
2010-05-14 18:19:14,025 [ActiveMQ Task] INFO failover.FailoverTransport - Successfully reconnected to vm://amqmasterbroker?create=false&waitForStart=10000
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-2734) VMTransport fails with IOException
Posted by "Erich Liebmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erich Liebmann updated AMQ-2734:
--------------------------------
Attachment: VMTransport_IOException.txt
the full stacktrace and associated reconnection logs
> VMTransport fails with IOException
> ----------------------------------
>
> Key: AMQ-2734
> URL: https://issues.apache.org/activemq/browse/AMQ-2734
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.0
> Environment: OS: Linux / 2.6.9-67.ELsmp / i386 / 8
> JVM: Java HotSpot(TM) Server VM / Sun Microsystems Inc. / 1.6.0_17-b04
> Reporter: Erich Liebmann
> Attachments: VMTransport_IOException.txt
>
>
> We are able to successfully publish a large number of messages to an embedded ActiveMQ instance via the VM protocol/transport. After a while, however, further publishing starts to fail with IOExceptions (caused by InterruptedExceptions). For some (unknown) reason the invocation of LinkedBlockingQueue#put() in the VMTransport#oneway() method starts to fail with InterruptedExceptions. Please find the full stack trace below.
> Since we use the VM transport together with the FAILOVER transport the connection is recovered automatically (a fraction of a second later). Our connection URL currently is as follows:
> failover:(vm://amqmasterbroker?create=false&waitForStart=10000)?initialReconnectDelay=100&maxReconnectAttempts=5
> We intend to set the transport parameters backup=true&timeout=100 on the FAILOVER protocol to work around the problem of stuck publishing threads and message publishing errors in case of the IOException (not sure if this will help) but would want see the root-cause to be fixed by ActiveMQ as this issue is causing a lot of troubles for us in production.
> 2010-05-14 18:19:13,950 [EventServiceChannel$ReceiverThread[daemon]_357] DEBUG failover.FailoverTransport - Send oneway attempt: 0 failed for command:ActiveMQObjectMessage {commandId = 1929587556, responseRequired = false, messageId = ID:saturntstsgp-40467-1273493117026-2:0:35:2:8847665, originalDestination = null, originalTransactionId = null, producerId = ID:saturntstsgp-40467-127349311702
> 2010-05-14 18:19:13,950 [EventServiceChannel$ReceiverThread[daemon]_357] WARN failover.FailoverTransport - Transport failed to vm://amqmasterbroker?create=false&waitForStart=10000 , attempting to automatically reconnect due to: java.io.IOException
> 2010-05-14 18:19:13,951 [EventServiceChannel$ReceiverThread[daemon]_357] DEBUG failover.FailoverTransport - Transport failed with the following exception:
> java.io.IOException
> at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:45)
> at org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:104)
> at org.apache.activemq.transport.failover.FailoverTransport.oneway(FailoverTransport.java:471)
> at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
> at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
> at org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1225)
> at org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1219)
> at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1680)
> at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:231)
> at org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:269)
> at org.springframework.jms.connection.CachedMessageProducer.send(CachedMessageProducer.java:117)
> at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:597)
> at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:574)
> at org.springframework.jms.core.JmsTemplate$3.doInJms(JmsTemplate.java:541)
> at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:471)
> at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:539)
> at com.scb.saturn.services.messagebroker.MessageBrokerImpl.publishMessage(MessageBrokerImpl.java:118)
> at com.scb.saturn.services.messagebroker.MessageBrokerImpl.publishMessage(MessageBrokerImpl.java:102)
> at sun.reflect.GeneratedMethodAccessor38.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at org.springframework.osgi.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:64)
> at org.springframework.osgi.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:78)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
> at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
> at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
> at org.springframework.osgi.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:36)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
> at org.springframework.osgi.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:59)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
> at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
> at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
> at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> at $Proxy45.publishMessage(Unknown Source)
> at com.scb.saturn.services.apamaclient.scenario.ScenarioService.publishToMessageBroker(ScenarioService.java:362)
> at com.scb.saturn.services.apamaclient.scenario.ScenarioService.updateInstance(ScenarioService.java:97)
> at com.scb.saturn.services.apamaclient.scenario.ScenarioService.propertyChange(ScenarioService.java:176)
> at java.beans.PropertyChangeSupport.firePropertyChange(Unknown Source)
> at java.beans.PropertyChangeSupport.firePropertyChange(Unknown Source)
> at com.apama.services.scenario.internal.ScenarioDefinition.handle_instanceUpdated(ScenarioDefinition.java:1424)
> at com.apama.services.scenario.internal.ScenarioDefinition.access$300(ScenarioDefinition.java:52)
> at com.apama.services.scenario.internal.ScenarioDefinition$2.handleEvent(ScenarioDefinition.java:1627)
> at com.apama.services.event.internal.EventServiceChannel.processEvent(EventServiceChannel.java:1057)
> at com.apama.services.event.internal.EventServiceChannel$ReceiverThread.run(EventServiceChannel.java:1179)
> Caused by: java.lang.InterruptedException
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(Unknown Source)
> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(Unknown Source)
> at java.util.concurrent.LinkedBlockingQueue.put(Unknown Source)
> at org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:94)
> ... 44 more
> 2010-05-14 18:19:13,960 [EventServiceChannel$ReceiverThread[daemon]_357] WARN util.LoggingTransportListener - transport [amqMessageBrokerAmqConnectionTransport] has been interrupted
> 2010-05-14 18:19:13,961 [ActiveMQ Task] DEBUG failover.FailoverTransport - urlList connectionList:[vm://amqmasterbroker?create=false&waitForStart=10000]
> 2010-05-14 18:19:13,961 [ActiveMQ Task] DEBUG failover.FailoverTransport - Attempting connect to: vm://amqmasterbroker?create=false&waitForStart=10000
> 2010-05-14 18:19:13,962 [ActiveMQ Task] DEBUG failover.FailoverTransport - Connection established
> 2010-05-14 18:19:13,962 [ActiveMQ Task] WARN util.LoggingTransportListener - transport [amqMessageBrokerAmqConnectionTransport] has been resumed
> 2010-05-14 18:19:14,025 [ActiveMQ Task] INFO failover.FailoverTransport - Successfully reconnected to vm://amqmasterbroker?create=false&waitForStart=10000
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (AMQ-2734) VMTransport fails with IOException
Posted by "Erich Liebmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=59436#action_59436 ]
Erich Liebmann commented on AMQ-2734:
-------------------------------------
Forgot to mention that we amended the FailoverTransport#handleTransportFailure() method to capture the full stack trace (could probably have been done with AOP or BTrace as well). Would appreciate if someone could look into this.
> VMTransport fails with IOException
> ----------------------------------
>
> Key: AMQ-2734
> URL: https://issues.apache.org/activemq/browse/AMQ-2734
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.0
> Environment: OS: Linux / 2.6.9-67.ELsmp / i386 / 8
> JVM: Java HotSpot(TM) Server VM / Sun Microsystems Inc. / 1.6.0_17-b04
> Reporter: Erich Liebmann
> Attachments: VMTransport_IOException.txt
>
>
> We are able to successfully publish a large number of messages to an embedded ActiveMQ instance via the VM protocol/transport. After a while, however, further publishing starts to fail with IOExceptions (caused by InterruptedExceptions). For some (unknown) reason the invocation of LinkedBlockingQueue#put() in the VMTransport#oneway() method starts to fail with InterruptedExceptions. Please find the full stack trace below.
> Since we use the VM transport together with the FAILOVER transport the connection is recovered automatically (a fraction of a second later). Our connection URL currently is as follows:
> failover:(vm://amqmasterbroker?create=false&waitForStart=10000)?initialReconnectDelay=100&maxReconnectAttempts=5
> We intend to set the transport parameters backup=true&timeout=100 on the FAILOVER protocol to work around the problem of stuck publishing threads and message publishing errors in case of the IOException (not sure if this will help) but would want see the root-cause to be fixed by ActiveMQ as this issue is causing a lot of troubles for us in production.
> 2010-05-14 18:19:13,950 [EventServiceChannel$ReceiverThread[daemon]_357] DEBUG failover.FailoverTransport - Send oneway attempt: 0 failed for command:ActiveMQObjectMessage {commandId = 1929587556, responseRequired = false, messageId = ID:saturntstsgp-40467-1273493117026-2:0:35:2:8847665, originalDestination = null, originalTransactionId = null, producerId = ID:saturntstsgp-40467-127349311702
> 2010-05-14 18:19:13,950 [EventServiceChannel$ReceiverThread[daemon]_357] WARN failover.FailoverTransport - Transport failed to vm://amqmasterbroker?create=false&waitForStart=10000 , attempting to automatically reconnect due to: java.io.IOException
> 2010-05-14 18:19:13,951 [EventServiceChannel$ReceiverThread[daemon]_357] DEBUG failover.FailoverTransport - Transport failed with the following exception:
> java.io.IOException
> at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:45)
> at org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:104)
> at org.apache.activemq.transport.failover.FailoverTransport.oneway(FailoverTransport.java:471)
> at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
> at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
> at org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1225)
> at org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1219)
> at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1680)
> at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:231)
> at org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:269)
> at org.springframework.jms.connection.CachedMessageProducer.send(CachedMessageProducer.java:117)
> at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:597)
> at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:574)
> at org.springframework.jms.core.JmsTemplate$3.doInJms(JmsTemplate.java:541)
> at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:471)
> at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:539)
> at com.scb.saturn.services.messagebroker.MessageBrokerImpl.publishMessage(MessageBrokerImpl.java:118)
> at com.scb.saturn.services.messagebroker.MessageBrokerImpl.publishMessage(MessageBrokerImpl.java:102)
> at sun.reflect.GeneratedMethodAccessor38.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at org.springframework.osgi.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:64)
> at org.springframework.osgi.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:78)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
> at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
> at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
> at org.springframework.osgi.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:36)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
> at org.springframework.osgi.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:59)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
> at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
> at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
> at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> at $Proxy45.publishMessage(Unknown Source)
> at com.scb.saturn.services.apamaclient.scenario.ScenarioService.publishToMessageBroker(ScenarioService.java:362)
> at com.scb.saturn.services.apamaclient.scenario.ScenarioService.updateInstance(ScenarioService.java:97)
> at com.scb.saturn.services.apamaclient.scenario.ScenarioService.propertyChange(ScenarioService.java:176)
> at java.beans.PropertyChangeSupport.firePropertyChange(Unknown Source)
> at java.beans.PropertyChangeSupport.firePropertyChange(Unknown Source)
> at com.apama.services.scenario.internal.ScenarioDefinition.handle_instanceUpdated(ScenarioDefinition.java:1424)
> at com.apama.services.scenario.internal.ScenarioDefinition.access$300(ScenarioDefinition.java:52)
> at com.apama.services.scenario.internal.ScenarioDefinition$2.handleEvent(ScenarioDefinition.java:1627)
> at com.apama.services.event.internal.EventServiceChannel.processEvent(EventServiceChannel.java:1057)
> at com.apama.services.event.internal.EventServiceChannel$ReceiverThread.run(EventServiceChannel.java:1179)
> Caused by: java.lang.InterruptedException
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(Unknown Source)
> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(Unknown Source)
> at java.util.concurrent.LinkedBlockingQueue.put(Unknown Source)
> at org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:94)
> ... 44 more
> 2010-05-14 18:19:13,960 [EventServiceChannel$ReceiverThread[daemon]_357] WARN util.LoggingTransportListener - transport [amqMessageBrokerAmqConnectionTransport] has been interrupted
> 2010-05-14 18:19:13,961 [ActiveMQ Task] DEBUG failover.FailoverTransport - urlList connectionList:[vm://amqmasterbroker?create=false&waitForStart=10000]
> 2010-05-14 18:19:13,961 [ActiveMQ Task] DEBUG failover.FailoverTransport - Attempting connect to: vm://amqmasterbroker?create=false&waitForStart=10000
> 2010-05-14 18:19:13,962 [ActiveMQ Task] DEBUG failover.FailoverTransport - Connection established
> 2010-05-14 18:19:13,962 [ActiveMQ Task] WARN util.LoggingTransportListener - transport [amqMessageBrokerAmqConnectionTransport] has been resumed
> 2010-05-14 18:19:14,025 [ActiveMQ Task] INFO failover.FailoverTransport - Successfully reconnected to vm://amqmasterbroker?create=false&waitForStart=10000
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-2734) VMTransport fails with IOException
Posted by "Bruce Snyder (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bruce Snyder updated AMQ-2734:
------------------------------
Fix Version/s: 5.5.0
(was: 5.4.1)
> VMTransport fails with IOException
> ----------------------------------
>
> Key: AMQ-2734
> URL: https://issues.apache.org/activemq/browse/AMQ-2734
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.3.0
> Environment: OS: Linux / 2.6.9-67.ELsmp / i386 / 8
> JVM: Java HotSpot(TM) Server VM / Sun Microsystems Inc. / 1.6.0_17-b04
> Reporter: Erich Liebmann
> Fix For: 5.5.0
>
> Attachments: VMTransport_IOException.txt
>
>
> We are able to successfully publish a large number of messages to an embedded ActiveMQ instance via the VM protocol/transport. After a while, however, further publishing starts to fail with IOExceptions (caused by InterruptedExceptions). For some (unknown) reason the invocation of LinkedBlockingQueue#put() in the VMTransport#oneway() method starts to fail with InterruptedExceptions. Please find the full stack trace below.
> Since we use the VM transport together with the FAILOVER transport the connection is recovered automatically (a fraction of a second later). Our connection URL currently is as follows:
> failover:(vm://amqmasterbroker?create=false&waitForStart=10000)?initialReconnectDelay=100&maxReconnectAttempts=5
> We intend to set the transport parameters backup=true&timeout=100 on the FAILOVER protocol to work around the problem of stuck publishing threads and message publishing errors in case of the IOException (not sure if this will help) but would want see the root-cause to be fixed by ActiveMQ as this issue is causing a lot of troubles for us in production.
> 2010-05-14 18:19:13,950 [EventServiceChannel$ReceiverThread[daemon]_357] DEBUG failover.FailoverTransport - Send oneway attempt: 0 failed for command:ActiveMQObjectMessage {commandId = 1929587556, responseRequired = false, messageId = ID:saturntstsgp-40467-1273493117026-2:0:35:2:8847665, originalDestination = null, originalTransactionId = null, producerId = ID:saturntstsgp-40467-127349311702
> 2010-05-14 18:19:13,950 [EventServiceChannel$ReceiverThread[daemon]_357] WARN failover.FailoverTransport - Transport failed to vm://amqmasterbroker?create=false&waitForStart=10000 , attempting to automatically reconnect due to: java.io.IOException
> 2010-05-14 18:19:13,951 [EventServiceChannel$ReceiverThread[daemon]_357] DEBUG failover.FailoverTransport - Transport failed with the following exception:
> java.io.IOException
> at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:45)
> at org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:104)
> at org.apache.activemq.transport.failover.FailoverTransport.oneway(FailoverTransport.java:471)
> at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
> at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:60)
> at org.apache.activemq.ActiveMQConnection.doAsyncSendPacket(ActiveMQConnection.java:1225)
> at org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1219)
> at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1680)
> at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:231)
> at org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:269)
> at org.springframework.jms.connection.CachedMessageProducer.send(CachedMessageProducer.java:117)
> at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:597)
> at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:574)
> at org.springframework.jms.core.JmsTemplate$3.doInJms(JmsTemplate.java:541)
> at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:471)
> at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:539)
> at com.scb.saturn.services.messagebroker.MessageBrokerImpl.publishMessage(MessageBrokerImpl.java:118)
> at com.scb.saturn.services.messagebroker.MessageBrokerImpl.publishMessage(MessageBrokerImpl.java:102)
> at sun.reflect.GeneratedMethodAccessor38.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at org.springframework.osgi.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:64)
> at org.springframework.osgi.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:78)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
> at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
> at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
> at org.springframework.osgi.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:36)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
> at org.springframework.osgi.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:59)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
> at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
> at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
> at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> at $Proxy45.publishMessage(Unknown Source)
> at com.scb.saturn.services.apamaclient.scenario.ScenarioService.publishToMessageBroker(ScenarioService.java:362)
> at com.scb.saturn.services.apamaclient.scenario.ScenarioService.updateInstance(ScenarioService.java:97)
> at com.scb.saturn.services.apamaclient.scenario.ScenarioService.propertyChange(ScenarioService.java:176)
> at java.beans.PropertyChangeSupport.firePropertyChange(Unknown Source)
> at java.beans.PropertyChangeSupport.firePropertyChange(Unknown Source)
> at com.apama.services.scenario.internal.ScenarioDefinition.handle_instanceUpdated(ScenarioDefinition.java:1424)
> at com.apama.services.scenario.internal.ScenarioDefinition.access$300(ScenarioDefinition.java:52)
> at com.apama.services.scenario.internal.ScenarioDefinition$2.handleEvent(ScenarioDefinition.java:1627)
> at com.apama.services.event.internal.EventServiceChannel.processEvent(EventServiceChannel.java:1057)
> at com.apama.services.event.internal.EventServiceChannel$ReceiverThread.run(EventServiceChannel.java:1179)
> Caused by: java.lang.InterruptedException
> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(Unknown Source)
> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(Unknown Source)
> at java.util.concurrent.LinkedBlockingQueue.put(Unknown Source)
> at org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:94)
> ... 44 more
> 2010-05-14 18:19:13,960 [EventServiceChannel$ReceiverThread[daemon]_357] WARN util.LoggingTransportListener - transport [amqMessageBrokerAmqConnectionTransport] has been interrupted
> 2010-05-14 18:19:13,961 [ActiveMQ Task] DEBUG failover.FailoverTransport - urlList connectionList:[vm://amqmasterbroker?create=false&waitForStart=10000]
> 2010-05-14 18:19:13,961 [ActiveMQ Task] DEBUG failover.FailoverTransport - Attempting connect to: vm://amqmasterbroker?create=false&waitForStart=10000
> 2010-05-14 18:19:13,962 [ActiveMQ Task] DEBUG failover.FailoverTransport - Connection established
> 2010-05-14 18:19:13,962 [ActiveMQ Task] WARN util.LoggingTransportListener - transport [amqMessageBrokerAmqConnectionTransport] has been resumed
> 2010-05-14 18:19:14,025 [ActiveMQ Task] INFO failover.FailoverTransport - Successfully reconnected to vm://amqmasterbroker?create=false&waitForStart=10000
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.