You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Jan Lievens (JIRA)" <ji...@apache.org> on 2010/11/30 08:31:11 UTC
[jira] Commented: (AMQ-1352) Shutdown fails if DB connection is
lost
[ https://issues.apache.org/jira/browse/AMQ-1352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12965135#action_12965135 ]
Jan Lievens commented on AMQ-1352:
----------------------------------
I get the same ungraceful shutdown when our PostgreSQL shuts down underneath ActiveMQ. We configured a DataSource connection-pool with a validation SQL statement.
Isn't the ActiveMQ process supposed to be running but not handling incoming or outgoing messages when that happens (at least until the DB goes up again)?
> Shutdown fails if DB connection is lost
> ---------------------------------------
>
> Key: AMQ-1352
> URL: https://issues.apache.org/jira/browse/AMQ-1352
> Project: ActiveMQ
> Issue Type: Bug
> Components: Broker
> Affects Versions: 5.0.0
> Reporter: Mario Siegenthaler
> Fix For: NEEDS_REVIEWED
>
>
> The broker does not shutdown cleanly if the db connection for the JDBC persistence gets disconnected.
> I discovered this issue when implementing the fix for the AMQ-1350. When the broker discovers it doesn't hold the db-lock anymore (because the connection is lost, killed via the db-administration tool) it initiates its shutdown. This shutdown fails most of the time, especially if there is a camel-rule or a webconsole started within the broker.
> The thing is that the broker actually has been shut down (as it's stated and intended), but the other things (jetty, commandAgent, camel etc.) will not be shut down and the java-process does not terminate.
> IMO the broker shutdown didn't even fail, it's just issued an InterruptedException somewhere in order to stop some connector thread. I think this just needs a proper exception handling/ignoring.
> The brokers console output:
> INFO JDBCPersistenceAdapter - No longer able to keep the exclusive lock so giving up being a master
> INFO BrokerService - ActiveMQ Message Broker (marios-master, ID:mario-siegenthalers-computer.local-64808-1185663859419-1:0) is shutting down
> WARN ActiveMQConnection - Async exception with no exception listener: java.io.EOFException
> java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:356)
> at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:269)
> at org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:156)
> at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:136)
> at java.lang.Thread.run(Thread.java:637)
> INFO AdvisoryConsumer - Failed to send remove command: javax.jms.JMSException: The transport is not running.
> javax.jms.JMSException: The transport is not running.
> at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:58)
> at org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1205)
> at org.apache.activemq.AdvisoryConsumer.dispose(AdvisoryConsumer.java:55)
> at org.apache.activemq.ActiveMQConnection.cleanup(ActiveMQConnection.java:1378)
> at org.apache.activemq.ActiveMQConnection.transportFailed(ActiveMQConnection.java:2074)
> at org.apache.activemq.ActiveMQConnection$4.run(ActiveMQConnection.java:1708)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:679)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:704)
> at java.lang.Thread.run(Thread.java:637)
> Caused by: java.io.IOException: The transport is not running.
> at org.apache.activemq.transport.TransportSupport.checkStarted(TransportSupport.java:104)
> at org.apache.activemq.transport.tcp.TcpTransport.oneway(TcpTransport.java:117)
> at org.apache.activemq.transport.InactivityMonitor.oneway(InactivityMonitor.java:147)
> at org.apache.activemq.transport.TransportFilter.oneway(TransportFilter.java:80)
> at org.apache.activemq.transport.WireFormatNegotiator.oneway(WireFormatNegotiator.java:93)
> at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:44)
> at org.apache.activemq.transport.ResponseCorrelator.oneway(ResponseCorrelator.java:56)
> at org.apache.activemq.ActiveMQConnection.asyncSendPacket(ActiveMQConnection.java:1203)
> ... 7 more
> INFO TransportConnector - Connector openwire Stopped
> ERROR efaultMessageListenerContainer - Setup of JMS message listener invoker failed - trying to recover
> javax.jms.IllegalStateException: The Consumer is closed
> at org.apache.activemq.ActiveMQMessageConsumer.checkClosed(ActiveMQMessageConsumer.java:650)
> at org.apache.activemq.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:484)
> at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveMessage(AbstractPollingMessageListenerContainer.java:375)
> at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:300)
> at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:254)
> at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:871)
> at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:818)
> at java.lang.Thread.run(Thread.java:637)
> ERROR ManagedQueueRegion - Could not stop service: QueueRegion: destinations=1, subscriptions=0, memory=0%. Reason: java.lang.InterruptedException
> java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at org.apache.activemq.thread.DedicatedTaskRunner.shutdown(DedicatedTaskRunner.java:73)
> at org.apache.activemq.thread.DedicatedTaskRunner.shutdown(DedicatedTaskRunner.java:83)
> at org.apache.activemq.broker.region.Queue.stop(Queue.java:565)
> at org.apache.activemq.broker.region.AbstractRegion.stop(AbstractRegion.java:94)
> at org.apache.activemq.util.ServiceStopper.stop(ServiceStopper.java:42)
> at org.apache.activemq.broker.region.RegionBroker.doStop(RegionBroker.java:582)
> at org.apache.activemq.broker.jmx.ManagedRegionBroker.doStop(ManagedRegionBroker.java:112)
> at org.apache.activemq.broker.region.RegionBroker.stop(RegionBroker.java:196)
> at org.apache.activemq.broker.TransactionBroker.stop(TransactionBroker.java:115)
> at org.apache.activemq.broker.BrokerFilter.stop(BrokerFilter.java:158)
> at org.apache.activemq.broker.BrokerFilter.stop(BrokerFilter.java:158)
> at org.apache.activemq.broker.MutableBrokerFilter.stop(MutableBrokerFilter.java:168)
> at org.apache.activemq.broker.BrokerService$2.stop(BrokerService.java:1353)
> at org.apache.activemq.util.ServiceStopper.stop(ServiceStopper.java:42)
> at org.apache.activemq.broker.BrokerService.stop(BrokerService.java:471)
> at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.stopBroker(JDBCPersistenceAdapter.java:481)
> at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.databaseLockKeepAlive(JDBCPersistenceAdapter.java:473)
> at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter$3.run(JDBCPersistenceAdapter.java:268)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:301)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:679)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:704)
> at java.lang.Thread.run(Thread.java:637)
> ERROR BrokerService$2 - Could not stop service: org.apache.activemq.broker.BrokerService$2@b59fd9. Reason: java.lang.InterruptedException
> java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at org.apache.activemq.thread.DedicatedTaskRunner.shutdown(DedicatedTaskRunner.java:73)
> at org.apache.activemq.thread.DedicatedTaskRunner.shutdown(DedicatedTaskRunner.java:83)
> at org.apache.activemq.broker.region.Queue.stop(Queue.java:565)
> at org.apache.activemq.broker.region.AbstractRegion.stop(AbstractRegion.java:94)
> at org.apache.activemq.util.ServiceStopper.stop(ServiceStopper.java:42)
> at org.apache.activemq.broker.region.RegionBroker.doStop(RegionBroker.java:582)
> at org.apache.activemq.broker.jmx.ManagedRegionBroker.doStop(ManagedRegionBroker.java:112)
> at org.apache.activemq.broker.region.RegionBroker.stop(RegionBroker.java:196)
> at org.apache.activemq.broker.TransactionBroker.stop(TransactionBroker.java:115)
> at org.apache.activemq.broker.BrokerFilter.stop(BrokerFilter.java:158)
> at org.apache.activemq.broker.BrokerFilter.stop(BrokerFilter.java:158)
> at org.apache.activemq.broker.MutableBrokerFilter.stop(MutableBrokerFilter.java:168)
> at org.apache.activemq.broker.BrokerService$2.stop(BrokerService.java:1353)
> at org.apache.activemq.util.ServiceStopper.stop(ServiceStopper.java:42)
> at org.apache.activemq.broker.BrokerService.stop(BrokerService.java:471)
> at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.stopBroker(JDBCPersistenceAdapter.java:481)
> at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.databaseLockKeepAlive(JDBCPersistenceAdapter.java:473)
> at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter$3.run(JDBCPersistenceAdapter.java:268)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:301)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:679)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:704)
> at java.lang.Thread.run(Thread.java:637)
> INFO BrokerService - ActiveMQ JMS Message Broker (marios-master, ID:mario-siegenthalers-computer.local-64808-1185663859419-1:0) stopped
> WARN JDBCPersistenceAdapter - Failed to stop broker
> java.lang.InterruptedException
> at java.lang.Object.wait(Native Method)
> at org.apache.activemq.thread.DedicatedTaskRunner.shutdown(DedicatedTaskRunner.java:73)
> at org.apache.activemq.thread.DedicatedTaskRunner.shutdown(DedicatedTaskRunner.java:83)
> at org.apache.activemq.broker.region.Queue.stop(Queue.java:565)
> at org.apache.activemq.broker.region.AbstractRegion.stop(AbstractRegion.java:94)
> at org.apache.activemq.util.ServiceStopper.stop(ServiceStopper.java:42)
> at org.apache.activemq.broker.region.RegionBroker.doStop(RegionBroker.java:582)
> at org.apache.activemq.broker.jmx.ManagedRegionBroker.doStop(ManagedRegionBroker.java:112)
> at org.apache.activemq.broker.region.RegionBroker.stop(RegionBroker.java:196)
> at org.apache.activemq.broker.TransactionBroker.stop(TransactionBroker.java:115)
> at org.apache.activemq.broker.BrokerFilter.stop(BrokerFilter.java:158)
> at org.apache.activemq.broker.BrokerFilter.stop(BrokerFilter.java:158)
> at org.apache.activemq.broker.MutableBrokerFilter.stop(MutableBrokerFilter.java:168)
> at org.apache.activemq.broker.BrokerService$2.stop(BrokerService.java:1353)
> at org.apache.activemq.util.ServiceStopper.stop(ServiceStopper.java:42)
> at org.apache.activemq.broker.BrokerService.stop(BrokerService.java:471)
> at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.stopBroker(JDBCPersistenceAdapter.java:481)
> at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter.databaseLockKeepAlive(JDBCPersistenceAdapter.java:473)
> at org.apache.activemq.store.jdbc.JDBCPersistenceAdapter$3.run(JDBCPersistenceAdapter.java:268)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:301)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:679)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:704)
> at java.lang.Thread.run(Thread.java:637)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.