You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Joseph Chang (JIRA)" <ji...@apache.org> on 2012/12/28 20:58:12 UTC
[jira] [Created] (AMQ-4239) Possible Race Condition in
FilePendingMessageCursor when sending messages quickly
Joseph Chang created AMQ-4239:
---------------------------------
Summary: Possible Race Condition in FilePendingMessageCursor when sending messages quickly
Key: AMQ-4239
URL: https://issues.apache.org/jira/browse/AMQ-4239
Project: ActiveMQ
Issue Type: Bug
Components: Broker
Affects Versions: 5.7.0
Environment: OS : CentOS release 6.3 (Final)
KERNEL : 2.6.32-279.el6.x86_64
CPU : Intel(R) Xeon(R) CPU X3470 @ 2.93GHz
CPU COUNT : 8
CPU TYPE : x86_64
RAM : 32 GB
Manufacturer : Dell Inc.
Model : PowerEdge R310
BIOS Version : 1.8.2
Reporter: Joseph Chang
When sending persistent messages extremely quickly, we often see the following stack trace.
Broker Persistence is turned on, but not using KahaDB. Instead, using JDBC Persistence (MySQL). KahaDB is commented out on the xml configuration file.
Reading through source, seems like the Kaha engine is still used for temporary storage of pending messages (/data/**/tmp_storage). However, in reading through code, seems like the "loaded" member variable is not intialized in time, causing this stack to be thrown.
Any help appreciated, simply gums up our logging, and the exceptions make it back to the client.
[04 Dec 2012 08:02:08] [scheduler_Worker-2] 36391186 ERROR
(com.ms.jms.publisher.BaseMessagePublisher:143) -
java.lang.IllegalStateException: TheListIndex is not loaded
javax.jms.JMSException: java.lang.IllegalStateException: TheListIndex
is not loaded
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49)
at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1377)
at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1305)
at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1795)
at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:277)
at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:212)
at org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:269)
at org.springframework.jms.connection.CachedMessageProducer.send(CachedMessageProducer.java:121)
at com.ms.jms.publisher.BaseMessagePublisher.talker(BaseMessagePublisher.java:133)
at com.ms.jms.publisher.BaseMessagePublisher.sendMessage(BaseMessagePublisher.java:63)
at com.ms.mscm.bus.ClientBasedMessagePublisher.sendMessage(ClientBasedMessagePublisher.java:169)
at com.ms.cart.ScheduledCartProcessor.checkout(ScheduledCartProcessor.java:101)
at com.ms.cart.ScheduledCartJob.doExecuteInternal(ScheduledCartJob.java:63)
at com.ms.cart.ScheduledCartJob.executeInternal(ScheduledCartJob.java:50)
at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:86)
at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
Caused by: java.lang.RuntimeException:
java.lang.IllegalStateException: TheListIndex is not loaded
at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.tryAddMessageLast(FilePendingMessageCursor.java:239)
at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.addMessageLast(FilePendingMessageCursor.java:202)
at org.apache.activemq.broker.region.Queue.sendMessage(Queue.java:1669)
at org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:785)
at org.apache.activemq.broker.region.Queue.send(Queue.java:717)
at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:407)
at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:503)
at org.apache.activemq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:311)
at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129)
at org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96)
at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:317)
at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129)
at org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:135)
at org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:450)
at org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:680)
at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:294)
at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:152)
at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:256)
at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
at org.apache.activemq.transport.nio.NIOTransport.serviceRead(NIOTransport.java:138)
at org.apache.activemq.transport.nio.NIOTransport$1.onSelect(NIOTransport.java:69)
at org.apache.activemq.transport.nio.SelectorSelection.onSelect(SelectorSelection.java:94)
at org.apache.activemq.transport.nio.SelectorWorker$1.run(SelectorWorker.java:119)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.IllegalStateException: TheListIndex is not loaded
at org.apache.kahadb.index.ListIndex.assertLoaded(ListIndex.java:301)
at org.apache.kahadb.index.ListIndex.add(ListIndex.java:199)
at org.apache.activemq.store.kahadb.plist.PList$2.execute(PList.java:84)
at org.apache.kahadb.page.Transaction.execute(Transaction.java:769)
at org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:82)
at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.flushToDisk(FilePendingMessageCursor.java:438)
at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.tryAddMessageLast(FilePendingMessageCursor.java:226)
... 27 more
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira