You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Gary Tully (JIRA)" <ji...@apache.org> on 2010/09/16 19:11:44 UTC

[jira] Resolved: (AMQ-2910) NullPointerException in kahadb linked list

     [ https://issues.apache.org/activemq/browse/AMQ-2910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gary Tully resolved AMQ-2910.
-----------------------------

         Assignee: Gary Tully
    Fix Version/s: 5.4.1
       Resolution: Fixed

resolve in r997849

would  be great if you could validate tonight's 5.5-SNAPSHOT or build trunk yourself to validate.

> NullPointerException in kahadb linked list
> ------------------------------------------
>
>                 Key: AMQ-2910
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2910
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Message Store
>    Affects Versions: 5.4.0
>         Environment: java version "1.6.0_20"
> Java(TM) SE Runtime Environment (build 1.6.0_20-b02)
> Clients connection to the broker use Active MQ 5.3.0 or 5.3.2.
>            Reporter: Gerd Behrmann
>            Assignee: Gary Tully
>            Priority: Critical
>             Fix For: 5.4.1
>
>         Attachments: AMQ2910Test.java
>
>
> We got the following NullPointerException in kahadb.
> 13 Sep 2010 05:53:30 (JMS) [] Caught an Exception adding a message: ActiveMQObjectMessage {commandId = 111839609, responseRequired = false, messageId = ID:statler.ndgf.org-42286-1282568899957-0:0:2:1:43748846, originalDestination = null, originalTransactionId = null, producerId = ID:statler.ndgf.org-42286-1282568899957-0:0:2:1, destination = queue://cells.domain.dCacheDomain, transactionId = null, expiration = 0, timestamp = 0, arrival = 0, brokerInTime = 1284350009950, brokerOutTime = 0, correlationId = null, replyTo = null, persistent = false, type = null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null, compressed = false, userID = null, content = org.apache.activemq.util.ByteSequence@1bada5d, marshalledProperties = null, dataStructure = null, redeliveryCounter = 0, size = 0, properties = null, readOnlyProperties = false, readOnlyBody = false, droppable = false} first to FilePendingMessageCursor 
> java.lang.NullPointerException: null
>         at org.apache.kahadb.util.LinkedNode.unlink(LinkedNode.java:231) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.util.SequenceSet.removeFirstSequence(SequenceSet.java:203) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.PageFile.allocate(PageFile.java:753) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.allocate(Transaction.java:137) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.allocate(Transaction.java:121) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.store.kahadb.plist.PList.createEntry(PList.java:425) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:170) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.store.kahadb.plist.PList$2.execute(PList.java:163) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.execute(Transaction.java:728) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:161) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.tryAddMessageLast(FilePendingMessageCursor.java:221) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.addMessageLast(FilePendingMessageCursor.java:192) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.cursors.StoreQueueCursor.addMessageLast(StoreQueueCursor.java:96) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.Queue.sendMessage(Queue.java:1601) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:707) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.Queue.send(Queue.java:646) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:365) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:494) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.scheduler.SchedulerBroker.send(SchedulerBroker.java:136) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:230) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:135) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:460) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:663) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:309) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:185) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:217) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:219) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:201) [activemq-all-5.4.0.jar:5.4.0]
>         at java.lang.Thread.run(Thread.java:619) [na:1.6.0_20]
> 13 Sep 2010 05:53:30 (JMS) [] Async error occurred: java.lang.RuntimeException: java.lang.NullPointerException
> java.lang.RuntimeException: java.lang.NullPointerException
>         at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.tryAddMessageLast(FilePendingMessageCursor.java:228) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.addMessageLast(FilePendingMessageCursor.java:192) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.cursors.StoreQueueCursor.addMessageLast(StoreQueueCursor.java:96) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.Queue.sendMessage(Queue.java:1601) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:707) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.Queue.send(Queue.java:646) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:365) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:494) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.scheduler.SchedulerBroker.send(SchedulerBroker.java:136) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:230) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:135) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:460) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:663) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:309) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:185) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:217) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:219) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:201) [activemq-all-5.4.0.jar:5.4.0]
>         at java.lang.Thread.run(Thread.java:619) [na:1.6.0_20]
> Caused by: java.lang.NullPointerException: null
>         at org.apache.kahadb.util.LinkedNode.unlink(LinkedNode.java:231) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.util.SequenceSet.removeFirstSequence(SequenceSet.java:203) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.PageFile.allocate(PageFile.java:753) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.allocate(Transaction.java:137) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.allocate(Transaction.java:121) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.store.kahadb.plist.PList.createEntry(PList.java:425) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:170) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.store.kahadb.plist.PList$2.execute(PList.java:163) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.execute(Transaction.java:728) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:161) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.tryAddMessageLast(FilePendingMessageCursor.java:221) ~[activemq-all-5.4.0.jar:5.4.0]
>         ... 24 common frames omitted
> Looking at the code, it appears to me that the only way an NPE can be thrown at that place is if list is null. Since the same method however tests for that exact case, I suspect there is unsynchronized concurrent access to the LinkedNode.
> A few hours later we got more failures:
> 13 Sep 2010 08:38:19 (JMS) [] Caught an Exception adding a message: ActiveMQObjectMessage {commandId = 29114496, responseRequired = false, messageId = ID:arctic01.csc.fi-39780-1280232016428-0:0:2:1:333447, originalDestination = null, originalTransactionId = null, producerId = ID:arctic01.csc.fi-39780-1280232016428-0:0:2:1, destination = queue://cells.domain.dCacheDomain, transactionId = null, expiration = 0, timestamp = 0, arrival = 0, brokerInTime = 1284359899636, brokerOutTime = 0, correlationId = null, replyTo = null, persistent = false, type = null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null, compressed = false, userID = null, content = org.apache.activemq.util.ByteSequence@6f28b428, marshalledProperties = null, dataStructure = null, redeliveryCounter = 0, size = 0, properties = null, readOnlyProperties = false, readOnlyBody = false, droppable = false} first to FilePendingMessageCursor 
> java.lang.NullPointerException: null
>         at org.apache.kahadb.util.LinkedNode.isHeadNode(LinkedNode.java:64) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.util.LinkedNode.unlink(LinkedNode.java:230) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.util.SequenceSet.removeFirstSequence(SequenceSet.java:203) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.PageFile.allocate(PageFile.java:753) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.allocate(Transaction.java:137) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.allocate(Transaction.java:121) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.store.kahadb.plist.PList.createEntry(PList.java:425) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:170) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.store.kahadb.plist.PList$2.execute(PList.java:163) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.execute(Transaction.java:728) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:161) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.tryAddMessageLast(FilePendingMessageCursor.java:221) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.addMessageLast(FilePendingMessageCursor.java:192) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.cursors.StoreQueueCursor.addMessageLast(StoreQueueCursor.java:96) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.Queue.sendMessage(Queue.java:1601) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:707) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.Queue.send(Queue.java:646) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:365) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:494) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.scheduler.SchedulerBroker.send(SchedulerBroker.java:136) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:230) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:135) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:460) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:663) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:309) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:185) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:217) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:219) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:201) [activemq-all-5.4.0.jar:5.4.0]
>         at java.lang.Thread.run(Thread.java:619) [na:1.6.0_20]
> 13 Sep 2010 08:38:19 (JMS) [] Async error occurred: java.lang.RuntimeException: java.lang.NullPointerException
> java.lang.RuntimeException: java.lang.NullPointerException
>         at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.tryAddMessageLast(FilePendingMessageCursor.java:228) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.addMessageLast(FilePendingMessageCursor.java:192) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.cursors.StoreQueueCursor.addMessageLast(StoreQueueCursor.java:96) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.Queue.sendMessage(Queue.java:1601) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:707) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.Queue.send(Queue.java:646) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:365) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:494) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.scheduler.SchedulerBroker.send(SchedulerBroker.java:136) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:230) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:135) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:460) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:663) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:309) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:185) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.TransportFilter.onCommand(TransportFilter.java:69) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.InactivityMonitor.onCommand(InactivityMonitor.java:217) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:219) [activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:201) [activemq-all-5.4.0.jar:5.4.0]
>         at java.lang.Thread.run(Thread.java:619) [na:1.6.0_20]
> Caused by: java.lang.NullPointerException: null
>         at org.apache.kahadb.util.LinkedNode.isHeadNode(LinkedNode.java:64) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.util.LinkedNode.unlink(LinkedNode.java:230) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.util.SequenceSet.removeFirstSequence(SequenceSet.java:203) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.PageFile.allocate(PageFile.java:753) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.allocate(Transaction.java:137) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.allocate(Transaction.java:121) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.store.kahadb.plist.PList.createEntry(PList.java:425) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:170) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.store.kahadb.plist.PList$2.execute(PList.java:163) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.kahadb.page.Transaction.execute(Transaction.java:728) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.store.kahadb.plist.PList.addLast(PList.java:161) ~[activemq-all-5.4.0.jar:5.4.0]
>         at org.apache.activemq.broker.region.cursors.FilePendingMessageCursor.tryAddMessageLast(FilePendingMessageCursor.java:221) ~[activemq-all-5.4.0.jar:5.4.0]
>         ... 24 common frames omitted
> We got a number of those exceptions with the same stack trace and then followed by tons of other NullPointerExceptions, likely because the list was corrupted at that point.
> Not long after these stack traces message delivery began to hang. A restart of the broker only helped temporarily as message delivery would hang again after half a minute or so. To our big suprise moving the db directory to let ActiveMQ create a new database didn't solve the problem either. I am unable to tell to what extend that was caused by the above exceptions or not.
> We are happy to provide any other information you may need to debug this problem.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.